Character processing method and device

Abstract

The invention relates to a character processing method and device, and the method comprises the steps: obtaining a character, rendering the character, and obtaining a character picture; and carrying out classification processing on the character picture to obtain a state corresponding to the character, and processing the character according to the state. Therefore, the states of the characters are recognized, corresponding processing is carried out according to the states, and the correctness of the characters is guaranteed. In the scene of converting the format electronic document into the streaming electronic document, the problems of messy codes or incomplete display and the like in the streaming electronic document converted from the format electronic document are avoided, the conversion effect is improved, and the influence on user reading is avoided.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a character processing method and apparatus. Background Art

[0002] Currently, a font, that is, a character typeface (such as a writing style), or a font style, is a collection of glyphs with the same appearance style (style) and the same typesetting size (character width, character height, and line spacing). Users can set corresponding fonts in corresponding applications according to their own needs.

[0003] In a conventional application, the characters (such as "one") in the default font file (such as the regular script font) do not have state anomalies. However, in reality, it is possible to have state anomalies, which leads to problems such as text garbled characters or incomplete display in the scenario of converting a layout electronic document into a flowable electronic document, reducing the conversion effect and affecting user reading. Summary of the Invention

[0004] In order to solve the technical problem that in the above conventional application, the characters in the default font file do not have state anomalies, but in reality, it is possible to have state anomalies, which leads to problems such as text garbled characters or incomplete display in the scenario of converting a layout electronic document into a flowable electronic document, reducing the conversion effect and affecting user reading, embodiments of this application provide a character processing method, apparatus, electronic device, and storage medium. The specific technical solutions are as follows:

[0005] In the first aspect of the embodiments of this application, first, a character processing method is provided. The method includes:

[0006] Obtain a character, render the character, and obtain a character image;

[0007] Perform classification processing on the character image to obtain the state corresponding to the character, and process the character according to the state.

[0008] In an optional embodiment, the obtaining a character, rendering the character, and obtaining a character image includes:

[0009] Obtain at least one font file. For any one of the font files, store each character in the font file into a corresponding text object, and form a text object set from the text objects;

[0010] Traverse the text objects in any one of the text object sets, and render the characters included in the text objects to obtain character images.

[0011] In an optional embodiment, traversing the text objects in any of the text object sets and rendering the characters contained in the text objects to obtain character images includes:

[0012] For any of the text object sets, determining whether the number of elements of the text objects in the text object set exceeds a preset number threshold;

[0013] When the number of elements of the text objects in the text object set does not exceed the preset number threshold, performing the following processing:

[0014] The text objects in the text object set are traversed, and the characters contained in the text objects are rendered to obtain character images.

[0015] In an optional embodiment, the method further comprises:

[0016] When the number of elements of the text objects in the text object set exceeds the preset number threshold, the following processing is performed:

[0017] Sorting the text objects in the text object set, and extracting the text objects with the preset number threshold from the sorted text objects;

[0018] The text objects of the preset number threshold are traversed, and the characters contained in the text objects are rendered to obtain character images.

[0019] In an optional embodiment, the classifying the character image to obtain the status corresponding to the character includes:

[0020] Inputting the character image into a pre-trained classification model to obtain probability values ​​of each state corresponding to the character image;

[0021] The state corresponding to the maximum probability value is selected, and the state corresponding to the maximum probability value is determined as the state corresponding to the character.

[0022] In an optional implementation, processing the character according to the state includes:

[0023] determining whether the state is a normal state, and if the state is the normal state, adding a preset value to the first occurrence number corresponding to the normal state;

[0024] In the case where the state is an abnormal state, adding a preset value to the second number of occurrences corresponding to the abnormal state;

[0025] After the traversal of the text object is completed, determining whether the second number of occurrences exceeds the target number of occurrences;

[0026] The target occurrence count is the product of the traversal count of the text object and a preset value;

[0027] In a case where the second number of occurrences exceeds the target number of occurrences, determining the character corresponding to the abnormal state;

[0028] According to the abnormal state, the character corresponding to the abnormal state is inversely transformed to obtain the character in the normal state.

[0029] In an optional implementation, determining whether the state is a normal state includes:

[0030] Obtaining a preset confusion table, wherein the preset confusion table stores confusion pairs, each of which consists of a sample character and a state;

[0031] In the case where there is no matching confusion pair between the character and the state corresponding to the character, determining whether the state is a normal state;

[0032] In the case where a matching confusion pair exists between the character and the state corresponding to the character, the next text object is traversed until the traversal is completed.

[0033] In an optional embodiment, before executing the method, the method further includes:

[0034] Acquire a sample font file, and filter out a target sample font file from the sample font file, wherein the sample characters in the target sample font file cover a preset character set;

[0035] generating target sample characters according to the sample characters in the target sample font file, wherein the sample characters are consistent with the characters in the preset character set;

[0036] Rendering the target sample character to obtain a sample character image, and storing the sample character image into an image set;

[0037] Determine a feature matrix corresponding to any of the sample character images in the image set, and generate a similarity matrix based on the feature matrix;

[0038] According to the similarity matrix, confusion pairs are determined, and the confusion pairs are stored in a preset confusion table.

[0039] In an optional embodiment, generating target sample characters according to the sample characters in the target sample font file includes:

[0040] transforming the sample characters in the normal state in the target sample font file to obtain the sample characters in the abnormal state;

[0041] The sample characters in the normal state and the sample characters in the abnormal state are determined as target sample characters.

[0042] In an optional embodiment, determining a feature matrix corresponding to any of the sample character images in the image set includes:

[0043] For any of the sample character pictures in the picture set, input the sample character picture into a pre-trained classification model;

[0044] The feature matrix corresponding to the sample character image output by the backbone network in the pre-trained classification model is intercepted.

[0045] In an optional embodiment, the backbone network includes M feature extraction networks, each of which includes N convolutional layers and pooling layers, where M and N are both positive integers;

[0046] The intercepting of the feature matrix corresponding to the sample character image output by the backbone network in the pre-trained classification model includes:

[0047] Iterate and perform the following operations until the Mth feature matrix is ​​obtained, then stop the following operations and intercept the Mth feature matrix:

[0048] Inputting the i-th object to be extracted into the i-th feature extraction network, performing convolution processing on the i-th object to be extracted through the N convolution layers in the i-th feature extraction network to obtain a convolution result, and performing pooling processing on the convolution result through the pooling layer in the i-th feature extraction network to obtain an i-th feature matrix;

[0049] Among them, the i-th feature matrix is ​​the i+1-th object to be extracted, the 1st object to be extracted is the sample character image, the M-th feature matrix is ​​determined as the feature matrix corresponding to the sample character image, and i is 1, 2, 3, ..., M in sequence.

[0050] In an optional implementation, generating a similarity matrix based on the feature matrix includes:

[0051] Sort all the sample character pictures in the picture set to obtain a sample character picture sorting sequence;

[0052] The following processing is performed on a first sample character picture, where the first sample character picture is any sample character picture in the sample character picture sorting sequence:

[0053] Determining a similarity between the feature matrix of the first sample character image and the feature matrix of the second sample character image;

[0054] Wherein, the second sample character picture is any sample character picture in the sample character picture sorting sequence;

[0055] determining a first sorting order of the first sample character images and a second sorting order of the second sample character images;

[0056] The matrix positions of the similarities are formed according to the first sorting order and the second sorting order, and a similarity matrix is ​​formed by the similarities and the matrix positions.

[0057] In an optional implementation, determining confusion pairs according to the similarity matrix includes:

[0058] For the similarity in the i-th row and j-th column of the similarity matrix, determine whether the similarity is greater than a preset similarity threshold, where i and j are both positive integers;

[0059] When the similarity is greater than the similarity threshold, the following process is performed to determine the confusion pair:

[0060] Determine the feature matrix corresponding to the i-th row, determine the sample character image corresponding to the feature matrix; determine the target sample character corresponding to the sample character image, determine the sample character corresponding to the target sample character; determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state;

[0061] Determine the feature matrix corresponding to the j-th column, determine the sample character image corresponding to the feature matrix; determine the target sample character corresponding to the sample character image, determine the sample character corresponding to the target sample character; determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state.

[0062] In an optional embodiment, when the state is an abnormal state, adding a preset value to the second number of occurrences corresponding to the abnormal state includes:

[0063] In the case where the state is an abnormal state, performing an inverse transformation on the character according to the abnormal state to obtain a character in a normal state;

[0064] Render the characters in normal state to obtain a character verification image;

[0065] Inputting the character verification image into a pre-trained classification model to obtain probability values ​​of various states corresponding to the character verification image;

[0066] The state corresponding to the maximum probability value is selected, and when the state corresponding to the maximum probability value is a normal state, a preset value is added to the second number of occurrences corresponding to the abnormal state.

[0067] In an optional embodiment, before executing the method, the method further includes:

[0068] Obtaining a sample font file, rendering sample characters in the sample font file, and obtaining a training character image;

[0069] According to the abnormal state, a preset proportion of the training character images are transformed, and the transformed training character images and the remaining training character images are determined as training images;

[0070] Determine a sample label corresponding to the training image, where the sample label is an initial probability value of each state corresponding to the training image;

[0071] Input the training image into the classification model to obtain the sample probability value of each state corresponding to the training image;

[0072] Determining a loss value based on the sample probability value and the initial probability value, and training the classification model based on the loss value;

[0073] When the loss value is less than a preset loss value threshold, the training is stopped to obtain the pre-trained classification model.

[0074] In an optional embodiment, determining the transformed training character pictures and the remaining training character pictures as training pictures includes:

[0075] Obtaining a preset training data set, wherein the images in the preset training data set are used for optical character recognition model training;

[0076] Using a pre-trained optical character recognition model, identifying a single character region in the image, and segmenting the single character region to obtain a single character image;

[0077] The transformed training character pictures, the remaining training character pictures, and the single-character pictures are determined as the training pictures.

[0078] In a second aspect of the embodiments of the present application, a character processing device is further provided, the device comprising:

[0079] A character rendering module is used to obtain characters, render the characters, and obtain character images;

[0080] A state recognition module is used to classify the character image to obtain the state corresponding to the character;

[0081] The character processing module is used to process the character according to the state.

[0082] In an optional embodiment, the character rendering module specifically includes:

[0083] A font file acquisition submodule is used to acquire at least one font file;

[0084] A set composition submodule, for storing each character in any of the font files into a corresponding text object, and forming a text object set from the text objects;

[0085] The character rendering submodule is used to traverse the text objects in any of the text object sets, render the characters contained in the text objects, and obtain character images.

[0086] In an optional embodiment, the character rendering submodule is specifically configured to:

[0087] For any of the text object sets, determining whether the number of elements of the text objects in the text object set exceeds a preset number threshold;

[0088] When the number of elements of the text objects in the text object set does not exceed the preset number threshold, performing the following processing:

[0089] The text objects in the text object set are traversed, and the characters contained in the text objects are rendered to obtain character images.

[0090] In an optional embodiment, the character rendering submodule is further configured to:

[0091] When the number of elements of the text objects in the text object set exceeds the preset number threshold, the following processing is performed:

[0092] Sorting the text objects in the text object set, and extracting the text objects with the preset number threshold from the sorted text objects;

[0093] The text objects of the preset number threshold are traversed, and the characters contained in the text objects are rendered to obtain character images.

[0094] In an optional embodiment, the state identification module is specifically configured to:

[0095] Inputting the character image into a pre-trained classification model to obtain probability values ​​of each state corresponding to the character image;

[0096] The state corresponding to the maximum probability value is selected, and the state corresponding to the maximum probability value is determined as the state corresponding to the character.

[0097] In an optional embodiment, the character processing module specifically includes:

[0098] A state judgment submodule is used to judge whether the state is normal;

[0099] a first number counting submodule, configured to, when the state is the normal state, add a preset value to the first number of occurrences corresponding to the normal state;

[0100] A second number counting submodule is configured to add a preset value to a second number of occurrences corresponding to the abnormal state when the state is an abnormal state;

[0101] A number judgment submodule is used to judge whether the second number of occurrences exceeds the target number of occurrences after the traversal of the text object is completed;

[0102] The target occurrence count is the product of the traversal count of the text object and a preset value;

[0103] a character determination submodule, configured to determine the character corresponding to the abnormal state when the second number of occurrences exceeds the target number of occurrences;

[0104] The character inverse transformation module is used to perform an inverse transformation on the character corresponding to the abnormal state according to the abnormal state to obtain the character in the normal state.

[0105] In an optional embodiment, the state judgment submodule is specifically configured to:

[0106] Obtaining a preset confusion table, wherein the preset confusion table stores confusion pairs, each of which consists of a sample character and a state;

[0107] In the case where there is no matching confusion pair between the character and the state corresponding to the character, determining whether the state is a normal state;

[0108] In the case where a matching confusion pair exists between the character and the state corresponding to the character, the next text object is traversed until the traversal is completed.

[0109] In an optional embodiment, the device further comprises:

[0110] a font file screening module, configured to obtain sample font files and screen out a target sample font file from the sample font files, wherein the sample characters in the target sample font file cover a preset character set;

[0111] a character generation module, configured to generate target sample characters based on sample characters in the target sample font file, wherein the sample characters are consistent with characters in the preset character set;

[0112] A picture storage module, configured to render the target sample character to obtain a sample character picture, and store the sample character picture into a picture set;

[0113] a matrix determination module, configured to determine a feature matrix corresponding to any of the sample character images in the image set;

[0114] A matrix generation module, configured to generate a similarity matrix based on the feature matrix;

[0115] a confusion pair determination module, configured to determine confusion pairs according to the similarity matrix;

[0116] The confusion pair storing module is used to store the confusion pair into a preset confusion table.

[0117] In an optional embodiment, the character generation module is specifically configured to:

[0118] transforming the sample characters in the normal state in the target sample font file to obtain the sample characters in the abnormal state;

[0119] The sample characters in the normal state and the sample characters in the abnormal state are determined as target sample characters.

[0120] In an optional implementation, the matrix determination module specifically includes:

[0121] A picture input submodule, configured to input any of the sample character pictures in the picture set into a pre-trained classification model;

[0122] The feature matrix interception submodule is used to intercept the feature matrix corresponding to the sample character image output by the backbone network in the pre-trained classification model.

[0123] In an optional embodiment, the backbone network includes M feature extraction networks, each of which includes N convolutional layers and pooling layers, where M and N are both positive integers;

[0124] The feature matrix interception submodule is specifically used for:

[0125] Iterate and perform the following operations until the Mth feature matrix is ​​obtained, then stop the following operations and intercept the Mth feature matrix:

[0126] Inputting the i-th object to be extracted into the i-th feature extraction network, performing convolution processing on the i-th object to be extracted through the N convolution layers in the i-th feature extraction network to obtain a convolution result, and performing pooling processing on the convolution result through the pooling layer in the i-th feature extraction network to obtain an i-th feature matrix;

[0127] Among them, the i-th feature matrix is ​​the i+1-th object to be extracted, the 1st object to be extracted is the sample character image, the M-th feature matrix is ​​determined as the feature matrix corresponding to the sample character image, and i is 1, 2, 3, ..., M in sequence.

[0128] In an optional embodiment, the matrix generation module is specifically configured to:

[0129] Sort all the sample character pictures in the picture set to obtain a sample character picture sorting sequence;

[0130] The following processing is performed on a first sample character picture, where the first sample character picture is any sample character picture in the sample character picture sorting sequence:

[0131] Determining a similarity between the feature matrix of the first sample character image and the feature matrix of the second sample character image;

[0132] Wherein, the second sample character picture is any sample character picture in the sample character picture sorting sequence;

[0133] determining a first sorting order of the first sample character images and a second sorting order of the second sample character images;

[0134] The matrix positions of the similarities are formed according to the first sorting order and the second sorting order, and a similarity matrix is ​​formed by the similarities and the matrix positions.

[0135] In an optional embodiment, the confusion pair determination module is specifically configured to:

[0136] For the similarity in the i-th row and j-th column of the similarity matrix, determine whether the similarity is greater than a preset similarity threshold, where i and j are both positive integers;

[0137] When the similarity is greater than the similarity threshold, the following process is performed to determine the confusion pair:

[0138] Determine the feature matrix corresponding to the i-th row, determine the sample character image corresponding to the feature matrix; determine the target sample character corresponding to the sample character image, determine the sample character corresponding to the target sample character; determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state;

[0139] Determine the feature matrix corresponding to the j-th column, determine the sample character image corresponding to the feature matrix; determine the target sample character corresponding to the sample character image, determine the sample character corresponding to the target sample character; determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state.

[0140] In an optional embodiment, the second number statistics submodule is specifically configured to:

[0141] In the case where the state is an abnormal state, performing an inverse transformation on the character according to the abnormal state to obtain a character in a normal state;

[0142] Render the characters in normal state to obtain a character verification image;

[0143] Inputting the character verification image into a pre-trained classification model to obtain probability values ​​of various states corresponding to the character verification image;

[0144] The state corresponding to the maximum probability value is selected, and when the state corresponding to the maximum probability value is a normal state, a preset value is added to the second number of occurrences corresponding to the abnormal state.

[0145] In an optional embodiment, the device further comprises:

[0146] A sample character rendering module is used to obtain a sample font file, render the sample characters in the sample font file, and obtain a training character image;

[0147] An image transformation module, configured to transform the training character image of a preset proportion according to an abnormal state;

[0148] A training picture determination module, configured to determine the transformed training character picture and the remaining training character pictures as training pictures;

[0149] a label determination module, configured to determine a sample label corresponding to the training image, wherein the sample label is an initial probability value of each state corresponding to the training image;

[0150] A training picture input module is used to input the training picture into the classification model to obtain the sample probability value of each state corresponding to the training picture;

[0151] A model training module, configured to determine a loss value based on the sample probability value and the initial probability value, and train the classification model based on the loss value;

[0152] The model training stop module is used to stop training when the loss value is less than a preset loss value threshold to obtain the pre-trained classification model.

[0153] In an optional embodiment, the training image determination module is specifically configured to:

[0154] The step of determining the transformed training character picture and the remaining training character pictures as training pictures includes:

[0155] Obtaining a preset training data set, wherein the images in the preset training data set are used for optical character recognition model training;

[0156] Using a pre-trained optical character recognition model, identifying a single character region in the image, and segmenting the single character region to obtain a single character image;

[0157] The transformed training character pictures, the remaining training character pictures, and the single-character pictures are determined as the training pictures.

[0158] In a third aspect of the embodiments of the present application, an electronic device is further provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0159] Memory for storing computer programs;

[0160] The processor is configured to implement any character processing method described in the first aspect when executing a program stored in the memory.

[0161] In a fourth aspect of the embodiments of the present application, a storage medium is further provided, wherein instructions are stored in the storage medium, and when the storage medium is run on a computer, the computer executes any character processing method described in the first aspect above.

[0162] In a fifth aspect of the embodiments of the present application, a computer program product comprising instructions is further provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned character processing methods.

[0163] The technical solution provided by the embodiment of the present application obtains characters, renders the characters, obtains character images, classifies the character images, obtains the status corresponding to the characters, and processes the characters according to the status.

[0164] By rendering characters to generate character images, classifying these character images to determine their corresponding states, and processing the characters accordingly, the system identifies the character states and performs appropriate processing, ensuring character accuracy. In the conversion of formatted electronic documents to streaming electronic documents, the resulting streaming electronic documents avoid issues such as garbled text or incomplete display, improving conversion quality and minimizing impact on user reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0165] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0166] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0167] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0168] Figure 1 A schematic diagram of an implementation flow of a character processing method shown in an embodiment of the present application;

[0169] Figure 2 This is a schematic diagram of an implementation flow of another character processing method shown in an embodiment of the present application;

[0170] Figure 3 A schematic diagram of an implementation flow of a model training method shown in an embodiment of the present application;

[0171] Figure 4 A schematic diagram of a training character image shown in an embodiment of the present application;

[0172] Figure 5 Schematic diagram of an implementation process of a method for constructing a confusion table shown in an embodiment of the present application;

[0173] Figure 6 A schematic diagram of a sample character image shown in an embodiment of the present application;

[0174] Figure 7 This is a structural diagram of a character processing device shown in an embodiment of the present application;

[0175] Figure 8 This is a schematic structural diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0176] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0177] The disclosure below provides many different embodiments or examples for implementing different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described below. Of course, these are merely examples and are not intended to limit the present application. In addition, the present application may repeat reference numbers and / or letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0178] like Figure 1 FIG. 1 is a flow chart of an implementation of a character processing method provided by an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0179] S101, obtaining characters, rendering the characters, and obtaining character images.

[0180] S102, classify the character image to obtain the status corresponding to the character, and process the character according to the status.

[0181] In an embodiment of the present application, a character is obtained, where the character may be a character in a font file, and then the character is rendered to obtain a character image (the character image here is generally a single character image), and the character image is classified and processed to obtain the state corresponding to the character, so that the character can be processed according to the state. Here, the processing of the character varies depending on the different states of the character.

[0182] It should be noted that the status corresponding to the characters can generally be divided into two categories: non-abnormal (Normal, or normal) and abnormal. The abnormality can be further divided into five cases: horizontal flip (XFlip), mirror flip (YFlip), clockwise rotation 90° (Rot90), clockwise rotation 180° (Rot180), clockwise rotation 270° (Rot270), etc., which means that the status corresponding to the character can be one of the following: non-abnormal, horizontal flip, mirror flip, clockwise rotation 90°, clockwise rotation 180°, clockwise rotation 270°.

[0183] Through the description of the technical solution provided in the embodiments of the present application above, characters are obtained, the characters are rendered to obtain character pictures, the character pictures are classified to obtain the states corresponding to the characters, and the characters are processed according to the states.

[0184] By rendering the characters to obtain character pictures, classifying the character pictures to obtain the states corresponding to the characters, and processing the characters according to the states, the states of the characters are identified in this way, and corresponding processing is performed accordingly, ensuring the correctness of the characters. In the scenario of converting a layout electronic document into a flowing electronic document, problems such as text garbled characters or incomplete display are avoided in the flowing electronic document converted from the layout electronic document, the conversion effect is improved, and the influence on user reading is avoided.

[0185] In addition, for the above step S101 of obtaining characters and rendering the characters to obtain character pictures, it may specifically include: obtaining at least one font file, for any font file, storing each character in the font file into a corresponding text object, and the text objects form a text object set, traversing the text objects in any text object set, and rendering the characters included in the text objects to obtain character pictures.

[0186] Among them, in the embodiments of the present application, when obtaining at least one font file, the font file here may be, for example, a Song typeface file, a Kai typeface file, etc., and the at least one font file obtained may be a font file in ttf format, or a font file embedded in a word processing software (such as Word), or a font file used in the document, or may be a single font picture after rendering, etc. The present application does not limit the specific source of the font file.

[0187] For the at least one font file obtained, for any font file, store each character in the font file into a corresponding text object (TextObject), and thus the text objects can form a text object set, which means storing each character of the same font file into the corresponding text object, and thus forming a set.

[0188] It should be noted that for the characters in the font file, there may be duplicate cases. For example, for the Song typeface file, there is a duplicate Song typeface character '一'. At this time, duplicate removal processing needs to be performed. Thus, for the text object set, if there are duplicate characters included in the text objects, the redundant text objects can be deleted, and only one text object needs to be retained.

[0189] In addition, the text object (TextObject) is a data structure customized for characters, which contains attributes such as font name, unicode, whether it is bold, whether it is italic, and whether there are any exceptions.

[0190] In an embodiment of the present application, if it is necessary to calibrate each character in a font file, then for any text object set, the text objects in the text object set are traversed in sequence, the characters contained in the text objects are rendered, and character images are obtained (the character images here are generally single-character images). The character images are classified and processed to obtain the status corresponding to the characters, so that the characters can be processed according to the status. Here, the processing of the characters varies depending on the different status of the characters.

[0191] In addition, in the embodiment of the present application, in order to speed up the verification speed of the characters in the font file, a sampling method can be used to randomly sample the characters in the font file. In this way, for the text object set, its internal elements are randomly sorted, and then the sampling number minThr is given, the randomly sorted text object set is sampled, and then the sampled text objects are traversed in turn. If the number of elements in the text object set is less than minThr, all of them are traversed, that is, the number of traversals n = min(minThr, TextObject.size), minThr is the sampling number, and TextObject.size is the number of text objects in the text object set. Based on this, Figure 2 FIG. 1 is a schematic diagram of an implementation flow of another character processing method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0192] S201 , obtaining at least one font file, and for any font file, storing each character in the font file into a corresponding text object, so that the text objects form a text object set.

[0193] In an embodiment of the present application, at least one font file is obtained. For the at least one font file obtained, for any font file, each character in the font file is stored in a corresponding text object (TextObject), so that the text objects can form a text object set.

[0194] S202 : For any text object set, determine whether the number of elements of the text objects in the text object set exceeds a preset number threshold.

[0195] In an embodiment of the present application, for at least one text object set obtained in the above steps, for any text object set, it is determined whether the number of elements of the text objects in the text object set exceeds a preset number threshold (eg, minThr).

[0196] It should be noted that the determination here of whether the number of elements of text objects in the text object set exceeds the preset threshold is essentially to detect whether the text object set meets the conditions for random sampling. If not, there is no need for random sampling and all of them can be traversed.

[0197] S203: When the number of text object elements in the text object set does not exceed a preset number threshold, the following processing is performed.

[0198] S204 , traversing the text objects in the text object set, rendering the characters contained in the text objects, and obtaining character images.

[0199] In an embodiment of the present application, for a text object set, if the number of elements of the text objects in the text object set does not exceed a preset number threshold, it means that the conditions for random sampling are not met and all of them need to be traversed. Therefore, the following processing is performed: traverse the text objects in the text object set, render the characters contained in the text objects, and obtain character images.

[0200] S205: When the number of text object elements in the text object set exceeds a preset number threshold, the following processing is performed.

[0201] S206 , sorting the text objects in the text object set, and extracting a preset number threshold of text objects from the sorted text objects.

[0202] S207 , traversing a preset number threshold of text objects, rendering the characters contained in the text objects, and obtaining character images.

[0203] In the embodiment of the present application, for a text object set, if the number of elements of the text objects in the text object set exceeds a preset number threshold, it indicates that the conditions for random sampling are met and random sampling can be performed. Therefore, the following processing is performed:

[0204] The text objects in the text object set are sorted, a preset number threshold of text objects are extracted from the sorted text objects, the text objects with the preset number threshold are traversed, and the characters contained in the text objects are rendered to obtain character images.

[0205] In this way, the text objects in the text object set are randomly sampled, and then traversed in the randomly sampled results. Compared with the full traversal, the two traversed text objects have different ranges, and the former can speed up the verification of characters in the font file.

[0206] S208: Input the character image into a pre-trained classification model to obtain probability values ​​of the character image corresponding to each state.

[0207] S209: Select the state corresponding to the maximum probability value, and determine the state corresponding to the maximum probability value as the state corresponding to the character.

[0208] In an embodiment of the present application, for a character image, the character image can be input into a pre-trained classification model to obtain probability values ​​for each state corresponding to the character image, and the state corresponding to the maximum probability value is selected, and the state corresponding to the maximum probability value is determined as the state corresponding to the character. For example, the character image is input into a pre-trained classification model to obtain probability values ​​for each state corresponding to the character image, as shown in Table 1 below.

[0209] state Probability value Non-anomalous 0.1 Flip Horizontal 0.7 Mirror Flip 0.03 Rotate 90° clockwise 0.07 Rotate 180° clockwise 0.06 Rotate 270° clockwise 0.04

[0210] Table 1

[0211] It should be noted that for pre-trained classification models, such as VGG or MobileNet, they can be decomposed into backbone+cls_head according to the structure. Backbone is the backbone network of the pre-trained classification model. Its usual function is to extract the input feature matrix. Cls_head is the convolutional layer or fully connected layer of the pre-trained classification model. Its usual function is to use the feature matrix extracted by backbone for prediction.

[0212] It should be noted that the backbone network usually includes M feature extraction networks, and each feature extraction network includes N convolutional layers and (1) pooling layer, where M and N are both positive integers. For example, the backbone network is specifically convolutional layer 1 + pooling layer 1 + convolutional layer 2 + convolutional layer 2, which means that the input is first convolved once to obtain the first convolution result, the first convolution result is pooled once to obtain the first pooling result, the first pooling result is convolved again to obtain the second convolution result, the second convolution result is pooled once to obtain the final feature matrix.

[0213] For another example, the backbone network is specifically convolution layer 1 + convolution layer 2 + pooling layer 1 + convolution layer 3 + convolution layer 4 + convolution layer 2, which means that the input is first convolved once to obtain the first convolution result, the first convolution result is convolved once to obtain the second convolution result, the second convolution result is pooled once to obtain the first pooling result, the first pooling result is convolved again to obtain the third convolution result, the third convolution result is convolved once to obtain the fourth convolution result, the fourth convolution result is pooled once to obtain the final feature matrix.

[0214] Among them, for the pre-trained classification model, you can refer to Figure 3 The model training method shown is obtained. Figure 3FIG. 1 is a schematic diagram of an implementation flow of a model training method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0215] S301, obtaining a sample font file, rendering sample characters in the sample font file, and obtaining a training character image.

[0216] In an embodiment of the present application, some font files are pre-collected as sample font files, thereby obtaining the sample font files, and rendering the sample characters in the sample font files to obtain training character images. During the rendering process, operations such as bolding, italics, or underlining can be randomly added to increase the diversity of the training character images.

[0217] For the sample font file, each sample character in the sample font file is rendered to obtain a training character image. In the process of rendering the sample characters, a bold operation can be performed on the sample characters, so that the sample characters in the training character image obtained in this way appear in a bold form, such as Figure 4 shown.

[0218] S302 : transforming a preset proportion of training character images according to the abnormal state, and determining the transformed training character images and the remaining training character images as training images.

[0219] In an embodiment of the present application, for the sample characters in the sample font file, the default state is generally normal, that is, non-abnormal, so the synthetically rendered training character images are default positive samples. In order to obtain negative samples, the training character images need to be transformed by a certain proportion.

[0220] Based on this, according to the abnormal state, a preset proportion of the training character images are transformed, so that the transformed training character images and the remaining training character images can be determined as training images to participate in the training of the subsequent classification model.

[0221] For example, 30% of the training character images are transformed according to five abnormal states, namely, horizontal flipping, mirror flipping, 90° clockwise rotation, 180° clockwise rotation, and 270° clockwise rotation, so that the transformed training character images and the remaining training character images can be determined as training images.

[0222] In addition, to increase the diversity of training images, some character images from real scenes can be added as training images based on the synthetically rendered training character images. To this end, a preset training dataset is obtained, wherein the images in the preset training dataset are used to train the optical character recognition model. The pre-trained optical character recognition model is used to identify single character areas in the images, and the single character areas are segmented to obtain single character images. The transformed training character images, the remaining training character images, and the single character images are determined as training images.

[0223] S303: Determine sample labels corresponding to the training images. The sample labels are initial probability values ​​of various states corresponding to the training images.

[0224] In an embodiment of the present application, for a training image, a sample label corresponding to the training image is determined, wherein the sample label is the initial probability value of each state corresponding to the training image, that is, it represents the initial probability value of each state shown in Table 1 above corresponding to the training image.

[0225] S304: Input the training image into the classification model to obtain the sample probability value of each state corresponding to the training image.

[0226] In an embodiment of the present application, for a training image, the training image can be input into a classification model to obtain sample probability values ​​of various states corresponding to the training image. The classification model here can be, for example, VGG, MobileNet, etc.

[0227] S305: Determine a loss value based on the sample probability value and the initial probability value, and train the classification model based on the loss value.

[0228] In an embodiment of the present application, for the initial probability values ​​of each state corresponding to the training image, and the sample probability values ​​of each state corresponding to the training image, the loss value can be determined based on the sample probability value and the initial probability value, and the classification model can be trained based on the loss value.

[0229] Among them, a cross-entropy loss function can be selected, and the sample probability value and the initial probability value can be input into the cross-entropy loss function to obtain a cross-entropy loss value, thereby training the classification model according to the cross-entropy loss value. This embodiment of the present application is not limited to this.

[0230] S306: When the loss value is less than the preset loss value threshold, stop training to obtain a pre-trained classification model.

[0231] In an embodiment of the present application, for the loss value, when the loss value converges, that is, when the loss value is less than the preset loss value threshold, it means that the classification model converges at this time, and training can be stopped, so that a pre-trained classification model can be obtained.

[0232] Through the above model training, a final pre-trained classification model can be obtained. Its input is a single-character image, and the output is the probability values corresponding to each state. The state corresponding to the maximum probability value is selected as the predicted state of the character.

[0233] S210, determine whether the state is a normal state, and if the state is a normal state, add a preset value to the first occurrence count corresponding to the normal state.

[0234] In the embodiment of the present application, a state vector vec is set, which can count the occurrence times of each state. For example, for the 6 states shown in Table 1 above, a state vector vec with a length of 6 is set. Among them, vec[0] is used to count the occurrence times of non-abnormal, vec[1] is used to count the occurrence times of horizontal flipping, and so on.

[0235] Based on this, for the state corresponding to the character, determine whether the state is a normal state, and if the state is a normal state, add a preset value to the first occurrence count corresponding to the normal state. For example, the preset value can be 1 or any positive integer. For example, for the state corresponding to the character, determine whether the state is non-abnormal, and if the state is non-abnormal, add 1 to vec[0].

[0236] Among them, a confusion table is preset. The confusion table refers to the fact that some characters in the font file have similar characteristics with other characters in different states and are prone to confusion. For example, the character "N" rotated 90° clockwise and the character "Z" are similar when rendered in some font files, and the mirror flip of the character "口" is also similar to the character "口" in the non-abnormal state.

[0237] Thus, for the state corresponding to the character, before determining whether the state is a normal state, obtain the preset confusion table. The preset confusion table stores confusion pairs, and each confusion pair consists of a sample character and a state. Among them, the sample character and the state correspond to each other. If there is no matching confusion pair for the character and the state corresponding to the character, determine whether the state is a normal state.

[0238] If there is a matching confusion pair for the character and the state corresponding to the character, it means that the character is extremely prone to confusion with other characters at this time. At this time, the step of counting the occurrence times can be skipped, and directly select to traverse the next text object, which means identifying the state corresponding to the next character until the traversal ends.

[0239] It should be noted that for the character and the state corresponding to the character, if there is a sample character in the preset confusion table that matches the character, and the state corresponding to the sample character matches the state corresponding to the character, it means that there is a matching confusion pair for the character and the state corresponding to the character.

[0240] Among them, for the preset confusion table, you can use Figure 5 The method shown is obtained. Figure 5 FIG. 1 is a schematic diagram of an implementation flow of a method for constructing an obfuscation table provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0241] S501 , obtaining sample font files, and screening out a target sample font file from the sample font files, wherein the sample characters in the target sample font file cover a preset character set.

[0242] In an embodiment of the present application, a number of font files are pre-collected as sample font files, thereby obtaining sample font files, and a target sample font file (e.g., a Songti file) is selected from the sample font files, wherein the sample characters in the target sample font file cover a preset character set. "Covering" here means "including," meaning that the sample characters in the target sample font file must completely include the characters in the preset character set.

[0243] For example, the preset character set is "1", "2", "a", "b", "-", and so on. For sample font file A, the sample characters it contains are "1", "2", "a", "b", "-", and so on. The sample characters contained in sample font file A completely cover the preset character set, so sample font file A can be selected as the target sample font file. The purpose of selecting the target sample font file here is to render the subsequent target sample characters, that is, the display of simple characters needs to rely on a specific font.

[0244] In addition, it should be noted that under normal circumstances, the sample characters in the target sample font file can completely cover the preset character set, but inevitably, there may be a situation where the preset character set cannot be completely covered. At this time, the target sample font file can be further screened from the sample font file, and the sample characters in the target sample font file can cover the remaining characters in the preset character set, and so on, until all the characters in the preset character set are covered. The embodiments of the present application are not limited to this.

[0245] S502: Generate target sample characters according to sample characters in the target sample font file, where the sample characters are consistent with characters in a preset character set.

[0246] In an embodiment of the present application, target sample characters are generated based on sample characters in a selected target sample font file. The sample characters in the target sample font file are consistent with characters in a preset character set. The sample characters are generally assumed to be in a non-abnormal state, i.e., a normal state. Therefore, target sample characters in the six states shown in Table 1 above need to be generated based on the sample characters.

[0247] To this end, the sample characters in the normal state in the target sample font file are transformed. For example, the sample characters in the normal state in the target sample font file can be randomly transformed according to the five abnormal states described in Table 1 above, and five sample characters in abnormal states can be obtained. The sample characters in the normal state and the sample characters in the abnormal state are determined as target sample characters, so that target sample characters in six states as shown in Table 1 above can be obtained.

[0248] S503: Render the target sample character to obtain a sample character image, and store the sample character image into an image set.

[0249] In the embodiment of the present application, for the target sample character, the target sample character can be rendered to obtain a sample character image, and the sample character image is stored in a picture set, so that the picture set contains sample character images of the six states shown in Table 1 above corresponding to each sample character in the target sample font file, such as Figure 6 shown.

[0250] S504 , determining a feature matrix corresponding to any sample character image in the image set, and generating a similarity matrix based on the feature matrix.

[0251] In an embodiment of the present application, for a picture set, a feature matrix corresponding to any sample character picture in the picture set is determined, and a similarity matrix is ​​generated based on the feature matrix, wherein the feature matrix refers to a feature matrix extracted by the backbone network of the pre-trained classification model.

[0252] To this end, for any sample character image in the image collection, the sample character image is input into the pre-trained classification model, and the feature matrix corresponding to the sample character image output by the backbone network in the pre-trained classification model is intercepted.

[0253] Specifically, the backbone network usually includes M feature extraction networks, each of which includes N convolutional layers and (1) pooling layer, where M and N are positive integers. Thus, the following operations are iteratively performed until the Mth feature matrix is ​​obtained, and the following operations are stopped, and the Mth feature matrix is ​​intercepted: the i-th object to be extracted is input into the i-th feature extraction network, the i-th object to be extracted is convolved through the N convolutional layers in the i-th feature extraction network to obtain the convolution result, the convolution result is pooled through the pooling layer in the i-th feature extraction network to obtain the i-th feature matrix; wherein the i-th feature matrix is ​​the i+1th object to be extracted, the first object to be extracted is a sample character image, the M-th feature matrix is ​​determined as the feature matrix corresponding to the sample character image, and i is 1, 2, 3, ..., M in sequence.

[0254] It should be noted that for the N convolutional layers in the feature extraction network, the first convolutional layer performs convolution processing on the i-th object to be extracted to obtain the first convolution processing result. The second convolutional layer performs convolution processing on the first convolution processing result to obtain the second convolution processing result. And so on, the N-th convolutional layer performs convolution processing on the N-1-th convolution processing result to obtain the N-th convolution processing result. For the pooling layer in the feature extraction network, the pooling layer performs pooling processing on the N-th convolution processing result to obtain the i-th feature matrix.

[0255] The similarity matrix can be obtained by calculating the similarity between the feature matrix of each sample character image and the feature matrix of any sample character image. The diagonal of the similarity matrix represents the similarity between the image itself and itself.

[0256] To this end, all sample character pictures in the picture set are sorted to obtain a sample character picture sorting sequence; for the first sample character picture, the following processing is performed, where the first sample character picture is any sample character picture in the sample character picture sorting sequence: the similarity between the feature matrix of the first sample character picture and the feature matrix of the second sample character picture is determined; wherein the second sample character picture is any sample character picture in the sample character picture sorting sequence; a first sorting order of the first sample character picture and a second sorting order of the second sample character picture are determined; according to the first sorting order and the second sorting order, a matrix position of the similarity is formed, and a similarity matrix is ​​formed by the similarity and the matrix position.

[0257] For example, if M sample characters are selected to construct the confusion table, and each sample character has 5 transformations, 6M sample character images will be obtained in the end. By sorting the 6M sample character images, a sample character image sorting sequence can be obtained. The sorting rule can be to sort the characters, and the sample character images of the 6 states of the characters are arranged together in a certain order, such as Figure 6 shown.

[0258] For the first sample character picture, taking the first sample character picture in the sample character picture sorting sequence as an example, the remaining sample character pictures in the sample character picture sorting sequence are similar, and the following processing is performed: determine the similarity between the feature matrix of the first sample character picture and the feature matrix of the second sample character picture, the second sample character picture is any sample character picture in the sample character picture sorting sequence, determine the first sorting order 1 of the first sample character picture, and the second sorting order i (i is 1, 2, 3, ...) of the second sample character picture, according to the first sorting order and the second sorting order, form a matrix position (1, i) of the similarity, and form a similarity matrix by the similarity and the matrix position.

[0259] From the above, it can be seen that by calculating the similarity between the feature matrix of the first sample character picture in the sample character picture sorting sequence and the feature matrix of any sample character picture in the sample character picture sorting sequence, the similarity of the first row in the similarity matrix can be obtained. By calculating the similarity between the feature matrix of the second sample character picture in the sample character picture sorting sequence and the feature matrix of any sample character picture in the sample character picture sorting sequence, the similarity of the second row in the similarity matrix can be obtained. And so on, the complete similarity matrix can be finally obtained.

[0260] Among them, assuming that context represents the feature matrix extracted by the backbone network (that is, the extracted image features), the context is first normalized to obtain context_norm, and then the dot product method is used to calculate the similarity between different context_norms, that is, the similarity between the feature matrix of the first sample character image in the above-mentioned sample character image sorting sequence and the feature matrix of any sample character image in the sample character image sorting sequence can be obtained, or the cosine distance or Euclidean distance between the feature matrix of the first sample character image in the above-mentioned sample character image sorting sequence and the feature matrix of any sample character image in the sample character image sorting sequence can be calculated as the similarity, so that a 6M*6M similarity matrix is ​​finally obtained, which is represented by sim_matrix, where the diagonal is the similarity between itself.

[0261] S505: Determine confusion pairs according to the similarity matrix.

[0262] In an embodiment of the present application, the similarity matrix obtained in the above steps can be used to determine confusion pairs, where the confusion pairs are composed of sample characters and states. A similarity threshold thr can be given to filter out confusion pairs that meet the conditions in the similarity matrix and store them in a preset confusion table.

[0263] Based on this, for the similarity of the i-th row and j-th column in the similarity matrix, determine whether the similarity is greater than a preset similarity threshold, where i and j are both positive integers; if the similarity is greater than the similarity threshold, perform the following processing to determine the confusion pair:

[0264] Determine the feature matrix corresponding to the i-th row, determine the sample character image corresponding to the feature matrix; determine the target sample character corresponding to the sample character image, determine the sample character corresponding to the target sample character; determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state.

[0265] Determine the feature matrix corresponding to the j-th column, determine the sample character image corresponding to the feature matrix; determine the target sample character corresponding to the sample character image, determine the sample character corresponding to the target sample character; determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state.

[0266] For example, for the similarity of the 1st row and 3rd column in the similarity matrix, represented by sim_matrix [1] [3], it is determined whether it is greater than the preset similarity threshold thr. If the similarity is greater than the similarity threshold thr, the following processing is performed to determine the confusion pair:

[0267] Determine the feature matrix corresponding to the first row, that is, the feature matrix of the first sample character image in the sample character image sorting sequence, thereby determining the first sample character image, and then determining the target sample character corresponding to the first sample character image. The target sample character is generated by the sample character, thereby also determining the sample character corresponding to the target sample character, and determining the state corresponding to the target sample character. The sample character and the state constitute a confusion pair.

[0268] Determine the feature matrix corresponding to the third column, that is, the feature matrix of the third sample character image in the sample character image sorting sequence, thereby determining the third sample character image, and then determining the target sample character corresponding to the third sample character image. The target sample character is generated by the sample character, thereby also determining the sample character corresponding to the target sample character, and determining the state corresponding to the target sample character. The sample character and the state constitute a confusion pair.

[0269] In addition, since all the sample character images in the image collection are sorted according to a specific sorting rule, a more convenient way can be used here to determine the confusion pairs. If M sample characters are selected to construct the confusion table, and each sample character has 5 transformations, then 6M sample character images will be obtained in the end. The 6M sample character images are sorted, and the sorting rule can be to sort the characters, and the sample character images of the 6 states of the characters are arranged together in order.

[0270] To this end, when the similarity is greater than the similarity threshold, the following process is performed to determine the confusion pair:

[0271] By calculating i / the number of states (for example, 6) rounded to the nearest integer, the corresponding sample character can be found. By calculating i% the number of states (for example, 6), the specific state used to transform the sample character can be found, and thus the sample character and the state form a confusion pair. Similarly, the sample character corresponding to the jth column can be calculated, and the state used to transform it can be calculated, and thus the sample character and the state form a confusion pair.

[0272] For example, for the similarity of the 1st row and 3rd column in the similarity matrix, represented by sim_matrix [1] [3], it is determined whether it is greater than the preset similarity threshold thr. If the similarity is greater than the similarity threshold thr, the following processing is performed to determine the confusion pair:

[0273] By rounding off 1 / 6, the value 1 is obtained, which means it is the first sample character. By calculating 1% 6 (the value range is 0 to 5, the same as the value range of the subsequent Key), the value 1 is obtained. 1 represents horizontal flipping, which means that the first sample character is transformed using horizontal flipping. Thus, the confusion pair is composed of the first sample character and the horizontal flipping. Similarly, the sample character corresponding to the third column can be calculated, as well as the state used to transform it. Thus, the confusion pair is composed of the sample character and the state.

[0274] S506: Store the obfuscated pair into a preset obfuscation table.

[0275] In the embodiment of the present application, the obfuscation pairs are stored in a preset obfuscation table. The preset obfuscation table can be constructed in different ways depending on the application scenario. Here, a certain way of construction is used as an example for explanation.

[0276] Key indicates the status of the character. The value range of Key is 0 to 5. 0 indicates normal status, 1 indicates horizontal flipping, 2 indicates mirror flipping, 3 indicates 90° clockwise rotation (Rot90), 4 indicates 180° clockwise rotation (Rot180), and 5 indicates 270° clockwise rotation (Rot270).

[0277] The value represents the character, which means that under the current state represented by the key, the character represented by the value will be confused with the character in other states. Therefore, the above sample characters are stored in the preset confusion table in the form of value and state as key.

[0278] In addition, in the embodiment of the present application, some commonly used sample characters that are easily confused after state transformation can be manually screened out. At this time, the sample characters are stored in the preset confusion table in the form of Value and the state is stored in the form of Key.

[0279] For example, the common sample character "N" is similar to the sample character "Z" in some font files after being rotated 90° clockwise. At this time, the sample character "N" is stored in the preset confusion table as Value, while the sample character "N" rotated 90° clockwise is stored as Key.

[0280] It should be noted that the reason for using manual screening here is that some sample characters are very easy to be confused after state transformation. In order to reduce subsequent state recognition errors, it is very necessary to manually screen the sample characters that are easily confused after state transformation.

[0281] S211: When the state is an abnormal state, a preset value is added to the second occurrence number corresponding to the abnormal state.

[0282] In the embodiment of the present application, a state vector vec is set, which can count the number of times each state occurs. For example, for the six states shown in Table 1 above, a state vector vec of length 6 is set, where vec[0] is used to count the number of times non-abnormal states occur, vec[1] is used to count the number of times horizontal flipping occurs, and so on.

[0283] Based on this, for the state corresponding to the character, it is determined whether the state is a normal state, and if the state is an abnormal state, a preset value is added to the second occurrence number corresponding to the abnormal state, and the preset value may be 1, for example. For example, for the state corresponding to the character state, if the state is an abnormal state, specifically a horizontal flip, vec [1] is added by 1.

[0284] Among them, in order to ensure accuracy, the characters need to be checked twice. If the check is successful, the preset value is added to the second occurrence number corresponding to the abnormal state. Otherwise, there is no need to perform this operation, and the next text object is traversed instead, that is, the next character is traversed until the traversal is completed.

[0285] Based on this, when the state is an abnormal state, the characters are inversely transformed according to the abnormal state to obtain the characters in the normal state; the characters in the normal state are rendered to obtain a character verification picture; the character verification picture is input into the pre-trained classification model to obtain the probability values ​​of each state corresponding to the character verification picture; the state corresponding to the maximum probability value is selected, and when the state corresponding to the maximum probability value is the normal state, the preset value is added to the second number of occurrences corresponding to the abnormal state.

[0286] It should be noted that an abnormal state indicates that a character has undergone an abnormal transformation. In this case, the character can be transformed according to the transformation opposite to the abnormal transformation to complete the inverse transformation of the character. For example, if the abnormal state is a 90° clockwise rotation, which means that the character has been rotated 90° clockwise, the character can be rotated 90° counterclockwise to complete the inverse transformation of the character.

[0287] S212: After the text object traversal is completed, it is determined whether the second occurrence number exceeds the target occurrence number.

[0288] S213: When the second number of occurrences exceeds the target number of occurrences, determine the character corresponding to the abnormal state.

[0289] S214, performing an inverse transformation on the character corresponding to the abnormal state according to the abnormal state to obtain the character in the normal state.

[0290] In the embodiment of the present application, after the text object traversal is completed (referring to the completion of the traversal of all or randomly sampled text objects in the text object set), it is determined whether the second occurrence count corresponding to the abnormal state exceeds the target occurrence count. The target occurrence count is the product of the number of text object traversals and a preset value (e.g., 0.6, an empirical value).

[0291] When the second number of occurrences exceeds the target number of occurrences, determine the character corresponding to the abnormal state, and perform an inverse transformation on the character corresponding to the abnormal state according to the abnormal state to obtain the character in the normal state. When the second number of occurrences does not exceed the target number of occurrences, there is no need to perform an inverse transformation on the character corresponding to the abnormal state.

[0292] It should be noted that an abnormal state indicates that a character has undergone an abnormal transformation. In this case, the character can be transformed according to the transformation opposite to the abnormal transformation to complete the inverse transformation of the character. For example, if the abnormal state is a 90° clockwise rotation, which means that the character has been rotated 90° clockwise, the character can be rotated 90° counterclockwise to complete the inverse transformation of the character.

[0293] By rendering the characters in the font file to obtain character images, classifying the character images, obtaining the corresponding status of the characters, and processing the characters according to the status, the character status is identified and corresponding processing is performed accordingly. In the scenario of converting layout electronic documents to streaming electronic documents, the streaming electronic documents converted from layout electronic documents avoid problems such as garbled text or incomplete display, thereby improving the conversion effect and avoiding the impact on user reading.

[0294] In addition, for the above-mentioned font file, the font file can be repaired, that is, when the second number of occurrences exceeds the target number of occurrences, the character corresponding to the abnormal state is determined, and according to the abnormal state, the character image corresponding to the character is inversely transformed to obtain the character image in the normal state, and then the format is converted, for example, from png format to svg format to obtain an svg file, and then the third-party library FontForge is used to convert the svg file into a font file, thereby completing the repair of the abnormal characters in the font file.

[0295] Corresponding to the above method embodiment, the embodiment of the present application also provides a character processing device, such as Figure 7As shown, the device may include: a character rendering module 710 , a state recognition module 720 , and a character processing module 730 .

[0296] The character rendering module 710 is used to obtain characters, render the characters, and obtain character images;

[0297] A state recognition module 720 is used to classify the character image to obtain the state corresponding to the character;

[0298] The character processing module 730 is configured to process the character according to the status.

[0299] In an optional embodiment, the character processing module specifically includes:

[0300] A font file acquisition submodule is used to acquire at least one font file;

[0301] A set composition submodule, for storing each character in any of the font files into a corresponding text object, and forming a text object set from the text objects;

[0302] The character rendering submodule is used to traverse the text objects in any of the text object sets, render the characters contained in the text objects, and obtain character images.

[0303] In an optional embodiment, the character rendering submodule is specifically configured to:

[0304] For any of the text object sets, determining whether the number of elements of the text objects in the text object set exceeds a preset number threshold;

[0305] When the number of elements of the text objects in the text object set does not exceed the preset number threshold, performing the following processing:

[0306] The text objects in the text object set are traversed, and the characters contained in the text objects are rendered to obtain character images.

[0307] In an optional embodiment, the character rendering submodule is further configured to:

[0308] When the number of elements of the text objects in the text object set exceeds the preset number threshold, the following processing is performed:

[0309] Sorting the text objects in the text object set, and extracting the text objects with the preset number threshold from the sorted text objects;

[0310] The text objects of the preset number threshold are traversed, and the characters contained in the text objects are rendered to obtain character images.

[0311] In an optional embodiment, the state identification module is specifically configured to:

[0312] Inputting the character image into a pre-trained classification model to obtain probability values ​​of each state corresponding to the character image;

[0313] The state corresponding to the maximum probability value is selected, and the state corresponding to the maximum probability value is determined as the state corresponding to the character.

[0314] In an optional embodiment, the character processing module specifically includes:

[0315] A state judgment submodule is used to judge whether the state is normal;

[0316] a first number counting submodule, configured to, when the state is the normal state, add a preset value to the first number of occurrences corresponding to the normal state;

[0317] A second number counting submodule is configured to add a preset value to a second number of occurrences corresponding to the abnormal state when the state is an abnormal state;

[0318] A number judgment submodule is used to judge whether the second number of occurrences exceeds the target number of occurrences after the traversal of the text object is completed;

[0319] The target occurrence count is the product of the traversal count of the text object and a preset value;

[0320] a character determination submodule, configured to determine the character corresponding to the abnormal state when the second number of occurrences exceeds the target number of occurrences;

[0321] The character inverse transformation module is used to perform an inverse transformation on the character corresponding to the abnormal state according to the abnormal state to obtain the character in the normal state.

[0322] In an optional embodiment, the state judgment submodule is specifically configured to:

[0323] Obtaining a preset confusion table, wherein the preset confusion table stores confusion pairs, each of which consists of a sample character and a state;

[0324] In the case where there is no matching confusion pair between the character and the state corresponding to the character, determining whether the state is a normal state;

[0325] In the case where a matching confusion pair exists between the character and the state corresponding to the character, the next text object is traversed until the traversal is completed.

[0326] In an optional embodiment, the device further comprises:

[0327] a font file screening module, configured to obtain sample font files and screen out a target sample font file from the sample font files, wherein the sample characters in the target sample font file cover a preset character set;

[0328] a character generation module, configured to generate target sample characters based on sample characters in the target sample font file, wherein the sample characters are consistent with characters in the preset character set;

[0329] A picture storage module, configured to render the target sample character to obtain a sample character picture, and store the sample character picture into a picture set;

[0330] a matrix determination module, configured to determine a feature matrix corresponding to any of the sample character images in the image set;

[0331] A matrix generation module, configured to generate a similarity matrix based on the feature matrix;

[0332] a confusion pair determination module, configured to determine confusion pairs according to the similarity matrix;

[0333] The confusion pair storing module is used to store the confusion pair into a preset confusion table.

[0334] In an optional embodiment, the character generation module is specifically configured to:

[0335] transforming the sample characters in the normal state in the target sample font file to obtain the sample characters in the abnormal state;

[0336] The sample characters in the normal state and the sample characters in the abnormal state are determined as target sample characters.

[0337] In an optional implementation, the matrix determination module specifically includes:

[0338] A picture input submodule, configured to input any of the sample character pictures in the picture set into a pre-trained classification model;

[0339] The feature matrix interception submodule is used to intercept the feature matrix corresponding to the sample character image output by the backbone network in the pre-trained classification model.

[0340] In an optional embodiment, the backbone network includes M feature extraction networks, each of which includes N convolutional layers and pooling layers, where M and N are both positive integers;

[0341] The feature matrix interception submodule is specifically used for:

[0342] Iterate and perform the following operations until the Mth feature matrix is ​​obtained, then stop the following operations and intercept the Mth feature matrix:

[0343] Inputting the i-th object to be extracted into the i-th feature extraction network, performing convolution processing on the i-th object to be extracted through the N convolution layers in the i-th feature extraction network to obtain a convolution result, and performing pooling processing on the convolution result through the pooling layer in the i-th feature extraction network to obtain an i-th feature matrix;

[0344] Among them, the i-th feature matrix is ​​the i+1-th object to be extracted, the 1st object to be extracted is the sample character image, the M-th feature matrix is ​​determined as the feature matrix corresponding to the sample character image, and i is 1, 2, 3, ..., M in sequence.

[0345] In an optional embodiment, the matrix generation module is specifically configured to:

[0346] Sort all the sample character pictures in the picture set to obtain a sample character picture sorting sequence;

[0347] The following processing is performed on a first sample character picture, where the first sample character picture is any sample character picture in the sample character picture sorting sequence:

[0348] Determining a similarity between the feature matrix of the first sample character image and the feature matrix of the second sample character image;

[0349] Wherein, the second sample character picture is any sample character picture in the sample character picture sorting sequence;

[0350] determining a first sorting order of the first sample character images and a second sorting order of the second sample character images;

[0351] The matrix positions of the similarities are formed according to the first sorting order and the second sorting order, and a similarity matrix is ​​formed by the similarities and the matrix positions.

[0352] In an optional embodiment, the confusion pair determination module is specifically configured to:

[0353] For the similarity in the i-th row and j-th column of the similarity matrix, determine whether the similarity is greater than a preset similarity threshold, where i and j are both positive integers;

[0354] When the similarity is greater than the similarity threshold, the following process is performed to determine the confusion pair:

[0355] Determine the feature matrix corresponding to the i-th row, and determine the sample character image corresponding to the feature matrix;

[0356] Determine a target sample character corresponding to the sample character picture, and determine the sample character corresponding to the target sample character;

[0357] Determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state;

[0358] Determine the feature matrix corresponding to the j-th column, and determine the sample character image corresponding to the feature matrix;

[0359] Determine a target sample character corresponding to the sample character picture, and determine the sample character corresponding to the target sample character;

[0360] The state corresponding to the target sample character is determined, and a confusion pair is formed by the sample character and the state.

[0361] In an optional embodiment, the second number statistics submodule is specifically configured to:

[0362] In the case where the state is an abnormal state, performing an inverse transformation on the character according to the abnormal state to obtain a character in a normal state;

[0363] Render the characters in normal state to obtain a character verification image;

[0364] Inputting the character verification image into a pre-trained classification model to obtain probability values ​​of various states corresponding to the character verification image;

[0365] The state corresponding to the maximum probability value is selected, and when the state corresponding to the maximum probability value is a normal state, a preset value is added to the second number of occurrences corresponding to the abnormal state.

[0366] In an optional embodiment, the device further comprises:

[0367] A sample character rendering module is used to obtain a sample font file, render the sample characters in the sample font file, and obtain a training character image;

[0368] An image transformation module, configured to transform the training character image of a preset proportion according to an abnormal state;

[0369] A training picture determination module, configured to determine the transformed training character picture and the remaining training character pictures as training pictures;

[0370] a label determination module, configured to determine a sample label corresponding to the training image, wherein the sample label is an initial probability value of each state corresponding to the training image;

[0371] A training picture input module is used to input the training picture into the classification model to obtain the sample probability value of each state corresponding to the training picture;

[0372] A model training module, configured to determine a loss value based on the sample probability value and the initial probability value, and train the classification model based on the loss value;

[0373] The model training stop module is used to stop training when the loss value is less than a preset loss value threshold to obtain the pre-trained classification model.

[0374] In an optional embodiment, the training image determination module is specifically configured to:

[0375] The step of determining the transformed training character picture and the remaining training character pictures as training pictures includes:

[0376] Obtaining a preset training data set, wherein the images in the preset training data set are used for optical character recognition model training;

[0377] Using a pre-trained optical character recognition model, identifying a single character region in the image, and segmenting the single character region to obtain a single character image;

[0378] The transformed training character pictures, the remaining training character pictures, and the single-character pictures are determined as the training pictures.

[0379] The present application also provides an electronic device, such as Figure 8 As shown, it includes a processor 81, a communication interface 82, a memory 83 and a communication bus 84, wherein the processor 81, the communication interface 82, and the memory 83 communicate with each other through the communication bus 84.

[0380] Memory 83, for storing computer programs;

[0381] The processor 81 is configured to execute the program stored in the memory 83 by performing the following steps:

[0382] Acquire characters, render the characters, and obtain character images; classify the character images to obtain states corresponding to the characters, and process the characters according to the states.

[0383] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the figure shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0384] The communication interface is used for communication between the above electronic device and other devices.

[0385] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0386] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0387] In another embodiment provided by the present application, a storage medium is further provided, in which instructions are stored. When the storage medium is run on a computer, the computer executes the character processing method described in any one of the above embodiments.

[0388] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the character processing methods described in the above embodiments.

[0389] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0390] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0391] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.

[0392] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the scope of protection of the present application.

Claims

1. A character processing method, characterized in that: The method comprises: Obtaining characters, rendering the characters, and obtaining character images; The character images are classified to obtain states corresponding to the characters, and the characters are processed according to the states.

2. The method according to claim 1, characterized in that The acquiring of characters, rendering of the characters, and obtaining of character images include: Obtain at least one font file, and for any of the font files, store each character in the font file into a corresponding text object, so that the text objects form a text object set; The text objects in any of the text object sets are traversed, and the characters contained in the text objects are rendered to obtain character images.

3. The method according to claim 2, characterized in that The traversing the text objects in any of the text object sets and rendering the characters contained in the text objects to obtain character images includes: For any of the text object sets, determining whether the number of elements of the text objects in the text object set exceeds a preset number threshold; When the number of elements of the text objects in the text object set does not exceed the preset number threshold, performing the following processing: The text objects in the text object set are traversed, and the characters contained in the text objects are rendered to obtain character images.

4. The method according to claim 3, characterized in that The method further comprises: When the number of elements of the text objects in the text object set exceeds the preset number threshold, the following processing is performed: Sorting the text objects in the text object set, and extracting the text objects with the preset number threshold from the sorted text objects; The text objects of the preset number threshold are traversed, and the characters contained in the text objects are rendered to obtain character images.

5. The method according to claim 1, wherein The classifying process of the character image to obtain the status corresponding to the character includes: Inputting the character image into a pre-trained classification model to obtain probability values ​​of each state corresponding to the character image; The state corresponding to the maximum probability value is selected, and the state corresponding to the maximum probability value is determined as the state corresponding to the character.

6. The method according to claim 2, characterized in that The processing of the character according to the state includes: determining whether the state is a normal state, and if the state is the normal state, adding a preset value to the first occurrence number corresponding to the normal state; In the case where the state is an abnormal state, adding a preset value to the second number of occurrences corresponding to the abnormal state; After the traversal of the text object is completed, determining whether the second number of occurrences exceeds the target number of occurrences; The target occurrence count is the product of the traversal count of the text object and a preset value; In a case where the second number of occurrences exceeds the target number of occurrences, determining the character corresponding to the abnormal state; According to the abnormal state, the character corresponding to the abnormal state is inversely transformed to obtain the character in the normal state.

7. The method according to claim 6, characterized in that The determining whether the state is a normal state includes: Obtaining a preset confusion table, wherein the preset confusion table stores confusion pairs, each of which consists of a sample character and a state; In the case where there is no matching confusion pair between the character and the state corresponding to the character, determining whether the state is a normal state; In the case where a matching confusion pair exists between the character and the state corresponding to the character, the next text object is traversed until the traversal is completed.

8. The method according to claim 7, characterized in that Before executing the method, the method further includes: Acquire a sample font file, and filter out a target sample font file from the sample font file, wherein the sample characters in the target sample font file cover a preset character set; generating target sample characters according to the sample characters in the target sample font file, wherein the sample characters are consistent with the characters in the preset character set; Rendering the target sample character to obtain a sample character image, and storing the sample character image into an image set; Determine a feature matrix corresponding to any of the sample character images in the image set, and generate a similarity matrix based on the feature matrix; According to the similarity matrix, confusion pairs are determined, and the confusion pairs are stored in a preset confusion table.

9. The method according to claim 8, characterized in that Generating target sample characters according to the sample characters in the target sample font file includes: transforming the sample characters in the normal state in the target sample font file to obtain the sample characters in the abnormal state; The sample characters in the normal state and the sample characters in the abnormal state are determined as target sample characters.

10. The method according to claim 8, characterized in that The determining of a feature matrix corresponding to any one of the sample character images in the image set includes: For any of the sample character pictures in the picture set, input the sample character picture into a pre-trained classification model; The feature matrix corresponding to the sample character image output by the backbone network in the pre-trained classification model is intercepted.

11. The method according to claim 10, characterized in that The backbone network includes M feature extraction networks, each of which includes N convolutional layers and pooling layers, where M and N are both positive integers; The intercepting of the feature matrix corresponding to the sample character image output by the backbone network in the pre-trained classification model includes: Iterate and perform the following operations until the Mth feature matrix is ​​obtained, then stop the following operations and intercept the Mth feature matrix: Inputting the i-th object to be extracted into the i-th feature extraction network, performing convolution processing on the i-th object to be extracted through the N convolution layers in the i-th feature extraction network to obtain a convolution result, and performing pooling processing on the convolution result through the pooling layer in the i-th feature extraction network to obtain an i-th feature matrix; Among them, the i-th feature matrix is ​​the i+1-th object to be extracted, the 1st object to be extracted is the sample character image, the M-th feature matrix is ​​determined as the feature matrix corresponding to the sample character image, and i is 1, 2, 3, ..., M in sequence.

12. The method according to claim 8, characterized in that Generating a similarity matrix according to the feature matrix includes: Sort all the sample character pictures in the picture set to obtain a sample character picture sorting sequence; The following processing is performed on a first sample character picture, where the first sample character picture is any sample character picture in the sample character picture sorting sequence: Determining a similarity between the feature matrix of the first sample character image and the feature matrix of the second sample character image; Wherein, the second sample character picture is any sample character picture in the sample character picture sorting sequence; determining a first sorting order of the first sample character images and a second sorting order of the second sample character images; The matrix positions of the similarities are formed according to the first sorting order and the second sorting order, and a similarity matrix is ​​formed by the similarities and the matrix positions.

13. The method according to claim 12, characterized in that Determining confusion pairs according to the similarity matrix includes: For the similarity in the i-th row and j-th column of the similarity matrix, determine whether the similarity is greater than a preset similarity threshold, where i and j are both positive integers; When the similarity is greater than the similarity threshold, the following process is performed to determine the confusion pair: Determine the feature matrix corresponding to the i-th row, determine the sample character image corresponding to the feature matrix; determine the target sample character corresponding to the sample character image, determine the sample character corresponding to the target sample character; determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state; Determine the feature matrix corresponding to the j-th column, determine the sample character image corresponding to the feature matrix; determine the target sample character corresponding to the sample character image, determine the sample character corresponding to the target sample character; determine the state corresponding to the target sample character, and form a confusion pair with the sample character and the state.

14. The method according to claim 6, characterized in that When the state is an abnormal state, adding a preset value to the second occurrence number corresponding to the abnormal state includes: In the case where the state is an abnormal state, performing an inverse transformation on the character according to the abnormal state to obtain a character in a normal state; Render the characters in normal state to obtain a character verification image; Inputting the character verification image into a pre-trained classification model to obtain probability values ​​of various states corresponding to the character verification image; The state corresponding to the maximum probability value is selected, and when the state corresponding to the maximum probability value is a normal state, a preset value is added to the second number of occurrences corresponding to the abnormal state.

15. The method according to claim 5, characterized in that Before executing the method, the method further includes: Obtaining a sample font file, rendering sample characters in the sample font file, and obtaining a training character image; According to the abnormal state, a preset proportion of the training character images are transformed, and the transformed training character images and the remaining training character images are determined as training images; Determine a sample label corresponding to the training image, where the sample label is an initial probability value of each state corresponding to the training image; Input the training image into the classification model to obtain the sample probability value of each state corresponding to the training image; Determining a loss value based on the sample probability value and the initial probability value, and training the classification model based on the loss value; When the loss value is less than a preset loss value threshold, the training is stopped to obtain the pre-trained classification model.

16. The method according to claim 15, characterized in that The step of determining the transformed training character picture and the remaining training character pictures as training pictures includes: Obtaining a preset training data set, wherein the images in the preset training data set are used for optical character recognition model training; Using a pre-trained optical character recognition model, identifying a single character region in the image, and segmenting the single character region to obtain a single character image; The transformed training character pictures, the remaining training character pictures, and the single-character pictures are determined as the training pictures.

17. A character processing device, characterized in that: The device comprises: A character rendering module is used to obtain characters, render the characters, and obtain character images; A state recognition module is used to classify the character image to obtain the state corresponding to the character; The character processing module is used to process the character according to the state.

Images