Method for reducing character layout in an image

Abstract

The application provides a method for restoring character layout in an image, and relates to the technical field of image recognition, and comprises the following steps: performing image recognition on a target image to obtain each character in the target image and the circumscribed rectangle information of the each character; for each character, according to the character and the circumscribed rectangle information of the character, target configuration data of the character is selected from a configuration data set of each character style; the target character style of the character is determined according to the character style corresponding to the target configuration data, and the target font size and position information of the character are determined according to the target configuration data and the circumscribed rectangle information; and the character layout of the each character in the target image is restored according to the target character style, target font size and position information of the each character. The original layout effect of the characters in the target image can be restored more accurately, and the accuracy and reliability of the restored character layout are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a method for restoring character layout in an image. BACKGROUND

[0002] With the continuous development of image recognition technology, optical character recognition (OCR) technology is used, and more and more text in pictures can be recognized.

[0003] When using an OCR text recognition engine to recognize text in a picture, the returned text information, in addition to text encoding, only returns the circumscribed rectangular region of the text image in relation to the size and position of the text, and does not return the font name, font size, and accurate position of the text display. Based on this, when restoring the recognized text into a document in a portable document format (PDF) or an open fixed-layout document format (OFD) or the like, or when drawing text using font name, font size, and text position for OCR text correction, etc., the layout effect is often not satisfactory, such as font, font style, text size, and text position, which cannot be consistent with the original text in the image, resulting in inconsistencies in the layout of the original image, such as different text sizes, different heights in the vertical direction, crowding or sparseness in the horizontal direction, and other inconsistencies in the layout of the original image, which makes it impossible to restore the text layout of the OCR recognition result, and limits the application of the OCR recognition result. Therefore, an effective method is needed to solve the above problems. SUMMARY

[0004] To solve the problems in the prior art, the present application provides a method for restoring character layout in an image.

[0005] The present application provides a method for restoring character layout in an image, comprising:

[0006] performing image recognition on a target image to obtain each character in the target image and the circumscribed rectangular information of each character;

[0007] For each character, according to the character and the circumscribed rectangular information of the character, respectively filter the target configuration data of the character from the configuration data set of each character style;

[0008] determine the target character style of the character according to the character style corresponding to the target configuration data, and determine the target font size and position information of the character according to the target configuration data and the circumscribed rectangular information;

[0009] restore the character layout of each character in the target image according to the target character style, the target font size and the position information of the character.

[0010] The method for restoring the character layout in the image according to the present application, wherein the target font size and the position information of the character are determined according to the target configuration data and the circumscribed rectangle information, comprises:

[0011] The target font size of the character is determined according to the font scaling ratio in the target configuration data and the rectangle size in the circumscribed rectangle information.

[0012] The position information of the character is determined according to the position offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information.

[0013] The method for restoring the character layout in the image according to the present application, wherein the position information of the character is determined according to the position offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information, comprises:

[0014] The position offset amount of the character is determined according to the position offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information.

[0015] The position information of the character is determined according to the position offset amount and the relative position of the circumscribed rectangle in the target image in the circumscribed rectangle information.

[0016] The method for restoring the character layout in the image according to the present application, wherein the target font size and the position information of the character are determined according to the target configuration data and the circumscribed rectangle information, comprises:

[0017] The target font size and the position information of the character are determined according to the height of the circumscribed rectangle in the target configuration data and the circumscribed rectangle information.

[0018] The method for restoring the character layout in the image according to the present application, wherein the target configuration data of each character is selected from the configuration data set of each character style according to the character and the circumscribed rectangle information of the character, comprises:

[0019] The initial configuration data of each character is selected from the configuration data set of each character style, wherein the configuration data set comprises the configuration data of all characters.

[0020] The target configuration data is determined from the initial configuration data according to the similarity between the circumscribed rectangle information of the character and each initial configuration data.

[0021] According to the method for restoring character layout in an image, the circumscribed rectangle information comprises height and width of a circumscribed rectangle, and the circumscribed rectangle corresponds to a rectangular character image;

[0022] Accordingly, the target configuration data is determined from the initial configuration data according to the similarity between the circumscribed rectangle information of the character and each initial configuration data.

[0023] According to the height and width of the circumscribed rectangle, a target font aspect ratio of the character is determined.

[0024] The ratio between the target font aspect ratio and the font aspect ratio configured in each initial configuration data is calculated, and a plurality of candidate configuration data is selected from the initial configuration data according to the ratio.

[0025] The image similarity between the rectangular character image and the character image configured in each candidate configuration data is calculated, and the target configuration data is determined from the plurality of candidate configuration data according to the image similarity.

[0026] According to the method for restoring character layout in an image, the character style comprises font and font shape.

[0027] Accordingly, the target character style of the character is determined according to the character style corresponding to the target configuration data.

[0028] The font corresponding to the target configuration data is determined as the initial font of the character.

[0029] The initial font of each character in the target line where the character is located is counted.

[0030] The initial font with the largest number of corresponding characters in the target line is determined as the target font of the character, and the font shape corresponding to the target configuration data is determined as the target font shape of the character.

[0031] According to the method for restoring character layout in an image, before the target configuration data of each character is selected from the configuration data set of each character style according to the character and the circumscribed rectangle information of the character, the method further comprises:

[0032] For each character style, the configuration data corresponding to the current character is obtained from the first character corresponding to the character style, until the last character corresponding to the character style, so as to obtain the configuration data set of the character style.

[0033] According to the method for restoring character layout in an image, before the configuration data corresponding to the current character is obtained, the method further comprises:

[0034] set the font size corresponding to the character style;

[0035] Correspondingly, the configuration data corresponding to the current character is obtained, comprising:

[0036] According to the font data of the current character and the character style, the current character is drawn in a square with the font size as the side length;

[0037] Identify the font scaling ratio, position offset ratio and configuration font width-height ratio corresponding to the current character in the square;

[0038] According to the current character, the font scaling ratio, the position offset ratio and the configuration font width-height ratio, the configuration data corresponding to the current character is generated.

[0039] According to the method for restoring character layout in an image provided by the application, the identification of the font scaling ratio, the position offset ratio and the configuration font width-height ratio corresponding to the current character in the square comprises:

[0040] Taking the upper left corner of the square as the origin, the minimum and maximum values of the character font of the current character in the horizontal direction and the minimum and maximum values in the vertical direction are determined;

[0041] The difference between the maximum and minimum values in the horizontal direction is determined as the width of the character font, and the difference between the maximum and minimum values in the vertical direction is determined as the height of the character font;

[0042] According to the font size and the character font size, the font scaling ratio of the character font is determined, the character font size is the height or the width; according to the minimum value and the character font size, the position offset ratio of the character font is determined; the ratio of the width to the height is determined as the configuration font width-height ratio of the character font.

[0043] The application further provides a device for restoring character layout in an image, comprising:

[0044] The identification module is configured to perform image recognition on the target image to obtain each character in the target image and the circumscribed rectangle information of each character;

[0045] The screening module is configured to, for each character, according to the character and the circumscribed rectangle information of the character, respectively screen the target configuration data of the character from the configuration data set of each character style;

[0046] The determining module is configured to determine a target character style of the character according to a character style corresponding to the target configuration data, and determine a target font size and position information of the character according to the target configuration data and the information of the circumscribed rectangle.

[0047] The restoring module is configured to restore the character layout of each character in the target image according to the target character style, the target font size and the position information of each character.

[0048] The present application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for restoring the character layout in the image according to any one of the above methods when executing the program.

[0049] The present application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the method for restoring the character layout in the image according to any one of the above methods.

[0050] The present application further provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the method for restoring the character layout in the image according to any one of the above methods.

[0051] The method for restoring the character layout in the image provided by the present application can accurately restore the original layout effect of the characters in the target image by inversely deducing the target character style (font and character shape), the target font size (font size) and the position information (such as the displacement of the character position in the horizontal direction or the vertical direction) of each character in the target image according to the recognized characters and the circumscribed rectangle information of each character, in combination with the configuration data set of each character style, thereby improving the accuracy and reliability of the restored character layout. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0053] Figure 1 is a flowchart of the method for restoring the character layout in the image provided by the present application;

[0054] Figure 2 is a schematic diagram of the configuration data provided by the present application;

[0055] Figure 3 is a target image provided by the present application;

[0056] Figure 4 is one of the schematic diagrams of the character layout in the restored image provided by the present application;

[0057] Figure 5 is one of the schematic diagrams of the character layout in the restored image provided by the present application;

[0058] Figure 6 is a structural schematic diagram of the device for the character layout in the restored image provided by the present application;

[0059] Figure 7 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0060] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0061] The method for the character layout in the restored image provided by the present application will be described below in combination with Figures 1-5

[0062] Figure 1 is a flow schematic diagram of the method for the character layout in the restored image provided by the present application, as shown in Figure 1 , which comprises steps 101-104, wherein:

[0063] Step 101: image recognition is performed on a target image to obtain each character in the target image and the circumscribed rectangle information of the each character.

[0064] First of all, it should be pointed out that the execution subject of the present application can be any electronic device capable of restoring the character layout in the image, for example, it can be any one of a smart phone, a smart watch, a desktop computer, a laptop computer, etc. The method for the character layout in the restored image provided by the present application can be used in the scene of outputting the PDF, OFD, etc. file of the restored image layout, or correcting the OCR recognized text, etc.

[0065] ​Specifically, the target image refers to an image that needs to be restored in character layout. The character (Char) refers to a character unit or a symbol, including letters, numbers, operator symbols, punctuation marks and other symbols, and some functional symbols, which can be obtained by encoding such as UTF-8, UTF-16 or GB18030. The circumscribed rectangle refers to a rectangle formed according to the contour of the character, that is, the circumscribed rectangle of the character font, that is, the circumscribed rectangle of the character image, which generally refers to the rectangle surrounded by the horizontal line corresponding to the uppermost vertex and the lowermost vertex of the character contour, and the vertical line corresponding to the leftmost vertex and the rightmost vertex. The circumscribed rectangle information refers to the information related to the circumscribed rectangle, such as position, length, width, etc., wherein the circumscribed rectangle information of the character refers to the circumscribed rectangle information of the font corresponding to the character, that is, the circumscribed rectangle information of the character font.

[0066] In actual application, after receiving the restoration character layout instruction for the target image or receiving the target image, the image recognition technology can be used to identify the target image, such as OCR technology, optical character verification (OCV) technology, to obtain all character information identified in the target image, including at least the character and the circumscribed rectangle information of the character.

[0067] It should be noted that the circumscribed rectangle is generally in units of pixels, so the width and height of the circumscribed rectangle need to be converted into physical units such as millimeters or pounds according to the resolution of the target image. The width and height of the circumscribed rectangle used in the present application are all physical units such as millimeters or pounds after conversion.

[0068] Step 102: For each character, according to the character and the circumscribed rectangle information of the character, respectively filter the target configuration data of the character from the configuration data set of each character style.

[0069] Specifically, the character style refers to the format that affects the shape of the character, mainly including font (Font) and glyph (Glyph), font such as SimSun, FangSong, KaiTi, etc., and glyph, that is, font style, including Regular, Italic, Bold and Italic Bold, etc. The configuration data set refers to a set composed of configuration data of all characters under the current character style. The configuration data refers to various parameters of the character corresponding to the current character style, for example, in the configuration data set of the character style of SimSun Bold, the configuration data of characters such as "one", "two", etc.

[0070] In actual application, for any character, the bounding rectangle information of the character is matched with the configuration data corresponding to the character in the configuration data set of different character styles, and the matching successful configuration data is taken as the target configuration data. All characters in the target image are traversed to obtain the configuration data of each character.

[0071] For example, the character "I" in the target image is matched with the configuration data of "I" in the configuration data set of Kai font bold and the configuration data of "I" in the configuration data set of Song font oblique respectively, and if the configuration data of "I" in the configuration data set of Song font oblique is matched successfully, the configuration data of "I" corresponding to Song font oblique is taken as the target configuration data of the character "I".

[0072] Step 103: determining the target character style of the character according to the character style corresponding to the target configuration data, and determining the target font size and position information of the character according to the target configuration data and the bounding rectangle information.

[0073] Specifically, the font size represents the size of the character. The position information represents the position of the character in the target image.

[0074] In actual application, the character style corresponding to the target configuration data can be taken as the target character style of the character, and the target font size and position information of the character can be obtained by data processing based on the bounding rectangle information of the character and the target configuration data.

[0075] In the above example, since the configuration data of "I" corresponding to Song font oblique is taken as the target configuration data of the character "I", the target character style of the character "I" is Song font oblique. The bounding rectangle information of the character "I" and the target configuration data are input into the pre-trained character processing model to obtain the target font size and position information of the character "I", or the bounding rectangle information of the character "I" and the target configuration data are calculated according to the set calculation rule to obtain the target font size and position information of the character "I".

[0076] Step 104: restoring the character layout of each character in the target image according to the target character style, the target font size and the position information of each character.

[0077] In actual application, for each character, the character style of the character in the display page is set to the target character style, the font size is set to the target font size, and the display position is set to the position pointed by the position information, and the character is displayed. That is, the character is displayed in the target character style and the target font size in the display page, and the position of the character in the display page is the position pointed by the position information. The display page is a page displaying the character layout in the target image, such as a page of a restored version of a PDF, OFD, Word or the like.

[0078] It should be noted that, on the basis of obtaining the target character style, target font size and position information of all characters in the target image, the character layout of each character in the target image can be restored in the display page according to the target character style, target font size and position information of each character in turn. Alternatively, the character layout of each character in the target image can be restored in the display page according to the target character style, target font size and position information of each character as the target character style, target font size and position information of each character are obtained, and then the target character style, target font size and position information of the next character are obtained until all characters on the target image are displayed.

[0079] The method for restoring character layout in an image provided by the application can reversely deduce the target character style (font and shape), target font size (font size) and position information (such as the displacement of the character position in the horizontal direction or the vertical direction) of each character in the target image by the recognized each character and the information of the circumscribed rectangle of each character in the target image, in combination with the configuration data set of each character style, and can restore the original layout effect of the characters in the target image more accurately, and improve the accuracy and reliability of the restored character layout.

[0080] In one or more optional embodiments of the application, the target font size and position information of the character are determined according to the target configuration data and the circumscribed rectangle information, including:

[0081] The target font size of the character is determined according to the font scaling ratio in the target configuration data and the rectangle size in the circumscribed rectangle information;

[0082] The position information of the character is determined according to the position offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information.

[0083] Specifically, the font scaling ratio (Font_Size_Ratio) refers to the scaling ratio of the character size (font size) based on the font height or font width, that is, the ratio of the font size to the font height, or the ratio of the font size to the font width. The rectangle size refers to the size of the circumscribed rectangle, generally referring to the width and height. The position offset ratio (Offset_Ratio) refers to the offset ratio in the horizontal direction and the vertical direction based on the font height or the font width.

[0084] In practical applications, in the case of a font scaling ratio being a ratio of a font size to a font height, the height of the bounding rectangle in the character bounding rectangle information is multiplied by the font scaling ratio to obtain a target font size of the character; in the case of the font scaling ratio being a ratio of the font size to a font width, the width of the bounding rectangle in the character bounding rectangle information is multiplied by the font scaling ratio to obtain the target font size of the character. Correspondingly, in the case of a position offset ratio being an offset ratio of the font height in a horizontal direction and a vertical direction, the height of the bounding rectangle in the character bounding rectangle information is multiplied by the position offset ratio to obtain position information of the character; in the case of the position offset ratio being an offset ratio of the font width in the horizontal direction and the vertical direction, the width of the bounding rectangle in the character bounding rectangle information is multiplied by the position offset ratio to obtain the position information of the character.

[0085] In this way, by the font scaling ratio and the position offset ratio in the target configuration data and in combination with the rectangle size in the bounding rectangle information, the position information of the character can be quickly and accurately determined while the determination procedure is simplified, and the accuracy and efficiency of character restoration are improved.

[0086] In addition, the position offset ratio includes a horizontal offset ratio (X_Offset_Ratio) and a vertical offset ratio (Y_Offset_Ratio), wherein the horizontal offset ratio is an offset ratio of the font height or the font width in the horizontal direction, i.e., an offset ratio of the character based on the font height or the font width in the horizontal direction (X axis); and the vertical offset ratio is an offset ratio of the font height or the font width in the vertical direction, i.e., an offset ratio of the character based on the font height or the font width in the vertical direction (Y axis). Correspondingly, the position information includes horizontal position information and vertical position information. The determination of the position information of the character according to the position offset ratio in the target configuration data and the rectangle size in the bounding rectangle information can be: determining the horizontal position information of the character according to the horizontal offset ratio in the target configuration data and the rectangle size in the bounding rectangle information; and determining the vertical position information of the character according to the vertical offset ratio in the target configuration data and the rectangle size in the bounding rectangle information.

[0087] Optionally, the position offset ratio in the target configuration data and the rectangle size in the bounding rectangle information can be input into a pre-trained character position determination model to obtain the position information of the character; or the position offset ratio in the target configuration data and the rectangle size in the bounding rectangle information can be input into a set position calculation formula to obtain the position information of the character.

[0088] Optionally, the determination of the position information of the character according to the position offset ratio in the target configuration data and the rectangle size in the bounding rectangle information can be implemented as follows:

[0089] determine the position offset of the character according to the position offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information;

[0090] determine the position information of the character according to the position offset and the relative position of the circumscribed rectangle in the target image in the circumscribed rectangle information.

[0091] Specifically, the position offset refers to the offset of a character of a certain font size relative to the square corresponding to the font size. For example, a square with A font size as the side length is set, and a character of A font size is displayed in the square. The position of the character is offset relative to the position of the square, which is the position offset.

[0092] In actual application, when the position offset ratio is the offset ratio of the font height in the horizontal direction and the vertical direction, the height of the circumscribed rectangle in the character circumscribed rectangle information is multiplied by the position offset ratio to obtain the position offset of the character, and the position offset is added to the relative position to obtain the position information of the character. When the position offset ratio is the offset ratio of the font width in the horizontal direction and the vertical direction, the width of the circumscribed rectangle in the character circumscribed rectangle information is multiplied by the position offset ratio to obtain the position offset of the character, and the position offset is added to the relative position to obtain the position information of the character.

[0093] In this way, the position offset ratio and the rectangle size are used to obtain the position offset of the character and the corresponding square, and the position offset is added to the relative position of the circumscribed rectangle in the target image to obtain the position information of the character, which can improve the accuracy of the position information and further improve the accuracy of character restoration.

[0094] In the case where the position offset ratio includes a horizontal offset ratio and a vertical offset ratio, the position offset includes a horizontal position offset and a vertical position offset. The determination of the position offset of the character according to the position offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information can be: determining the horizontal position offset (X_Offset) of the character according to the horizontal offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information, and determining the vertical position offset (X_Offset) of the character according to the vertical offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information.

[0095] Correspondingly, the relative position includes a horizontal relative position and a vertical relative position, the horizontal relative position representing a relative position of the circumscribed rectangle and the target image in a horizontal direction, and the vertical relative position representing a relative position of the circumscribed rectangle and the target image in a vertical direction. At this time, in a case where the position information includes horizontal position information and vertical position information, the position information of the character can be determined according to the position offset and the relative position of the circumscribed rectangle in the target image in the circumscribed rectangle information, that is, the horizontal position information of the character can be determined according to the horizontal position offset and the horizontal relative position of the circumscribed rectangle in the target image in the circumscribed rectangle information, and the vertical position information of the character can be determined according to the vertical position offset and the vertical relative position of the circumscribed rectangle in the target image in the circumscribed rectangle information.

[0096] Optionally, the configuration data is constructed based on the height of the font, and at this time, the target font size and the position information of the character can be determined according to the height in the target configuration data and the circumscribed rectangle information. That is, the target font size and the position information of the character can be determined according to the target configuration data and the circumscribed rectangle information, and the specific implementation process can be as follows:

[0097] The target font size and the position information of the character are determined according to the height of the circumscribed rectangle in the target configuration data and the circumscribed rectangle information.

[0098] In actual application, the configuration data is generally constructed based on the height of the font. At this time, the target font size and the position information of the character can be determined based on the height of the circumscribed rectangle in the target configuration data and the circumscribed rectangle information. In this way, the height in the size of the rectangle is used to determine the target font size and the position information of the character more finely, and the accuracy of the target font size and the position information is further improved.

[0099] In addition, in a case where the configuration data is constructed based on the height of the font, the font scaling ratio is a ratio of the font size to the font height, and the position offset ratio is an offset ratio of the font height in the horizontal direction and the vertical direction.

[0100] Exemplarily, the font scaling ratio in the target configuration data is multiplied by the height of the circumscribed rectangle in the A character OCR result (circumscribed rectangle information) to obtain the target font size of the A character, the horizontal offset ratio in the target configuration data is multiplied by the height of the circumscribed rectangle in the A character OCR result to obtain the horizontal position offset, and the vertical offset ratio in the target configuration data is multiplied by the height of the circumscribed rectangle in the A character OCR result to obtain the vertical position offset.

[0101] Optionally, the configuration data can also be constructed based on the width of the font, and in this case, the target font size and position information of the character can be determined according to the width in the target configuration data and the bounding rectangle information. That is, the target font size and position information of the character can be determined according to the target configuration data and the bounding rectangle information, and the specific implementation process can be as follows:

[0102] The target font size and position information of the character are determined according to the width of the bounding rectangle in the target configuration data and the bounding rectangle information.

[0103] In actual applications, the configuration data can be constructed based on the width of the font, and in this case, the target font size and position information of the character can be determined based on the width of the bounding rectangle in the target configuration data and the bounding rectangle information. In this way, the width in the size of the rectangle is used to determine the target font size and position information of the character in a more fine-grained manner, further improving the accuracy of the target font size and position information.

[0104] In addition, in the case where the configuration data is constructed based on the width of the font, the font scaling ratio is the ratio of the font size to the font width, and the position offset ratio is the offset ratio of the font width in the horizontal direction and the vertical direction.

[0105] It should be noted that the data used in the calculation of the present application, such as height, width, etc., are all floating-point numbers.

[0106] In one or more optional embodiments of the present application, the target configuration data of the character is selected from the configuration data set of each character style according to the character and the bounding rectangle information of the character, and the specific implementation process can be as follows:

[0107] The initial configuration data of the character is selected from the configuration data set of each character style, and the configuration data set includes the configuration data of all characters;

[0108] The target configuration data is determined from each initial configuration data according to the similarity between the bounding rectangle information of the character and each initial configuration data.

[0109] In actual applications, the configuration data corresponding to the current character, i.e., the initial configuration data, is extracted from the configuration data set of each character style in turn; then the similarity between each initial configuration data and the bounding rectangle information of the character is determined, and the initial configuration data with the highest similarity is determined as the target configuration data. In this way, the initial configuration data is first selected, and then the target configuration data is determined based on the similarity, which can improve the credibility of the target configuration data and further improve the credibility of the character layout.

[0110] For example, there are three character styles, and the configuration data of character B is extracted from the first character style to obtain first initial configuration data; the configuration data of character B is extracted from the second character style to obtain second initial configuration data; and the configuration data of character B is extracted from the third character style to obtain third initial configuration data. The first similarity between the first initial configuration data and the bounding rectangle information of character B, the second similarity between the second initial configuration data and the bounding rectangle information of character B, and the third similarity between the third initial configuration data and the bounding rectangle information of character B are calculated respectively. If the second similarity>the third similarity>the first similarity, the second initial configuration data is determined as the target configuration data.

[0111] Optionally, the bounding rectangle information of the character and the initial configuration data can be input into a pre-trained similarity calculation model, and the similarity calculation model outputs the similarity between the bounding rectangle information and the initial configuration data. In this way, the efficiency of determining the similarity can be improved.

[0112] Optionally, the bounding rectangle information includes the height and width of the bounding rectangle, and a rectangular character image corresponding to the bounding rectangle. Correspondingly, the process of determining the target configuration data from the initial configuration data according to the similarity between the bounding rectangle information of the character and each initial configuration data can be as follows:

[0113] According to the height and width of the bounding rectangle, the target font width-height ratio of the character is determined.

[0114] The ratio between the target font width-height ratio and the configured font width-height ratio in each initial configuration data is calculated, and a plurality of candidate configuration data is selected from the initial configuration data according to the ratio.

[0115] The image similarity between the rectangular character image and the configured character image in each candidate configuration data is calculated, and the target configuration data is determined from the plurality of candidate configuration data according to the image similarity.

[0116] Specifically, the rectangular character image refers to the image formed by the character and the bounding rectangle, that is, the image of the region where the bounding rectangle is located in the target image. The target width-height ratio refers to the ratio of the width and height of the current character recognized. The configured font width-height ratio (Width / Height_Ratio) refers to the ratio of the width and height of the character font of the current character in the initial configuration data. The configured character image refers to the image of the current character under the character style corresponding to the candidate configuration data, and preferably, the image is a square with a side length of a set font size, and the character displayed in the image is also a set font size.

[0117] In actual application, for each character, the width and height of the current character's bounding rectangle can be divided to obtain the target font aspect ratio of the current character. Then, for each initial configuration data, the target font aspect ratio is divided by the configuration font aspect ratio in the initial configuration data to obtain a ratio. Further, the distance of each ratio to 1 is determined, i.e., the absolute value of the difference between each ratio and 1. The smallest N absolute values from all the absolute values are taken as target absolute values, where N is a positive integer. For example, the absolute values are sorted in ascending order, and the top N absolute values are taken as the target absolute values; or the absolute values less than a set threshold are taken as the target absolute values. Then, the initial configuration parameters corresponding to each target absolute value are determined as the candidate configuration data.

[0118] Further, on the basis of determining the candidate configuration data, the image similarity between the rectangular character image corresponding to the current character and the configuration character image in each candidate configuration data is calculated in sequence; or the image similarity between the rectangular character image corresponding to the current character and the configuration character image in each candidate configuration data is calculated as each candidate configuration data is determined, until the image similarity between the rectangular character image and the configuration character image in all candidate configuration data is determined. Further, the candidate configuration data corresponding to the maximum image similarity in all image similarities is taken as the target configuration data.

[0119] In this way, through double screening, it can be ensured that the target configuration data finally determined is more matched with the character displayed in the target image, and thus the accuracy and reliability of the restoration when the character layout is restored based on the target configuration data are greatly improved.

[0120] In addition, in the case where the configuration data set is the configuration font aspect ratio, the target font aspect ratio of the character can be calculated, and the ratio of the target font aspect ratio to the configuration font aspect ratio can be calculated, and the multiple candidate configuration data can be selected according to the ratio. This process is similar to the process based on the target font aspect ratio described above, and thus will not be described again here to avoid repetition.

[0121] It should be noted that the configuration data set of each character style can be in the form of a data table, for example, the configuration data set of each character style corresponds to a table, and each row or column in each table records the configuration data of a character. The configuration data set of each character style can also be in the form of a database, for example, the configuration data set of each character style corresponds to a database, or the configuration data set of all character styles corresponds to a database, and the configuration data set of each character style corresponds to a data storage area in the database. The configuration data set of each character style can also be in any other form of data storage now or in the future, and the present application does not make specific limitations in this regard.

[0122] Optionally, the determining the target character style of the character according to the character style corresponding to the target configuration data can be: determining the character style corresponding to the target configuration data as the target character style of the character. That is, for each character, the character style corresponding to the target configuration data corresponding to the character is directly determined as the target character style of the character. In this way, the determination of the target character style of each character can ensure that each character is consistent with the target image when restored, and the restoration degree of character layout is improved.

[0123] Optionally, since the same character format is generally used for characters in the same line during layout to ensure the beauty of the layout, the determining the target character style of the character according to the character style corresponding to the target configuration data can also be: for each character, determining the character style corresponding to the target configuration data as the initial character style of the character; counting the initial character styles of the characters in the target line where the character is located; and determining the initial character style with the largest number of corresponding characters in the target line as the target character style of the character. That is, for each character, the character style corresponding to the target configuration data corresponding to the character is first determined as the initial character style of the character; and the initial character styles of the characters in the character line (target line) where the character is located are traversed, and the most initial character style matched in the target line is determined as the target character style of all characters in the target line. In this way, the target character style of the character is determined in units of lines, which can effectively avoid the problem of error in the target character style of a single character caused by error in the matching of the single character, and further improve the restoration degree of character layout.

[0124] In addition, in addition to determining the target character style of the character in units of lines, for the case that the character layout in the target image is vertical, the target character style of the character can also be determined in units of columns, and the specific process is the same as that of determining the target character style of the character in units of lines, which will not be described here.

[0125] Optionally, to ensure the beauty of the layout, the same font is generally used for characters in the same line, but to highlight some content, some fonts in the same line can use different font shapes, such as bold, italic, etc. Therefore, the target font of each character can be determined in units of lines, and then the target font shape of each character is determined. That is, the character style includes font and font shape; accordingly, the determining the target character style of the character according to the character style corresponding to the target configuration data can be implemented as follows:

[0126] determining the font corresponding to the target configuration data as the initial font of the character;

[0127] counting the initial fonts of the characters in the target line where the character is located;

[0128] The initial font with the largest number of corresponding characters in the target line is determined as the target font of the character, and the glyph corresponding to the target configuration data is determined as the target glyph of the character.

[0129] Specifically, the target line is also the character line where the character is located.

[0130] In actual application, for each character, the font corresponding to the target configuration data corresponding to the character is taken as the initial font of the character; the initial fonts of the characters in the character line (target line) where the character is located are traversed, and the most matched initial font in the target line is taken as the target font of all characters in the target line. Meanwhile, for each character, the glyph corresponding to the target configuration data of the character is determined as the target glyph of the character. In this way, the target font of the character is determined in units of lines, which can effectively avoid the error of matching a single character and causing the error of the target font of the character, and the target glyph of each character is determined one by one, which can ensure that each character is consistent with the target image when restored, and further improve the restoration degree of character layout.

[0131] It should be noted that the premise of character layout restoration is to set a configuration data set for each character style in advance. That is, before the target configuration data of each character is filtered from the configuration data set of each character style according to the character and the information of the circumscribed rectangle of the character, the following steps are further included.

[0132] For each character style, starting from the first character corresponding to the character style, the configuration data corresponding to the current character is obtained until the last character corresponding to the character style, and the configuration data set of the character style is obtained.

[0133] In actual application, for each character style, starting from the first character, the configuration data of the first character under the current character style is obtained, and then the configuration data of the second character under the current character style is obtained, and so on, until the last character. The configuration data of each character under the current character style forms the configuration data set of the current character style. All character styles are traversed to obtain the configuration data set of each character style.

[0134] It should be noted that for characters without font, such as space characters and tab characters, there is no font, and therefore there is no width, height, etc., resulting in no configuration data for such characters, i.e., no configuration data for characters without font.

[0135] In addition, the configuration data set can be obtained only for commonly used character styles, or all character styles can be traversed to obtain the configuration data set. The configuration data can be obtained for commonly used characters, or all characters can be traversed to obtain the configuration data.

[0136] Optionally, when obtaining the configuration data corresponding to the current character, the configuration data corresponding to the current character can be directly downloaded or obtained from the Internet or some web pages. In this way, the configuration data can be quickly obtained, thereby improving the efficiency of determining the configuration data set.

[0137] Optionally, the configuration data of the current character can also be collected character by character according to a set strategy. That is, before obtaining the configuration data corresponding to the current character, the method further includes:

[0138] setting a font size corresponding to the character style;

[0139] Correspondingly, the obtaining of the configuration data corresponding to the current character includes:

[0140] drawing the current character in a square with the font size as the side length according to the font data of the current character and the character style;

[0141] identifying a font scaling ratio, a position offset ratio, and a configuration font width-height ratio of the current character in the square;

[0142] generating the configuration data corresponding to the current character according to the current character, the font scaling ratio, the position offset ratio, and the configuration font width-height ratio.

[0143] In actual application, the configuration data is constructed based on the font height. At this time, a font size is set for each character style, and the size of the font size is Font_Size. For each character, a square is drawn with the Font_Size as the side length, and the character is drawn in the square according to the font data of the current character and the character style, to obtain a configuration character image. Then, the font scaling ratio, the position offset ratio, and the configuration font width-height ratio of the character font in the configuration character image (square) are identified, and the configuration data of the current character under the current character style is generated based on the current character and the font scaling ratio, the position offset ratio, and the configuration font width-height ratio of the current character. In this way, the font scaling ratio, the position offset ratio, and the configuration font width-height ratio corresponding to each character are collected one by one to generate the configuration data, which can improve the completeness and accuracy of the configuration data, thereby improving the accuracy of the configuration data set, and is beneficial to improve the reliability of character layout.

[0144] It should be noted that in the case of configuring data based on font width, when obtaining the configuration data corresponding to the current character, the configuration font width aspect ratio is changed to the configuration font height aspect ratio. That is, obtaining the configuration data corresponding to the current character includes: drawing the current character in a square with the font size as the side length according to the font data of the current character and the character style; identifying the font scaling ratio, the position offset ratio and the configuration font height aspect ratio corresponding to the current character in the square; and generating the configuration data corresponding to the current character according to the current character, the font scaling ratio, the position offset ratio and the configuration font height aspect ratio.

[0145] In an implementable embodiment of the present application, the configuration character image (a square with the current character drawn according to the font data of the current character and the character style) can be directly input into the pre-trained information extraction model, and the information extraction model outputs the font scaling ratio, the position offset ratio and the configuration font width aspect ratio corresponding to the current character. In this way, the recognition efficiency can be improved, and thus the efficiency of generating configuration data is improved.

[0146] In another implementable embodiment of the present application, the identification of the font scaling ratio, the position offset ratio and the configuration font width aspect ratio corresponding to the current character in the square can be implemented as follows:

[0147] Taking the upper left corner of the square as the origin, the minimum and maximum values of the character font of the current character in the horizontal direction and the minimum and maximum values in the vertical direction are determined;

[0148] The difference between the maximum and minimum values in the horizontal direction is determined as the width of the character font, and the difference between the maximum and minimum values in the vertical direction is determined as the height of the character font;

[0149] The font scaling ratio of the character font is determined according to the font size and the font size, and the font size is the height or the width; the position offset ratio of the character font is determined according to the minimum value and the font size; and the ratio of the width to the height is determined as the configuration font width aspect ratio of the character font.

[0150] In actual application, taking the upper left corner of the square as the origin, the horizontal direction as the X-axis and the vertical direction as the Y-axis, the minimum value (xMin) and the maximum value (xMax) of the character font of the current character in the horizontal direction (X-axis) and the minimum value (yMin) and the maximum value (yMax) in the vertical direction (Y-axis) are determined. The difference between xMax and xMin is determined as the width (Width) of the current character, and the difference between yMax and yMin is determined as the height (Height) of the current character.

[0151] The ratio of the font size to the height is determined as the font scaling ratio of the character, that is, Font_Size_Ratio=Font_Size / Height; the ratio of the negative value of the minimum value in the horizontal direction to the height is determined as the horizontal offset ratio, that is, X_Offset_Ratio=(-xMin) / Height; the ratio of the negative value of the minimum value in the vertical direction to the height is determined as the vertical offset ratio, that is, Y_Offset_Ratio=(-yMin) / Height; and the ratio of the width to the height is determined as the configuration font width-height ratio of the current character, that is, Width / Height_Ratio=Width / Height.

[0152] Optionally, in the case where the configuration data is constructed based on the font width, the configuration character image (a square in which the current character is drawn according to the font data and the character style of the current character) can be directly input to the pre-trained information extraction model, and the font scaling ratio, the position offset ratio and the configuration font width-height ratio corresponding to the current character are output by the information extraction model. In this way, the recognition efficiency can be improved, and thus the efficiency of generating configuration data is improved.

[0153] Optionally, the identification of the font scaling ratio, the position offset ratio and the configuration font width-height ratio corresponding to the current character in the square can be implemented as follows:

[0154] The minimum value and the maximum value of the character font of the current character in the horizontal direction and the minimum value and the maximum value in the vertical direction are determined with the upper left corner of the square as the origin.

[0155] The difference between the maximum value and the minimum value in the horizontal direction is determined as the width of the character font, and the difference between the maximum value and the minimum value in the vertical direction is determined as the height of the character font.

[0156] The font scaling ratio of the character font is determined according to the font size and the character font size, the character font size being the height or the width; the position offset ratio of the character font is determined according to the minimum value and the character font size; and the configuration font width-height ratio of the character font is determined according to the ratio of the width to the height.

[0157] In actual application, the minimum value (xMin) and the maximum value (xMax) of the character font of the current character in the horizontal direction (X axis) and the minimum value (yMin) and the maximum value (yMax) in the vertical direction (Y axis) are determined with the upper left corner of the square as the origin. The difference between xMax and xMin is determined as the width (Width) of the current character, and the difference between yMax and yMin is determined as the height (Height) of the current character.

[0158] The ratio of the size of the font to the width is determined as the font scaling ratio of the character, that is, Font_Size_Ratio=Font_Size / Width; the ratio of the negative value of the minimum value in the horizontal direction to the width is determined as the horizontal offset ratio, that is, X_Offset_Ratio=(-xMin) / Width; the ratio of the negative value of the minimum value in the vertical direction to the width is determined as the vertical offset ratio, that is, Y_Offset_Ratio=(-yMin) / Width; and the ratio of the height to the width is determined as the configuration font height-width ratio of the current character, that is, Height / Width_Ratio=Height / Width.

[0159] In this way, based on the position of the character in the square, the font scaling ratio, the position offset ratio and the configuration font height-width ratio (or the configuration font height-width ratio) are determined, which can improve the accuracy of recognition and thus improve the accuracy of the generated configuration data.

[0160] The method for reducing the character layout in the image provided by the application will be further described below. Figure 2 The method for reducing the character layout in the image provided by the application will be further described below.

[0161] Step 1: Set the first character style, that is, the first font + style (font), and mark it as A font + style. For example, "Songti + Regular", which means using the "Songti" font library and the style is Regular, not inclined and not bold.

[0162] Step 2: Set the size (pixel size) of the font corresponding to the A font + style, that is, the font size is FontSize. In order to ensure the calculation accuracy, Font_Size is generally greater than or equal to 512, for example, set Font_Size as 1024.

[0163] Step 3: Enumerate the first character from the font library.

[0164] Step 4: Draw a square black and white image (configuration character image) with the font data of the character and using the style specified in step 1, with the upper left corner as the origin (0, 0) point and the side length as Font_Size pixel points, the white color as the background color and the black color as the color of the character, as shown in Figure 2 . Figure 2 is a schematic diagram of the configuration data provided by the application.

[0165] Step 5: Scan the pixel values of the fixed size image (configuration character image) row by row, and record the minimum value of the black pixels in the horizontal direction (X axis) as xMin and the maximum value as xMax; the minimum value in the vertical direction (Y axis) as yMin and the maximum value as yMax. As shown in Figure 2 .

[0166] Step 6: If no black pixel is scanned, it means that the character has no font, such as space, carriage return, etc. characters, then start from Step 4 to enumerate the next character until all characters in the font library are enumerated.

[0167] Step 7: Design a data table, i.e. a configuration data set (based on font height), to store the character restoration data calculated according to Step 4 and the following steps. The fields of the table are as follows:

[0168] 1) Character (Char): The encoding can be UTF-8, UTF-16 or GB18030;

[0169] 2) Font scaling ratio (Font_Size_Ratio): The font size is scaled based on the height or width of the character font;

[0170] 3) Horizontal offset ratio (X_Offset_Ratio): The display of the character is offset in the X axis based on the height or width of the character font;

[0171] 4) Vertical offset ratio (Y_Offset_Ratio): The display of the character is offset in the Y axis based on the height or width of the character font;

[0172] 5) Configuration font width / height ratio (Width / Height_Ratio): The ratio of the width to the height of the character font.

[0173] 6) Configuration character image (Image_Sample): Store the font image data drawn in Step 4.

[0174] Step 8: If not created, create a table file marked with the font + style for the font + style, marked as A table file.

[0175] Step 9: Append a row to the A table file, marked as A row.

[0176] Step 10: Write the character to the Char field of the A row in the A table file, and store the font image data drawn in Step 4 to the Image_Sample field of the A row.

[0177] Step 11: Record Width=xMax-xMin; Height=yMax-yMin.

[0178] Step 12:

[0179] 1) Write the value of Font_Size / Height to the Font_Size_Ratio field of the A row;

[0180] 2) Write the value of (-xMin) / Height into the X_Offset_Ratio field of row A;

[0181] 3) Write the value of (-yMin) / Height into the Y_Offset_Ratio field of row A;

[0182] 4) Write the value of Width / Height into the Width / Height_Ratio field of row A.

[0183] Referring to the configuration data of the character "j" in Figure 2 , the font scaling ratio = font size / height, the horizontal offset ratio = (-xMin) / height, the vertical offset ratio = (-yMin) / height, and the configuration font aspect ratio = width / height.

[0184] Step 13: Restart enumerating the next character in the font library from step 4 until all characters in the font library are enumerated.

[0185] Step 14: Set the second character style, i.e., the second font + style (font), marked as B font + style, and execute from step 2 until all possible font + styles (character styles) are executed.

[0186] Step 15: Create a character restoration data table (configuration data set) for each font + style and import it into the following OCR recognition system for use.

[0187] Step 16: When the OCR recognition system starts, load all the character restoration data tables of all the above created font + styles, and mark the font name + style for each data table according to the file name.

[0188] Step 17: OCR recognizes the target image and obtains all the OCR character information recognized in the target image, each character information generally including at least the character and the bounding rectangle information of the character image, and the bounding rectangle information including the rectangular character image and the rectangular size.

[0189] Step 18: Restore the display of the character in the output of the restoration version of the PDF, OFD, etc. file, or the correction of the OCR recognized text, etc.

[0190] Step 19: Take the first character recognized by OCR in step 17 as the current display character, marked as A character OCR result (A character bounding rectangle information).

[0191] Step 20: Find the restoration information data (initial configuration data) of the current display character from the character restoration data table of the first font + style using Char as the key, marked as A character restoration data.

[0192] Step 21: Calculate the ratio of the width and height of the rectangle circumscribing the image in the OCR result of the A character, and mark it as OCR_Width / Height_Ratio.

[0193] Step 22: Calculate the ratio of OCR_Width / Height_Ratio and the value of the Width / Height_Ratio field in the A character restoration data as Width / Height_RatioA.

[0194] Step 23: Starting from step 20 to step 22, use the next font + style character restoration data table, find N font + styles closest to 1 in the ratio Width / Height_RatioA, N is a positive integer.

[0195] Step 24: According to the rectangular region in the A character OCR result, cut out the image in this region from the original image, which is the image of the character in the original image, mark it as the A character original font image (rectangular character image).

[0196] Step 25: Take the character image data (configuration character image) of the Image_Sample field of the current display character in the N font + styles matched in step 23, and compare it with the A character original font image respectively (calculate the image similarity), find the font + style corresponding to the highest image similarity, mark it as "restored font + style B", and the restoration information data of the current display character in the restoration information data table of the restored font + style B is marked as current display character restoration data B.

[0197] Step 26:

[0198] 1) Calculate the value of Font_Size_Ratio of current display character restoration data B multiplied by the value of the rectangular height in the A character OCR result, and mark it as Font_SizeB;

[0199] 2) Calculate the value of X_Offset_Ratio of current display character restoration data B multiplied by the value of the rectangular height in the A character OCR result, and mark it as x_OffsetB;

[0200] 3) Calculate the value of Y_Offset_Ratio of current display character restoration data B multiplied by the value of the rectangular height in the A character OCR result, and mark it as y_OffsetB;

[0201] Step 27: Set the display font to the font in "restored font + style B", and the character display style to the style specified in "restored font + style B".

[0202] Step 28: Set the font size as Font_SizeB.

[0203] Step 29: Set the left value of the rectangle in the OCR result of the A character (the distance between the left side of the bounding rectangle and the left side of the target image, i.e., the relative position of the left side) plus x_OffsetB as the top-left x-coordinate of the currently displayed character; set the top value of the rectangle in the OCR result of the A character (the distance between the top side of the bounding rectangle and the top side of the target image, i.e., the relative position of the top side) plus y_OffsetB as the top-left y-coordinate of the currently displayed character.

[0204] Step 30: Display the currently displayed character.

[0205] Step 31: Take the second character recognized by OCR as the currently displayed character, and continue from step 18 until all characters recognized by OCR are displayed.

[0206] In this way, the font, size, and horizontal and vertical offsets required when displaying the original character are deduced based on the bounding rectangle information of each character, achieving a method of accurately restoring the original layout effect of the characters in the image.

[0207] For example:

[0208] Step 1: Assume that the character format of the text in the target image to be recognized is only "SimSun + Regular".

[0209] Step 2: Enumerate the first character from the SimSun font library.

[0210] Step 3: Use the font data of the character to draw a square black and white image (configure character image) with a top-left corner as the (0, 0) origin and a side length of 1024 pixels in Regular style, with white as the background color and black as the character color.

[0211] Step 4: Scan the pixel values of the configure character image row by row, and record the minimum value of the black pixels in the horizontal direction as xMin and the maximum value as xMax; the minimum value in the vertical direction as yMin and the maximum value as yMax.

[0212] Step 5: If no black pixels are scanned, it means that the character has no font, so start enumerating the next character from step 3 until all characters in the SimSun font library are enumerated.

[0213] Step 6: Set up the data table (configuration data set) corresponding to "Songti+Regular", which only includes the following fields: Char, Font_Size_Ratio, X_Offset_Ratio, Y_Offset_Ratio, Width / Height_Ratio. In this embodiment, the Image_Sample field is not included.

[0214] Step 7: If the table file marked with "Songti+Regular" has not been created, create a new table file marked with "Songti+Regular".

[0215] Step 8: Append a row to the table file marked with "Songti+Regular", and mark it as row A.

[0216] Step 9: Write the restoration data (configuration data) of the character into the Char field, Font_Size_Ratio field, X_Offset_Ratio field, Y_Offset_Ratio field, and Width / Height_Ratio field of row A, respectively.

[0217] Step 10: Start enumerating the next character from step 3 until all characters in the font library have been enumerated.

[0218] Step 11: After the above steps, the restoration data table (configuration data set) of "Songti+Regular" has been created and imported into the OCR recognition system for use.

[0219] Step 12: When the OCR recognition system starts, load the restoration data table of "Songti+Regular".

[0220] Step 13: OCR recognizes the target image and obtains all OCR character information recognized in the target image. Each character information generally includes at least the character and the bounding rectangle information of the character image, which includes the rectangular character image and the rectangular size.

[0221] Step 14: Output the OFD file of the restoration version and start displaying the restored characters.

[0222] Step 15: Take the first character obtained in the restoration step 13 as the current display character, and mark it as character A OCR result (the bounding rectangle information of character A).

[0223] Step 16: Match the character code value in the character A OCR result with the value in the Char field of the restoration data table of "Songti+Regular" to find the restoration information data of the current display character, and mark it as character A restoration data R.

[0224] Since only the configuration data set of "Song Ti+Regular" exists, the A character restoration information data in the configuration data set is the target configuration data.

[0225] Step 17: Calculate Font_SizeR, x_OffsetR and y_OffsetR of the A character according to the A character restoration data R.

[0226] Step 18: Set the display font as "Song Ti" and the character display style as "Regular".

[0227] Step 19: Set the font size as Font_SizeR.

[0228] Step 20: Set the left upper corner x coordinate of the current display character as the left value of the rectangle in the OCR character information plus x_OffsetR, and set the left upper corner y coordinate of the current display character as the top value of the rectangle in the OCR character information plus y_OffsetR.

[0229] Step 21: Display the current display character.

[0230] Step 22: Take the next character obtained in step 13 as the current display character, and continue to execute from step 16 until all characters are displayed.

[0231] The method for restoring the character layout in the image provided by the present application is verified, and the target image is as shown in Figure 3 , Figure 3 is the target image provided by the present application, i.e. the original image for OCR recognition. The existing technology target image is recognized, and the recognized text is restored into an OFD format document, as shown in Figure 4 , Figure 4 is one of the schematic diagrams of the character layout in the restored image provided by the present application, and there are problems such as different fonts, different font sizes, different font shapes and disordered positions between the characters, letters or punctuation marks. The target image is recognized by using the method for restoring the character layout in the image provided by the present application, and the recognized text is restored into an OFD format document, as shown in Figure 5 , Figure 5 is another schematic diagram of the character layout in the restored image provided by the present application, and the fonts, shapes and positions of the characters, letters or punctuation marks are uniform. It can be seen from the comparison that, by comparing the original image for OCR recognition, the OFD file without restoration output and the OFD file output by using the method for restoring the character layout in the image provided by the present application, it can be seen that the layout effect of the restored OFD file is closer to that of the target image by using the method for restoring the character layout in the image provided by the present application.

[0232] The device for restoring character layout in an image provided by the present application is described below. The device for restoring character layout in an image described below can be referred to the method for restoring character layout in an image described above.

[0233] Figure 6 FIG. 1 is a structural schematic diagram of the device for restoring character layout in an image provided by the present application. As shown in FIG. 1, the device 600 for restoring character layout in an image comprises an identification module 601, a screening module 602, a determination module 603 and a restoration module 604, wherein: Figure 6

[0234] The identification module 601 is configured to perform image recognition on a target image to obtain each character in the target image and the circumscribed rectangle information of each character.

[0235] The screening module 602 is configured to, for each character, screen the target configuration data of the character from the configuration data set of each character style according to the character and the circumscribed rectangle information of the character.

[0236] The determination module 603 is configured to determine the target character style of the character according to the character style corresponding to the target configuration data, and determine the target font size and position information of the character according to the target configuration data and the circumscribed rectangle information.

[0237] The restoration module 604 is configured to restore the character layout of each character in the target image according to the target character style, target font size and position information of each character.

[0238] The device for restoring character layout in an image provided by the present application can reversely deduce the target character style (font and shape), target font size (font size) and position information (such as the displacement of the character position in the horizontal direction or the vertical direction) of each character in the target image through the recognized each character in the target image and the circumscribed rectangle information of each character, in combination with the configuration data set of each character style, so that the original layout effect of the characters in the target image can be restored more accurately, and the accuracy and reliability of the restored character layout can be improved.

[0239] Optionally, the determination module 603 is further configured to:

[0240] determine the target font size of the character according to the font scaling ratio in the target configuration data and the rectangle size in the circumscribed rectangle information;

[0241] determine the position information of the character according to the position offset ratio in the target configuration data and the rectangle size in the circumscribed rectangle information.

[0242] Optionally, the determination module 603 is further configured to:​

[0243] determine a position offset of the character according to a position offset ratio in the target configuration data and a rectangle size in the bounding rectangle information;

[0244] determine position information of the character according to the position offset and a relative position of the bounding rectangle in the target image in the bounding rectangle information.

[0245] Optionally, the determining module 603 is further configured to:

[0246] determine target font size and position information of the character according to the target configuration data and a height of the bounding rectangle in the bounding rectangle information.

[0247] Optionally, the screening module 602 is further configured to:

[0248] screen initial configuration data of the character respectively from a configuration data set of each character style, the configuration data set including configuration data of all characters;

[0249] determine target configuration data from each of the initial configuration data according to a similarity between the bounding rectangle information of the character and each of the initial configuration data.

[0250] Optionally, the bounding rectangle information includes a height and a width of the bounding rectangle, and a rectangular character image corresponding to the bounding rectangle.

[0251] Correspondingly, the screening module 602 is further configured to:

[0252] determine a target aspect ratio of the character according to the height and the width of the bounding rectangle.

[0253] calculate a ratio between the target aspect ratio and a configuration aspect ratio in each of the initial configuration data respectively, and screen a plurality of candidate configuration data from each of the initial configuration data according to the ratio.

[0254] calculate an image similarity between the rectangular character image and a configuration character image in each of the candidate configuration data respectively, and determine target configuration data from the plurality of candidate configuration data according to the image similarity.

[0255] Optionally, the character style includes a font and a font style.

[0256] Correspondingly, the determining module 603 is further configured to:

[0257] determine an initial font of the character as the font corresponding to the target configuration data.

[0258] acquire initial font of each character in the target line where the character is located;

[0259] determine the initial font corresponding to the largest number of characters in the target line as the target font of the character, and determine the glyph corresponding to the target configuration data as the target glyph of the character.

[0260] Optionally, the apparatus further comprises an acquisition module, which is further configured to:

[0261] For each character style, starting from the first character corresponding to the character style, acquire configuration data corresponding to the current character until the last character corresponding to the character style, to obtain a configuration data set of the character style.

[0262] Optionally, the acquisition module is further configured to:

[0263] set the font size corresponding to the character style;

[0264] draw the current character in a square with the font size as the side length according to the glyph data of the current character and the character style;

[0265] identify the glyph scaling ratio, the position offset ratio and the configuration glyph width-height ratio corresponding to the current character in the square;

[0266] generate configuration data corresponding to the current character according to the current character, the glyph scaling ratio, the position offset ratio and the configuration glyph width-height ratio.

[0267] Optionally, the acquisition module is further configured to:

[0268] determine the minimum value and the maximum value in the horizontal direction and the minimum value and the maximum value in the vertical direction of the character glyph of the current character with the top-left corner of the square as the origin;

[0269] determine the difference between the maximum value and the minimum value in the horizontal direction as the width of the character glyph, and determine the difference between the maximum value and the minimum value in the vertical direction as the height of the character glyph;

[0270] determine the glyph scaling ratio of the character glyph according to the font size and the font size of the character glyph, the font size of the character glyph being the height or the width; determine the position offset ratio of the character glyph according to the minimum value and the font size of the character glyph; and determine the configuration glyph width-height ratio of the character glyph as the ratio of the width to the height.

[0271] Figure 7 An example of a schematic diagram of the physical structure of an electronic device is shown in FIG. 1. Figure 7As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a method of restoring character layout in an image, the method including: performing image recognition on a target image to obtain each character in the target image and information of a circumscribed rectangle of the each character; for each character, according to the character and the information of the circumscribed rectangle of the character, respectively screening target configuration data of the character from a configuration data set of each character style; determining a target character style of the character according to a character style corresponding to the target configuration data, and determining target font size and position information of the character according to the target configuration data and the information of the circumscribed rectangle; and restoring character layout of the each character in the target image according to the target character style, the target font size, and the position information of the each character.

[0272] In addition, the logical instruction in the memory 730 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0273] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer readable storage medium, and the computer program is executable by a processor to cause a computer to perform the method for restoring character layout in an image as described above, which comprises: performing image recognition on a target image to obtain each character in the target image and the bounding rectangle information of the each character; for each character, according to the character and the bounding rectangle information of the character, screening the target configuration data of the character from a configuration data set of each character style respectively; determining the target character style of the character according to the character style corresponding to the target configuration data, and determining the target font size and position information of the character according to the target configuration data and the bounding rectangle information; and restoring the character layout of the each character in the target image according to the target character style, the target font size and the position information of the each character.

[0274] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which is executable by a processor to implement the method for restoring character layout in an image as described above, which comprises: performing image recognition on a target image to obtain each character in the target image and the bounding rectangle information of the each character; for each character, according to the character and the bounding rectangle information of the character, screening the target configuration data of the character from a configuration data set of each character style respectively; determining the target character style of the character according to the character style corresponding to the target configuration data, and determining the target font size and position information of the character according to the target configuration data and the bounding rectangle information; and restoring the character layout of the each character in the target image according to the target character style, the target font size and the position information of the each character.

[0275] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0276] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0277] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of reducing the typesetting of characters in an image, characterized by, The method comprises: performing image recognition on a target image to obtain each character in the target image and the bounding rectangle information of each character; for each character, according to the character and the bounding rectangle information of the character, filtering target configuration data of the character from a configuration data set of each character style; determining a target character style of the character according to the character style corresponding to the target configuration data, and determining target font size and position information of the character according to the target configuration data and the bounding rectangle information; restoring the character layout of each character in the target image according to the target character style, the target font size and the position information of each character; the filtering target configuration data of the character from a configuration data set of each character style according to the character and the bounding rectangle information of the character comprises: filtering initial configuration data of the character from a configuration data set of each character style, wherein the configuration data set includes configuration data of all characters; determining target configuration data from each initial configuration data according to the similarity between the bounding rectangle information of the character and each initial configuration data; the determining target font size and position information of the character according to the target configuration data and the bounding rectangle information comprises: determining the target font size of the character according to the font scaling ratio in the target configuration data and the rectangle size in the bounding rectangle information, wherein the font represents the outline of the font; determining the position information of the character according to the position offset ratio in the target configuration data and the rectangle size in the bounding rectangle information; the determining the position information of the character according to the position offset ratio in the target configuration data and the rectangle size in the bounding rectangle information comprises: determining the position offset of the character according to the position offset ratio in the target configuration data and the rectangle size in the bounding rectangle information; determining the position information of the character according to the position offset and the relative position of the bounding rectangle in the target image in the bounding rectangle information.

2. The method of reducing the setting of characters in an image according to claim 1, characterized in that, The bounding rectangle information includes the height and width of the bounding rectangle, and the rectangular character image corresponding to the bounding rectangle; correspondingly, the determining target configuration data from each initial configuration data according to the similarity between the bounding rectangle information of the character and each initial configuration data comprises: determining the target font aspect ratio of the character according to the height and width of the bounding rectangle; calculating the ratio between the target font aspect ratio and the configuration font aspect ratio in each initial configuration data, and filtering a plurality of candidate configuration data from each initial configuration data according to the ratio; calculating the image similarity between the rectangular character image and the configuration character image in each candidate configuration data, and determining target configuration data from the plurality of candidate configuration data according to the image similarity.

3. The method of reducing the composition of characters in an image of claim 1, wherein, The character style includes font and font, and the font represents the style of the font; correspondingly, the determining a target character style of the character according to the character style corresponding to the target configuration data comprises: determining a font corresponding to the target configuration data as an initial font of the character; counting initial fonts of each character in a target line where the character is located; determining an initial font corresponding to the largest number of characters in the target line as a target font of the character, and determining a glyph corresponding to the target configuration data as a target glyph of the character.

4. The method of reducing the composition of characters in an image of claim 1, wherein, Before the step of selecting the target configuration data of the character from the configuration data set of each character style according to the character and the information of the circumscribed rectangle of the character, the method further comprises: for each character style, starting from a first character corresponding to the character style, obtaining configuration data corresponding to a current character until a last character corresponding to the character style, to obtain a configuration data set of the character style.

5. The method of reducing the composition of characters in an image according to claim 4, wherein, Before the step of obtaining the configuration data corresponding to the current character, the method further comprises: setting a font size corresponding to the character style. Correspondingly, the step of obtaining the configuration data corresponding to the current character comprises: drawing the current character in a square with the font size as the side length according to the font data of the current character and the character style; identifying a font scaling ratio, a position offset ratio and a configuration glyph width-height ratio corresponding to the current character in the square; generating configuration data corresponding to the current character according to the current character, the font scaling ratio, the position offset ratio and the configuration glyph width-height ratio.

6. The method of reducing the composition of characters in an image according to claim 5, wherein, The step of identifying the font scaling ratio, the position offset ratio and the configuration glyph width-height ratio corresponding to the current character in the square comprises: determining minimum and maximum values of a character font of the current character in a horizontal direction and minimum and maximum values in a vertical direction, with the upper left corner of the square as the origin; determining a width of the character font as a difference between the maximum and minimum values in the horizontal direction, and determining a height of the character font as a difference between the maximum and minimum values in the vertical direction; determining a font scaling ratio of the character font according to the font size and a character font size, the character font size being the height or the width; determining a position offset ratio of the character font according to the minimum value and the character font size; and determining a configuration glyph width-height ratio of the character font as a ratio of the width to the height.

Images