Method and device for estimating distribution of character height in document image

Abstract

The present invention provides a method and device for estimating the distribution of character height in a document image, the method comprising: obtaining a first sample training set; training an initial network model based on the first sample training set to obtain a character height detection model; scaling the document image to be detected to different ratios and inputting it into the character height detection model to obtain character height recognition results corresponding to the document image to be detected at different ratios; establishing a character height distribution map for the document image to be detected at each ratio based on the obtained character height recognition results corresponding to the document image to be detected at different ratios; inputting the character height distribution map to be classified into a classifier model to obtain the optimal scaling ratio of the document image to be detected, and determining the character height distribution of the document image to be detected based on the character height corresponding to the optimal scaling ratio. The method can accurately detect the character height distribution on the document image.

Description

Technical Field

[0001] The present invention relates to the field of computer information technology, and in particular to a method and device for estimating the distribution of character heights in a document image. Background Art

[0002] At present, in the application scenarios of electronic document reading and OCR recognition, it is necessary to solve the problem of how to scale images to a size that suits the visual sense. Specifically, in the input preprocessing stage of OCR, the image needs to be scaled to a suitable size, which can significantly improve the accuracy of OCR recognition; and when reading PDF pages with a reader or browsing text web pages in a browser, scaling the page image to a suitable position and adjusting the text size on the page to a size suitable for reading can improve the user's reading experience.

[0003] When scaling existing PDF pages or text-based web page images, users usually scale the text-containing images they are reading or browsing to the size they want according to their reading preferences. Since existing methods cannot accurately identify the height distribution of text in an image, it is difficult to ensure that the image is scaled at a better scaling ratio during the scaling process. Therefore, how to accurately detect the height distribution of text on an image is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the present invention provides a method and device for estimating the distribution of character heights in a document image to solve one or more problems existing in the prior art.

[0005] According to one aspect of the present invention, the present invention discloses a method for estimating the distribution of character heights in a document image, the method comprising:

[0006] Obtaining a first sample training set, wherein the sample data in the first sample training set includes a text block image and a text height;

[0007] Training the initial network model based on the first sample training set to obtain a character height detection model;

[0008] Scaling the document image to be detected to different scales and inputting the scales to the character height detection model to obtain character height recognition results corresponding to the document image to be detected at different scales;

[0009] Based on the obtained character height recognition results corresponding to the document images to be detected at different scales, a character height distribution graph to be classified for the document images to be detected at each scale is established; wherein the horizontal axis of the character height distribution graph to be classified represents the actual character height, and the vertical axis represents the ratio of the number of characters with corresponding character height to the total number of characters;

[0010] The character height distribution map to be classified is input into the classifier model to obtain the optimal scaling ratio of the document image to be detected, and the character height distribution of the document image to be detected is determined based on the character height corresponding to the optimal scaling ratio.

[0011] In some embodiments of the present invention, obtaining a first sample training set includes:

[0012] Acquire a document image, cut the document image into a plurality of text block images, and mark the text height of each of the text block images;

[0013] The characters cut in the height direction in the character block image are subjected to mosaic processing.

[0014] In some embodiments of the present invention, obtaining a first sample training set further includes:

[0015] Randomly generate a document image, and add identification interference items to each area of ​​the generated document image, wherein the identification interference items include at least one of pure white background, Gaussian noise, crystal, salt and pepper noise, and real environment information; and / or

[0016] Data enhancement and rotation are performed on the text block image, and the brightness, contrast, saturation and hue of the text block image are adjusted.

[0017] In some embodiments of the present invention, the method comprises:

[0018] Construct a height loss function, which is: Among them, HLoss represents height loss, H1=min(D1,D3)+min(D2,D4), H2=D1+D2+D3+D4-H1, D1 is the distance between the predicted pixel point and the top of the text block image, D2 is the distance between the predicted pixel point and the bottom of the text block image, D3 is the distance between the labeled pixel point and the top of the text block image, and D4 is the distance between the labeled pixel point and the bottom of the text block image.

[0019] In some embodiments of the present invention, when obtaining the first sample training set includes performing data enhancement and rotation on the text block image, the method further includes: constructing an angle loss function, wherein the angle loss function is angleLoss=1-cos(θ1-θ2); wherein angleLoss represents angle loss, θ1 is the predicted rotation angle value of the text block image, and θ2 is the rotation angle value of the annotated text block image.

[0020] In some embodiments of the present invention, the character height distribution map is input into a classifier model to obtain an optimal scaling ratio of the document image to be detected, and the character height distribution of the document image to be detected is determined based on the character height corresponding to the optimal scaling ratio, before that, the method further includes:

[0021] Acquire a second sample training set, wherein the sample data in the second sample training set includes a text height distribution map and a corresponding optimal scaling ratio;

[0022] The classifier model is obtained by training an initial classifier model based on the second sample training set.

[0023] In some embodiments of the present invention, the method further comprises:

[0024] Construct a text recognition loss function, the text recognition loss function is Among them, DiceLoss represents the text recognition loss, X represents the probability that the labeled pixel point is text, and Y represents the probability that the predicted pixel point is text.

[0025] In some embodiments of the present invention, the classifier model is a SVM classifier; and / or

[0026] Based on the obtained text height recognition results corresponding to the document images to be detected at different scales, a text height distribution map to be classified for the document images to be detected at each scale is established, including:

[0027] Counting the number of characters of each character height corresponding to the document image to be detected at each ratio;

[0028] Calculate the ratio of the number of characters of each character height corresponding to the document image to be detected under each ratio to the total number of characters;

[0029] Based on the ratio, a drawing tool is used to establish a height distribution diagram of the characters to be classified in the document image to be detected at each ratio.

[0030] According to another aspect of the present invention, a system for estimating the distribution of text height in a document image is also disclosed. The system includes a processor and a memory, wherein the memory stores computer instructions. The processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described in any of the above embodiments.

[0031] According to yet another aspect of the present invention, a computer-readable storage medium is disclosed, on which a computer program is stored. When the program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0032] The method for estimating the distribution of text height in a document image constructs a large number of document image text blocks as the source of the network training set, obtains the height of the text in the document image to be detected through a character height detection model, and then obtains the optimal scaling ratio of the document image to be detected through a classifier model, thereby accurately detecting the text height distribution on the image, thereby ensuring that the image with text is scaled to the optimal ratio in document reading or OCR recognition scenarios.

[0033] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0034] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. The components in the drawings are not drawn to scale, but are only for illustrating the principles of the present invention. In order to facilitate the illustration and description of some parts of the present invention, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present invention. In the drawings:

[0036] Figure 1 The figure is a flow chart of a method for estimating the distribution of character heights in a document image according to an embodiment of the present invention.

[0037] Figure 2 It is a schematic flow chart of a method for estimating the distribution of character heights in a document image according to another embodiment of the present invention.

[0038] Figure 3 The figure is a schematic diagram of the construction process of the first sample training set according to an embodiment of the present invention.

[0039] Figure 4 FIG. 4 is a schematic diagram of a text block image according to an embodiment of the present invention.

[0040] Figure 5 FIG. 4 is a schematic diagram of a network structure of a character height detection model according to an embodiment of the present invention.

[0041] Figure 6 FIG. 4 is a schematic diagram of the height of characters in a character block image according to an embodiment of the present invention.

[0042] Figure 7 This is a height distribution diagram of characters to be classified according to an embodiment of the present invention.

[0043] Figure 8 FIG. 4 is a schematic diagram of character height lines in a character block image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0045] It should be noted here that in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0046] It should be emphasized that the terms “include / comprises / has” when used herein refer to the presence of features, elements, steps or components, but do not exclude the presence or addition of one or more other features, elements, steps or components.

[0047] In order to solve the problem of scaling document images to a suitable size in situations such as OCR recognition and document image reading, the present invention provides a method for estimating the distribution of text height in a document image. A large number of text block images are constructed as training data in a first sample data set, and the height of the text in the document image to be detected is obtained through a character height detection model. Then, the distribution of the page text height is estimated based on the obtained height to obtain the most suitable scaling ratio of the page.

[0048] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0049] Figure 1 FIG. 1 is a flow chart of a method for estimating the distribution of character heights in a document image according to an embodiment of the present invention. Figure 1 As shown, the method for estimating the distribution of character heights in a document image at least includes steps S10 to S50.

[0050] Step S10: Obtain a first sample training set, wherein the sample data in the first sample training set includes a text block image and a text height.

[0051] In this step, the first sample training set is used to train the initial network model; the first sample training set includes multiple sample data, but each sample data includes at least a text block image and a text height, and the text heights of the multiple sample data may be partially the same or all different. The text block image is shown in 4, Figure 4 What is shown in the figure are multiple text block images, that is, the text blocks of suitable size for training are obtained by cropping the document image. It should be understood that the number of text block images cropped from the document image is not specifically limited and can be set according to the actual application scenario; for example, the document image can be cropped into multiple 128*128 text block images. In addition, Figure 4 It is only for illustrating the expression form of the characters in the character block image of the present invention, and the specific content of the characters in each character block image is not within the research scope of the present invention.

[0052] Exemplarily, obtaining the first sample training set includes: obtaining a document image, cutting the document image into a plurality of text block images, marking the text height of each of the text block images; mosaic processing the text cut in the height direction in the text block image. In this embodiment, the content in the document image includes Chinese and English characters, uppercase and lowercase letters, punctuation marks, etc. Specifically, before obtaining the document image, it also includes randomly generating a document image, and the randomness of the generated document image is manifested in random Chinese and English characters, uppercase and lowercase letters, font size, font, and color, and various punctuation marks are randomly generated.

[0053] In the above embodiment, when the document image is cut, the text may be cut; however, in the present invention, the font height detection is required, so only the text block needs to be mosaiced when it is cut on the vertical axis (text height). In order to determine whether the text height in the text block image is cut, the coordinate information of each text block image can be obtained first, and the coordinates of the multiple text block images cut from the document image can be compared to determine whether the two text block images have overlapping areas in the vertical axis direction (text height direction). If so, it is determined that the text in the corresponding text block image is cut, and the corresponding text in the text block image is removed at this time, and the background color is covered; reflected on the image is to erase the text that is cut on the vertical axis in the image, and cover the background.

[0054] In addition, in order to increase the anti-interference ability of the first sample training set, identification interference items are added to each area of ​​the randomly generated document image, and the identification interference items include at least one of pure white background, Gaussian noise, crystal, salt and pepper noise and real environment information; the real environment information is the real environment image directly captured by the camera. Figure 4 , Figure 4 Some of the text block images in have pure white backgrounds, while some of the text block images have Gaussian noise added; Figure 4 In the figure, the blank area in each character block image is in a state after mosaic processing, that is, when the document image is cut, the characters cut in the height direction are covered with the background color.

[0055] Step S20: training the initial network model based on the first sample training set to obtain a character height detection model.

[0056] This step is to train the initial network model using the first sample training set constructed in step S10. The initial network model may specifically include four convolutional layers.

[0057] Specifically, before training the initial network model, the sample data used for training is preprocessed, that is, data enhancement and rotation of the text block image are performed, and the brightness, contrast, saturation and hue of the text block image are adjusted. In addition to the above, a height loss function is also constructed, and the height loss function is: Wherein, HLoss represents height loss, H1 = min(D1, D3) + min(D2, D4), H2 = D1 + D2 + D3 + D4 - H1, D1 is the distance between the predicted pixel point and the top of the text block image, D2 is the distance between the predicted pixel point and the bottom of the text block image, D3 is the distance between the marked pixel point and the top of the text block image, and D4 is the distance between the marked pixel point and the bottom of the text block image. Wherein, D3 and D4 can be calculated based on the size of the text block image and the height of the marked text. It should be understood that the pixel point refers to the pixel point corresponding to the top or bottom of the text in the text block image, because if you want to obtain the height of the text in the image, then under the premise that the size of the text block image is determined, the height of the corresponding text can be obtained based on the distance between the pixel point at the top of the text and the top boundary of the text block image, and the distance between the pixel point at the bottom of the text and the bottom boundary of the text block image.

[0058] In addition, after data enhancement and rotation of the text block image, it is further necessary to construct an angle loss function, which is angleLoss=1-cos(θ1-θ2); wherein angleLoss represents angle loss, θ1 is the predicted rotation angle value of the text block image, and θ2 is the marked rotation angle value of the text block image.

[0059] Furthermore, in this step, we need to construct a text recognition loss function, which is Among them, DiceLoss represents the text recognition loss, X represents the probability that the labeled pixel point is text, and Y represents the probability that the predicted pixel point is text.

[0060] Figure 5FIG. 1 is a schematic diagram of a network structure of a character height detection model according to an embodiment of the present invention. Figure 5 In the embodiment shown, the initial network model is trained based on the first sample training set to obtain a character height detection model, which specifically includes the following steps:

[0061] (1) Obtain sample data and perform preprocessing, perform data enhancement, adjust the height, and rotate the text block image. The rotated text block image is cropped to a uniform size that conforms to the input. Then, the text block image is transformed, that is, the brightness, contrast, saturation, and hue of the image are changed.

[0062] (2) Construct a height loss function. The height loss is related to the predicted merged area and the actually labeled area. The merged area refers to the common area of ​​the height lines detected in the text block image, such as Figure 8 The common part of the vertical lines in the height direction; the marked area refers to the actual area of ​​the text marked when the text block image is generated. Similarly, since only the text height information is concerned in the present invention, the loss function is also mainly the height loss, and the corresponding height loss calculation can also be reduced to one dimension, which can converge faster.

[0063] (3) Feature extraction of the text block image: Specifically, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the text block image are extracted as feature maps of different scales, which are marked as f1, f2, f3, and f4, respectively.

[0064] (4) Feature merging. That is, the deepest feature f1 is upsampled by 2 times and concatenated with f2, undergoes a 1x1 convolution to reduce the feature dimension, undergoes a 3x3 convolution and then upsampled by 2 times and concatenated with f3, and so on until the last feature f4 is concatenated and the result is output to the output layer.

[0065] (5) The output layer contains a score map corresponding to the probability that each pixel belongs to the text area, and the feature channels output by the output layer are 3, two of which represent the distance from the pixel position to the top and bottom boundaries of the text block image, and the other represents the rotation angle of the text block image.

[0066] (6) Perform local-aware NMS (non-maximum suppression). That is, all output text region sets are combined with corresponding thresholds (those greater than the threshold are merged, and those less than the threshold are not merged), and the confidence scores are used as weighted merges to obtain the merged text region set; the merged text region set is subjected to standard NMS operation.

[0067] (7) Save the trained character height detection model.

[0068] Step S30: scaling the document image to be detected to different scales and inputting the scales into the character height detection model to obtain character height recognition results corresponding to the document image to be detected at different scales.

[0069] In this step, the character height detection model trained in step S20 is used to identify the text height in the document image to be detected. Specifically, the document image to be detected can be first scaled to different proportions as required, and then the document images to be detected at different proportions are sequentially input into the character height detection model, thereby obtaining the text height information corresponding to the document image to be detected at different proportions. In one embodiment, the document image to be detected is specifically scaled at four proportions, such as 0.5 times, 1 times, 1.5 times and 2 times; at this time, step S30 can be used to obtain the text height information corresponding to the document image to be detected at four proportions. It is not difficult to understand that scaling the text image to be detected four times here is only an example, and text height detection can also be performed on document images to be detected at more or fewer proportions as needed.

[0070] Step S40: Based on the text height recognition results corresponding to the document images to be detected at different proportions, a text height distribution map to be classified for the document images to be detected at each proportion is established; wherein the horizontal axis of the text height distribution map to be classified represents the actual character height, and the vertical axis represents the ratio of the number of characters with the corresponding character height to the total number of characters.

[0071] In this step, a height distribution map of the characters to be classified is constructed based on the text height recognition results output by the height detection model. Specifically, the number of characters of each character height corresponding to the document image to be detected at each ratio can be counted first; the ratio of the number of characters of each character height corresponding to the document image to be detected at each ratio to the total number of characters can be calculated; and based on the ratio, a drawing tool can be used to establish a height distribution map of the characters to be classified for the document image to be detected at each ratio.

[0072] In this step, the character height value of the document image to be detected output by the character height detection model needs to be divided by the scaling ratio to obtain the actual character height. The drawing tool can be the plt toolkit, and if the document image to be detected is scaled by 0.5 times, 1 times, 1.5 times and 2 times, the height distribution diagram of the characters to be classified constructed by the plt toolkit is as follows: Figure 7 As shown, the horizontal axis represents the actual character height result based on the output of the character height detection model. Since the text block image to be detected recognized by the character height detection model is an image scaled by a certain ratio, the character height information output by the character height detection model is also the character height information after the character height is scaled. Therefore, the actual character height result is the actual character height calculated by the scaling ratio of the corresponding document image to be detected, and the vertical axis represents the ratio of the corresponding character height to the total number of characters.

[0073] Step S50: inputting the height distribution map of the characters to be classified into a classifier model to obtain the optimal scaling ratio of the document image to be detected, and determining the character height distribution of the document image to be detected based on the character height corresponding to the optimal scaling ratio.

[0074] This step is to select the most suitable scaling ratio of the image of the text block to be identified from the established height distribution graphs of the text to be classified by the classifier model. Before this step, the estimation method may also include the following steps: obtaining a second sample training set, wherein the sample data in the second sample training set includes the text height distribution graph and the corresponding optimal scaling ratio; training the initial classifier model based on the second sample training set to obtain the classifier model.

[0075] The second sample training set includes multiple sample data, each sample data includes a text height distribution map and a corresponding optimal scaling ratio, wherein each text height distribution map can be annotated to obtain the optimal scaling ratio in each text height distribution map. In one embodiment, 159 annotated text height distribution maps constitute the second sample training set. In addition, the classifier model used can be an SVM model. Specifically, an SVM classifier can be constructed based on sklearn.svm.SVC, with parameters kernel = "rbf" and constant C = 10.

[0076] When the optimal scaling ratio in the height distribution map of the text to be classified is obtained by the trained classifier model, the height distribution map of the text to be classified obtained in step S40 is first input into the SVM classifier model to obtain the optimal scaling ratios of 0.5 times, 1 times, 1.5 times and 2 times. The optimal scaling ratio means that the height detection result at this ratio is the most accurate, that is, the text reading and OCR recognition effects are the best at the font height corresponding to this ratio.

[0077] The above method is described below in conjunction with a specific embodiment. However, it should be noted that the specific embodiment is only for better illustrating the present application and does not constitute an improper limitation on the present application. Figure 3 As shown, in another embodiment, the method for estimating the distribution of character heights in a document image includes steps S101 to S105.

[0078] Step S101: construct text character height detection sample training.

[0079] This step can generate sample data of text block images with different text contents, different backgrounds, and different font sizes. Figure 1As shown, first, batches of document images with different text sizes, text colors, text contents (including punctuation marks, Chinese and English), and different backgrounds are generated; then the document images are cropped to obtain text block images of appropriate sizes for training; and mosaic processing is performed on the cropped text blocks.

[0080] Step S102: training a character height detection model.

[0081] In this step, a character height detection model is trained based on the sample data in step S101 and saved.

[0082] Specifically, a text block image is obtained, and data enhancement is performed on the text block image in terms of height, and the text block image is rotated and cropped. The brightness, contrast, saturation and hue of the text block image are changed; and a height loss function is constructed. Feature extraction is performed on the text block image. For example, four levels of feature maps can be extracted from the feature extraction stage based on the VGG16 model, and the sizes are 1 / 32, 1 / 16, 1 / 8, and 1 / 4 of the input image, respectively. Then, feature merging is performed in a layer-by-layer merging manner, that is, the feature map from the previous stage is first input to the upper pooling layer to expand its size, and then merged with the feature map of the current layer, and then the number of channels and the amount of calculation are reduced through conv1×1, and then the local information is fused through conv3×3 to finally generate the output of the merging stage. The output layer contains a probability map (the probability that each pixel belongs to the text area) and model output information (a total of 3 channels are output, of which 2 channels represent the distance from the pixel position to the top and bottom boundaries of the text block image, and the other channel represents the rotation angle of the text block image). Then, we perform local perception non-maximum suppression, and use the confidence scores as weights to weight the corresponding thresholds of all output text region sets to obtain the merged text region set, and perform standard non-maximum suppression on the merged text region set. Finally, we save the trained character height detection model.

[0083] Step S103: scaling the image document to different ratios and inputting it into the trained height detection model.

[0084] This step is to input the document images to be detected of different proportions into the trained character height detection model, and output the character height information corresponding to the text blocks to be detected of different proportions through the character height detection model.

[0085] Step S104: obtaining character height distribution maps corresponding to the document images to be detected of various proportions.

[0086] This step is to first count the proportion of characters of different character heights in the total number of characters with the help of statistical tools, and then create a character height distribution graph with the help of drawing tools.

[0087] Step S105: Inferring a suitable scaling ratio through the SVM classifier.

[0088] Specifically, a document image height distribution data set is constructed, each image is a document image height distribution at different scales, and the most appropriate scaling ratio is marked for each image; the document image height distribution data set is the second sample data set in the above embodiment. Then, an SVM classifier is trained, and the trained SVM classifier is saved. The text height distribution graphs of different scales of the document image to be detected are input into the SVM classifier to obtain the best scaling ratio.

[0089] Through the above embodiments, it can be found that the method for estimating the distribution of the height of characters in the document image first constructs a character height detection data set through an automated tool, and then uses the constructed data set to train the character height detection model. When in use, the document image to be detected is input into the trained character height detection model with different scaling ratios, and the character height information of the page at each scale is analyzed to obtain the character height distribution of the document image to be detected at each scale, and the character height distribution under different scaling ratios is compared. The most appropriate scaling ratio is selected according to certain rules, and finally the detected character height is divided by the corresponding scaling ratio to obtain the character height distribution of the page; based on this method, the accuracy of character height detection can be improved.

[0090] The document images to be detected input by this method include images converted from common document types such as Word, PDF, OFD, etc., as well as images of traditional paper carriers such as books and newspapers, and also include images obtained through screenshots on web pages; therefore, this method has a wide range of applications. In addition, this method can be applied to text reading scenarios, including browsers on electronic devices such as personal computers and mobile phones, and readers of various electronic documents; that is, when reading a document, by estimating the character height distribution of the document page image, it is convenient to scale the document page to a size more suitable for reading, avoiding situations where the font is too large or too small. In addition to the above, this method removes the width information of the text and only focuses on the height information, converting the two-dimensional output into a one-dimensional output to simplify the network, while completing the task of obtaining the character height, thereby improving the convergence ability of the network.

[0091] Accordingly, the present invention also discloses a system for estimating the distribution of text height in a document image, the system comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the method described in any of the above embodiments.

[0092] In addition, the invention also discloses a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0093] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted on a transmission medium or a communication link via a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, and the like. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0094] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0095] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for estimating the distribution of character heights in a document image, characterized in that: The method comprises: Obtaining a first sample training set, wherein the sample data in the first sample training set includes a text block image and a text height; Training the initial network model based on the first sample training set to obtain a character height detection model; Scaling the document image to be detected to different scales and inputting the scales to the character height detection model to obtain character height recognition results corresponding to the document image to be detected at different scales; Based on the obtained character height recognition results corresponding to the document images to be detected at different scales, a character height distribution graph to be classified for the document images to be detected at each scale is established; wherein the horizontal axis of the character height distribution graph to be classified represents the actual character height, and the vertical axis represents the ratio of the number of characters with corresponding character height to the total number of characters; The character height distribution map to be classified is input into the classifier model to obtain the optimal scaling ratio of the document image to be detected, and the character height distribution of the document image to be detected is determined based on the character height corresponding to the optimal scaling ratio.

2. The method for estimating the distribution of character heights in a document image according to claim 1, characterized in that: Get the first sample training set, including: Acquire a document image, cut the document image into a plurality of text block images, and mark the text height of each of the text block images; The characters cut in the height direction in the character block image are subjected to mosaic processing.

3. The method for estimating the distribution of character heights in a document image according to claim 2, characterized in that: Obtaining the first sample training set also includes: Randomly generate a document image, and add identification interference items to each area of ​​the generated document image, wherein the identification interference items include at least one of pure white background, Gaussian noise, crystal, salt and pepper noise, and real environment information; and / or Data enhancement and rotation are performed on the text block image, and the brightness, contrast, saturation and hue of the text block image are adjusted.

4. The method for estimating the distribution of character heights in a document image according to claim 3, characterized in that: The method comprises: Construct a height loss function, which is: Among them, HLoss represents height loss, H1=min(D1, D3)+min(D2, D4), H2=D1+D2+D3+D4-H1, D1 is the distance between the predicted pixel point and the top of the text block image, D2 is the distance between the predicted pixel point and the bottom of the text block image, D3 is the distance between the marked pixel point and the top of the text block image, and D4 is the distance between the marked pixel point and the bottom of the text block image.

5. The method for estimating the distribution of character heights in a document image according to claim 4, characterized in that: When obtaining the first sample training set includes performing data enhancement and rotation on the text block image, the method further includes: constructing an angle loss function, the angle loss function is angleLoss=1-cos(θ1-θ2); wherein angleLoss represents angle loss, θ1 is the predicted rotation angle value of the text block image, and θ2 is the marked rotation angle value of the text block image.

6. The method for estimating the distribution of character heights in a document image according to claim 1, characterized in that: The character height distribution map is input into a classifier model to obtain an optimal scaling ratio of the document image to be detected, and the character height distribution of the document image to be detected is determined based on the character height corresponding to the optimal scaling ratio, before that, the method further includes: Acquire a second sample training set, wherein the sample data in the second sample training set includes a text height distribution map and a corresponding optimal scaling ratio; The classifier model is obtained by training an initial classifier model based on the second sample training set.

7. The method for estimating the distribution of character heights in a document image according to claim 5, characterized in that: The method further comprises: Construct a text recognition loss function, the text recognition loss function is Among them, DiceLoss represents the text recognition loss, X represents the probability that the labeled pixel point is text, and Y represents the probability that the predicted pixel point is text.

8. The method for estimating the distribution of character heights in a document image according to any one of claims 1 to 7, characterized in that: The classifier model is a SVM classifier; and / or Based on the obtained text height recognition results corresponding to the document images to be detected at different scales, a text height distribution map to be classified for the document images to be detected at each scale is established, including: Counting the number of characters of each character height corresponding to the document image to be detected at each ratio; Calculate the ratio of the number of characters of each character height corresponding to the document image to be detected under each ratio to the total number of characters; Based on the ratio, a drawing tool is used to establish a height distribution diagram of the characters to be classified in the document image to be detected at each ratio.

9. A system for estimating the distribution of character heights in a document image, the system comprising a processor and a memory, characterized in that: The memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as claimed in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images