Method, apparatus, computer device, and storage medium for recognizing document font size

Abstract

The present application relates to a method, apparatus, computer device, storage medium, and computer program product for identifying document font sizes. By combining artificial intelligence technology for computer vision recognition and processing, it can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, and assisted driving. The method includes: obtaining a document picture to be processed, and detecting the initial line height of each text line in the document picture; determining at least one text block to be processed according to the document layout information of the document picture; determining line height statistical information according to the initial line height, and determining a target line height corresponding to the corresponding text block to be processed based on the line height statistical information; and obtaining the target font size corresponding to each text block in the document picture based on a pre-set font size mapping rule and the target line height corresponding to each text block to be processed. Using this method can solve the problem that the font sizes recognized for titles or paragraphs with the same original font size are inconsistent.

Description

Technical Field

[0001] The present application relates to the field of computer technologies, and particularly to a method, an apparatus, a computer device, a storage medium, and a computer program product for identifying the font size of a document. Background Art

[0002] Text recognition is one of the branches in the field of computer vision research, belonging to pattern recognition and artificial intelligence, and is an important part of computer science. Computer text recognition (Optical Character Recognition, OCR) is to use optical technology and computer technology to read the text printed or written on paper and convert it into a format that can be accepted by a computer and understood by people.

[0003] The existing technical solutions mainly detect text lines through OCR technology and directly obtain the font size of the text lines based on the recognition results of OCR. This method has a high dependence on the detection accuracy of text lines. The recognition effect of the font size of text lines has relatively large fluctuations, and the accuracy of the recognized font size is low, resulting in a situation where the original font sizes of the document are the same but the restored font sizes of the document are different when restoring the document, thereby affecting the accuracy of document restoration. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product for identifying the font size of a document in view of the above technical problems.

[0005] On the one hand, the present application provides a method for identifying the font size of a document. The method includes:

[0006] Obtain a document image to be processed, and detect the initial line height of each text line in the document image;

[0007] Identify the document layout information of the document image, and determine at least one text block to be processed according to the document layout information. Each text block to be processed includes at least one continuous text line;

[0008] For any text block to be processed, determine the line height statistical information according to the initial line height of the text lines included in the corresponding text block to be processed, and determine the target line height corresponding to the corresponding text block to be processed based on the line height statistical information;

[0009] Based on the pre-set font size mapping rule and the target line height corresponding to each text block to be processed, obtain the target font size corresponding to each text block in the document image.

[0010] On the other hand, the present application also provides an apparatus for identifying the font size of a document. The apparatus includes:

[0011] An acquisition module, configured to acquire a document image to be processed and detect an initial line height of each text line in the document image;

[0012] A processing module, configured to identify document layout information of the document image and determine at least one text block to be processed according to the document layout information, where each text block to be processed includes at least one consecutive text line;

[0013] The processing module is further configured to, for any text block to be processed, determine line height statistical information according to the initial line height of the text lines included in the corresponding text block to be processed, and determine a target line height corresponding to the corresponding text block to be processed based on the line height statistical information;

[0014] An output module, configured to obtain target font sizes corresponding to the respective text blocks in the document image based on a preset font size mapping rule and the target line heights corresponding to the respective text blocks to be processed.

[0015] On the other hand, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method for identifying the font size of a document are implemented.

[0016] On the other hand, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for identifying the font size of a document are implemented.

[0017] On the other hand, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for identifying the font size of a document are implemented.

[0018] In the above-mentioned method, device, computer device, storage medium, and computer program product for identifying the font size of a document, by detecting and identifying a document image, text lines and document layout information in the document image are obtained, and each text line is grouped according to the document layout information, so as to determine text blocks to be processed, whereby each group can be processed in parallel to improve efficiency and facilitate subsequent statistical analysis; line height statistical information is determined based on the initial line height of the text lines, and then a target line height is calculated based on the line height statistical information, so that the heights of the text lines in the text block are unified. Thus, when the target font size is obtained based on a preset font size mapping rule, the font sizes of the respective text lines obtained are also unified accordingly, thereby being able to solve the problem that the font sizes recognized for a title or a paragraph with the same original font size are different, and the accuracy of document restoration is higher. Description of the Drawings

[0019] Figure 1 It is an application environment diagram of the method for identifying the font size in a document in an embodiment;

[0020] Figure 2 It is a schematic flowchart of the method for identifying the font size in a document in an embodiment;

[0021] Figure 3 It is a schematic diagram of a line in a document in an embodiment;

[0022] Figure 4 It is a schematic diagram of a detection frame in an embodiment;

[0023] Figure 5 It is a schematic diagram of a text block included in a document picture in an embodiment;

[0024] Figure 6 It is a schematic diagram of calculating the line height statistical information in an embodiment;

[0025] Figure 7 It is a schematic architecture diagram of the method for identifying the font size in a document in an embodiment;

[0026] Figure 8 It is a schematic flowchart of the processing steps of the method for identifying the font size in a document in an embodiment;

[0027] Figure 9 It is a schematic diagram of marking the font size of a text block in an embodiment;

[0028] Figure 10 It is a structural block diagram of the device for identifying the font size in a document in an embodiment;

[0029] Figure 11 It is an internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0030] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] In fields such as document restoration and document reconstruction, it is usually necessary to convert a non-editable format document into an editable format document. The non-editable format is, for example, PDF (Portable Document Format), picture format, etc. The picture format includes, for example, JPG / JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), and TIF (Tag Image File Format), etc.

[0032] For PDF documents, in the existing methods, usually the PDF document is converted into document pictures page by page, and then the text in the document pictures is recognized by OCR technology. Corresponding text is input into a newly created empty document in editable format according to information such as the font size of the recognized text, and typesetting is performed according to the position of the recognized text. A similar processing method is also used for picture format documents. For the recognition of font size, in the existing methods, the height of the text is recognized based on the OCR recognition technology, and the font size of each text line is directly obtained according to the font size mapping rule. The readability and legibility of the editable document obtained in this way seriously depend on the accuracy of OCR recognition. Due to the diversity of the text content and form in the original document, there is a large fluctuation in the recognition effect of the font size. Often, the font sizes of the text in the original document are the same, but the font sizes in the editable document obtained by restoration or reconstruction are different, resulting in a decrease in the accuracy of document restoration and seriously affecting the readability and aesthetics of the restored document.

[0033] In view of this, the embodiments of this application provide a method for recognizing the font size of a document combined with artificial intelligence technology. By detecting and recognizing text lines in the document picture, and reorganizing each text line based on the document layout information to obtain a text block to be processed, and processing different text blocks according to different processing strategies, calculating the target line height of the text lines in the text block based on the line height statistical information of the text block, and then performing font size mapping and optimization correction based on the font size mapping rule, so as to obtain the font size of each text block in the actual document. Thus, by regularizing the height information of the originally possibly inconsistent and fluctuating text lines in the above manner, the appearance and style of the original document picture can be restored more appropriately, the overall font size restoration effect of the document can be optimized, and at the same time, the accuracy of document restoration is high, and the restored document has strong readability and aesthetics.

[0034] Among them, Artificial Intelligence (AI) uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, and is a theory, method, technology, and application system that can perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning, and decision-making.

[0035] Artificial intelligence technology is an interdisciplinary subject with a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0036] Among them, Computer Vision (CV) is a science that studies how to enable machines to "see". Further, it refers to using cameras and computers to replace human eyes for machine vision such as target recognition, tracking, and measurement, and further performing graphic processing to make the computer process images that are more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to establish artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0037] Among them, Machine Learning (ML) is an interdisciplinary subject across multiple fields, involving multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal teaching learning.

[0038] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0039] The following describes the technical solution of the method for identifying the font size of documents involving artificial intelligence provided by the embodiments of the present application.

[0040] The method for identifying the font size of documents provided by the embodiments of the present application can be applied to the application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or on other servers.

[0041] In some embodiments, the terminal 102 or the server 104 can separately execute the above method for identifying the font size of documents. For example, the terminal 102 or the server 104 obtains the document image to be processed, detects the text lines and the initial line height of the text lines contained therein, and extracts the text blocks and determines the text blocks to be processed by identifying the document layout information of the document image. Then, the target line height corresponding to the text block is obtained according to the line height statistical information calculated for the text block. Finally, the target font size corresponding to the text block is mapped based on the pre-set font size mapping rule. Thus, the unity of the font size can be ensured during the document restoration process according to the target font size.

[0042] In some embodiments, the terminal 102 and the server 104 can also cooperate to execute the above method for identifying the font size of documents. For example, the terminal 102 can transmit the document image to be processed to the server 104, and the server 104 performs corresponding processing and returns the target font size for the terminal 102 to perform document restoration; or, the server 102 performs document restoration based on the obtained target font size and sends the restored document to the terminal 102, etc.

[0043] Exemplarily, the terminal 102 can be loaded with an APP (Application) application program or an application program with document restoration function, including traditional application programs that need to be installed separately, and small program applications that can be used without downloading and installation. The terminal can perform document restoration processing and the like through the application program.

[0044] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc.

[0045] Among them, the server 104 can be an independent physical server, can also be a server cluster or a distributed system composed of multiple physical servers, and can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0046] The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, and assisted driving, etc.

[0047] In some embodiments, as Figure 2 shown, a method for identifying the font size of a document is provided. This method can be executed by a terminal or a server, or can be executed jointly by a terminal and a server. The following takes this method applied to a computer device (the computer device can specifically be Figure 1 the terminal or the server in) as an example for illustration, including the following steps:

[0048] Step S202, obtain a document image to be processed, and detect the initial line height of each text line in the document image.

[0049] Among them, the document image is a document in picture format, for example, it can be a document image obtained by converting a PDF document, etc. The document image can be obtained by the computer device on the Internet, or can be obtained by receiving a transmission from other computer devices. In some embodiments, the computer device can obtain the document image of the document by operating on an electronic version of the document, such as taking a photo, taking a screenshot, or performing format conversion, or by operating on a printed version of the document, such as photocopying or scanning.

[0050] Among them, a text line refers to the smallest unit of a combination of characters arranged in a certain rule in a document. A title, a line or a column of text, a header or a footer can all be a combination of characters. For example Figure 3 as shown in (a) in, for the entire paragraph of text in the document, each line of text is a text line. Of course, it is not limited to the form of horizontal arrangement of characters. For example Figure 3As shown in (b) therein, in some documents, the text is arranged in a vertical layout, and each column of text can also be considered as a line of text.

[0051] Specifically, after the computer device obtains the document picture, it performs text detection on the document picture and marks the detected text lines with detection frames. The initial line height of the text line is the height of the detection frame. Among them, the detection frame is a closed shape surrounding the text line, and this closed shape can be a regular shape or an irregular shape, usually a rectangle. In some cases, the characters in the text line are neatly arranged with the same size, spacing, and format, etc., as Figure 4 shown in (a) therein, the initial height of this text line is the height H of the detection frame. In other cases, such as Figure 4 shown in (b) therein, if the arrangement of the characters in the text line undergoes deformations such as rotation, distortion, or flipping, then the detection frame of this text line is the circumscribed rectangle of the text line, and the initial height of this text line is the height H of the detection frame.

[0052] In some embodiments, the computer device can detect the document picture through a neural network model to obtain the text lines in the document picture. Exemplarily, the computer device can use the EAST (An Efficient and Accurate Scene Text Detector) model to detect the text lines in the document picture, extract the detection frames of the text lines, and obtain the heights of the detection frames. Specifically, the computer device directly predicts the text lines through the fully convolutional network (FCN) in the EAST model and generates detection frames (which can be rotated rectangles or irregular quadrilaterals), and then performs non-maximum suppression (NMS) on the generated detection frames to produce the final result.

[0053] Of course, it is not limited to the above models. It is easy to understand that any algorithm or model that can recognize text lines and obtain detection boxes can be used in the embodiments of the present application. For example, the neural network model can also be one or more of the FTSN (Fused Text Segmentation Networks) model, the RRPN (Rotation Region Proposal Networks) model, the DMPNet (Deep Matching Prior Network) model, the CTPN (Connectionist Text Proposal Network) model, etc. Taking the CTPN model as an example, specifically, the computer device first extracts the local image features of each character through the CTPN model, and extracts the context features of the character sequence, and then outputs the coordinate values and classification result probability values of each character through the fully connected layer, and then merges adjacent characters into text lines in the data post-processing stage.

[0054] Step S204: Identify the document layout information of the document picture, and determine at least one text block to be processed according to the document layout information. Each text block to be processed includes at least one continuous text line.

[0055] The document layout information is an attribute description of the visible content in the document, and this attribute description characterizes the text blocks included in the document and the section categories of the text blocks. Exemplarily, the common attributes of the document include, but are not limited to, one or more of the main text title, main text paragraph, chart title, header, footer, picture, formula, table, and meaningless text, etc. Correspondingly, the section categories of the document include, but are not limited to, one or more of the title section category, paragraph section category, margin section category, picture section category, formula section category, and table section category, etc. Among them, the title section category also includes one or more of the main text title section category and the icon title section category, etc.

[0056] For example, as Figure 5 shown, a title constitutes a text block, such as text block 501, text block 502, text block 503, and text block 504. These text blocks belong to the title section category. In some embodiments, text lines with the same category and continuous can be merged into a text block. For example, text block 511 is composed of 4 text lines. For text block 511, text block 512, text block 513, text block 514, and text block 515, each text block contains multiple continuous text lines, and these text blocks belong to the paragraph section category. A header or footer constitutes a text block, such as text block 521, and this text block belongs to the margin section category.

[0057] It should be noted that when the text lines are not continuous or there are breaks, they should be divided into different text blocks; in other words, the text lines of the next paragraph form another text block. For example, although text blocks 511 and 512 both belong to the paragraph section category, the text lines are not continuous, so they are two different text blocks; similarly, text blocks 514 and 515 are also two different text blocks.

[0058] Specifically, the computer device obtains the document layout information of the document picture through layout analysis of the document picture. According to the recognized document layout information, the computer device determines one or more text blocks included in the document picture, and selects at least one text block from the one or more text blocks as the text block to be processed subsequently. Exemplarily, based on the document layout information, the computer device selects the text blocks belonging to the title section category and the paragraph section category from the recognized multiple text blocks as the text blocks to be processed, and performs subsequent processing.

[0059] In some embodiments, the computer device can identify the document picture through a neural network model to obtain the document layout information of the document picture. Exemplarily, the computer device can use the Scaled YOLOV4 (real-time object detection) model to detect the document layout information in the document picture. Specifically, the computer device inputs the document picture into the Scaled YOLOV4 model. The model first divides the picture into grids and generates detection frames based on the position information of the grids. Each grid predicts the probability scores belonging to various categories, thereby obtaining the document layout information of the document picture.

[0060] Of course, it is not limited to the above model. It is easy to understand that any algorithm or model that can recognize text lines and obtain detection frames can be used in the embodiments of the present application. For example, the neural network model can also be one or more of the YOLO series of neural network models (including YOLOv1 to YOLOv6, etc.), RCNN (Region with CNN features, region convolutional neural network), etc.

[0061] Step S206, for any text block to be processed, determine the line height statistical information according to the initial line height of the text lines included in the corresponding text block to be processed, and determine the target line height corresponding to the corresponding text block to be processed based on the line height statistical information.

[0062] Due to the influence of various factors, such as unclear or distorted document pictures, or limited detection accuracy, the initial line heights of each text line may be different. Specifically, based on the initial line heights of each text line, the computer device can obtain line height statistical information through various calculations. Based on the line height statistical information, the computer then determines the target line height corresponding to the text block. Among them, the line height statistical information refers to the statistical information calculated based on the initial line heights of the text lines, including but not limited to one or more of sum, mean, extreme values (including maximum and minimum values), standard deviation, variance, mode, median, and midrange, etc. The target line height corresponding to the text block is the target line height of each text line in the text block.

[0063] In some embodiments, the computer device calculates the mean line height of the text lines in a text block based on the initial line heights of each text line, and unifies the line heights of each text line in the text block to the mean line height to obtain the target line height corresponding to the text block.

[0064] Exemplarily, as Figure 6 shown, for a text block 60 to be processed, the text block 60 contains text lines 601 to 605. The initial line height of each text line is the height of the detection box. That is, the initial line height of text line 601 is H1, the initial line height of text line 602 is H2, the initial line height of text line 603 is H3, the initial line height of text line 604 is H4, and the initial line height of text line 605 is H5. The computer device can calculate the mean line height of each text line and use this mean line height as the target line height corresponding to the text block.

[0065] In other embodiments, the computer device calculates the mean line height of the text lines in a text block based on the initial line heights of each text line, and statistically calculates the extreme values of the height changes of each text block. Based on the difference between the mean line height and the extreme values of the height changes, the computer device determines the target line height corresponding to the text block.

[0066] Among them, the extreme values of the height changes are obtained based on the maximum and minimum values of the initial line heights within the text block. Exemplarily, the extreme values of the height changes are based on the difference between the maximum and minimum values of the initial line heights within the text block, and based on the maximum value between this difference and a preset default value. The extreme value of the height change h r can be shown by the following formula:

[0067]

[0068] where h is the initial line height of the text line, and h0 is the preset default value, which is defaulted to 2.

[0069] For the case where there is only one text line in a text block, the extreme value of the height change of the text block is the preset default value 2.

[0070] In some other embodiments, the computer device also takes a statistical value as a reference and unifies the heights of the text lines within the text block accordingly. Exemplarily, the computer device statistically determines the mode of the average line heights of each text block and uses this mode as the reference line height; for a text block, the computer device determines the target line height corresponding to the text block based on the difference between its average line height and the reference line height.

[0071] It should be understood that the above embodiments are only for illustrative purposes and do not limit the scope of the present application. Any adaptive modification or variation of the above examples without departing from the inventive concept of the present application shall be within the protection scope of the present application. For example, the reference line height as a reference can also be the median of the average line heights of each text block, or a preset fixed value, etc.

[0072] Step S208: Based on the preset font size mapping rule and the target line height corresponding to each text block to be processed, obtain the target font size corresponding to each text block in the document picture.

[0073] Among them, the font size mapping rule is used to establish the mapping relationship between the height of the text line and the font size. The font size mapping rule can be set manually or obtained through machine learning training. Exemplarily, a text line with a height less than 8 is mapped to a font size of 5.5, a text line with a height greater than or equal to 8 and less than or equal to 9 is mapped to a font size of 6.5, a text line with a height greater than 9 and less than or equal to 11 is mapped to a font size of 7.5, a text line with a height greater than 11 and less than or equal to 13 is mapped to a font size of 9, a text line with a height greater than 13 and less than or equal to 15 is mapped to a font size of 10.5, a text line with a height greater than 15 and less than or equal to 17 is mapped to a font size of 12, a text line with a height greater than 17 and less than or equal to 19 is mapped to a font size of 14, a text line with a height greater than 19 and less than 21 is mapped to a font size of 15, a text line with a height greater than or equal to 21 and less than 23 is mapped to a font size of 16, a text line with a height greater than or equal to 23 and less than or equal to 26 is mapped to a font size of 18, a text line with a height greater than 26 and less than 31 is mapped to a font size of 22, a text line with a height greater than or equal to 31 and less than or equal to 33 is mapped to a font size of 24, a text line with a height greater than 33 and less than 40 is mapped to a font size of 26, a text line with a height greater than or equal to 40 and less than or equal to 50 is mapped to a font size of 36, and a text line with a height greater than 50 is mapped to a font size of 42.

[0074] Specifically, after the computer device obtains the target line height corresponding to each text block to be processed through the above steps, it determines the mapping relationship between the target line height and the corresponding font size based on the pre-set font size mapping rule, so as to determine the font size corresponding to each text block. In some embodiments, the computer device may use the font size mapped based on the font size mapping rule as the target font size corresponding to the text block; in other embodiments, the computer device may further correct and beautify the mapped font size, and use the corrected font size as the target font size corresponding to the text block.

[0075] In the above method for identifying the font size of a document, by detecting and identifying the document picture, the text lines and document layout information in the document picture are obtained, and each text line is grouped according to the document layout information, so as to determine the text blocks to be processed. Thus, each group can be processed in parallel to improve efficiency and facilitate subsequent statistical analysis; the line height statistical information is determined based on the initial line height of the text line, and then the target line height is calculated based on the line height statistical information, so that the heights of all text lines in the text block are unified. Therefore, when the target font size is obtained based on the pre-set font size mapping rule, the font sizes of all text lines obtained are also correspondingly unified, thus being able to solve the problem that the font sizes recognized for the titles or paragraphs with the same original font size are different, and the accuracy of document restoration is higher. The above method has a high recognition accuracy and a high processing efficiency.

[0076] To facilitate the understanding of the inventive concept of the present application, a brief framework diagram will be used for illustration below. As Figure 7 shown, the method for identifying the font size of a document provided in the embodiments of the present application can be implemented by an overall system, that is, the entire document picture is used as the input of the system, and is respectively passed through the text line detection module and the document layout analysis module, and the results output by the text line detection module and the results output by the document layout module are input into the text font size recognition and processing module together. The font size of the text block is obtained through the processing of the text font size recognition and processing module, and the final font size of the text block is output. The processing steps of the text font size recognition and processing module are the content of the above steps S206 to S208. The specific steps and processes can refer to the content of the foregoing embodiments and will not be elaborated here. Thus, the irregular text line height information that may originally be inconsistent can be regularized, optimizing the overall document font size restoration effect, and further being able to more appropriately restore the original style of the document picture during document restoration.

[0077] As described above, the steps of detecting text lines can be implemented by a neural network model. To this end, in some embodiments, detecting the initial line height of each text line in a document image includes: extracting features from the document image to obtain multiple feature maps of different scales; performing feature fusion on the multiple feature maps of different scales to obtain a fused feature map; performing text line detection processing based on the fused feature map to obtain a target detection box that encloses the text line, and using the height of the target detection box as the initial line height of the text line.

[0078] Specifically, the computer device uses a neural network to extract features from the document image to obtain multiple feature maps of different scales, and then performs feature fusion on these feature maps of different scales to obtain a fused feature map; performs text line detection processing based on the fused feature map, for example, makes predictions based on the fused feature map to generate one or more detection boxes. Exemplarily, the computer device can use, for example, a fully convolutional neural network to perform the steps of feature extraction, feature fusion, and detection box prediction. During this process, the computer device can also perform feature fusion on the feature maps of different scales by continuously upsampling multiple times.

[0079] Then, the computer device can screen the generated detection boxes by means of non-maximum suppression, etc., thereby obtaining a target detection box that encloses the text line. The computer device can then use the height of the target detection box as the initial line height of the text line.

[0080] In the above embodiments, by extracting features and performing feature fusion on the document image and performing text line detection processing, detection boxes for each text line in the document image are generated, and each text line can be accurately identified and the initial line height of each text line can be obtained.

[0081] Similarly, the steps of the computer device performing layout analysis on the document image to identify the document layout information of the document image can also be implemented by a neural network model. To this end, in some embodiments, identifying the document layout information of the document image and determining at least one text block to be processed according to the document layout information includes: performing document layout recognition processing based on the document image to obtain multiple text blocks included in the document image and the respective section categories corresponding to each text block; obtaining a target section category, where the target section category includes at least one of a title section category and a paragraph section category; and using the text block corresponding to the target section category as the text block to be processed.

[0082] Specifically, the computer device performs document layout recognition processing on the document picture. This document layout recognition processing can be a common layout analysis method, such as performing layout analysis through an object detection algorithm. By performing layout analysis on the document picture, the computer device recognizes the layout information (such as title, body text, table, etc.) contained in the document picture, and determines multiple text blocks included in the document picture based on the layout information. Each text block belongs to its respective section category. For example, one or more text lines determined to be the title form a text block, and this text block belongs to the title section category. In some cases, the computer device does not need to convert the font sizes of all text blocks in the document picture, but can convert the key content in the document. In other cases, for the content that occupies a relatively large proportion of the space in the document, the computer device calculates the line height statistical information and converts the font size, while for other content that occupies a relatively small proportion of the space (such as the header and footer), it is directly set to a preset font size. Since most of the content in the document is either a title or a paragraph, in this embodiment, the computer device selects the text blocks belonging to the title section category and the paragraph section category from the text blocks of each section category as the target section categories, and selects one or more text blocks belonging to the target section category as the text blocks to be processed, and performs the steps of S206 later on these text blocks to be processed. For other text blocks (which can be called unprocessed text blocks relative to the text blocks to be processed), the computer device can either do nothing or directly set a preset font size for these text blocks. Thus, by performing layout analysis on the document picture, text blocks belonging to different section categories in the document picture are obtained, which facilitates grouping text lines belonging to the same text block later and facilitates unified analysis and processing later, improving the processing efficiency.

[0083] Exemplarily, for other unprocessed text blocks, the computer device can either do nothing to maintain the original style of the document picture, or determine the target font size according to the initial line height of the unprocessed text block. For this purpose, in some embodiments, the above method further includes: based on a preset font size mapping rule, converting the initial line heights respectively corresponding to the unprocessed text blocks in the document picture to obtain the target font sizes respectively corresponding to each unprocessed text block. Specifically, the computer device obtains multiple text blocks by performing text line detection on the document picture, and determines the unprocessed text blocks from these multiple text blocks. For the unprocessed text blocks, the computer device searches for the font size corresponding to its initial line height according to the preset font size mapping rule, and uses this font size as the target font size respectively corresponding to each unprocessed text block. Thus, for text blocks of a specific section category, it is not necessary to determine the line height statistical information and determine the target line height accordingly, simplifying the processing flow and improving the processing efficiency.

[0084] Exemplarily, for other unprocessed text blocks, the computer device can also directly set the text font size of the text block according to a preset fixed value. To this end, in some embodiments, the above method further includes: using the preset text font size corresponding to the margin section category as the text font size corresponding to the text block corresponding to the margin section category. Specifically, for the text block of the margin section category, the computer device directly uses the preset text font size as the text font size of the text block, without performing the subsequent steps of determining the line height statistical information and determining the target line height accordingly, thereby improving the processing efficiency of document restoration, and uniformly processing the text font sizes of the text blocks of the margin section category, resulting in a unified size, and thus the restored document is more beautiful.

[0085] To improve the efficiency of document processing, the computer device can also group each text line and process each group in parallel, thereby shortening the processing time of the document picture. Accordingly, in some embodiments, the above method further includes: based on the document layout information of the document picture, establishing a data storage structure indexed by the text blocks to be processed, where the data storage structure includes multiple groups, and each group corresponds to a text block to be processed; storing all the text lines belonging to the same text block to be processed into the group corresponding to the text block to be processed in the data storage structure; each group is used to adjust the line height of the text lines stored in it in parallel to obtain the target line height corresponding to the corresponding text block.

[0086] Specifically, based on the document layout information of the document picture, the computer device establishes a data storage structure for each text block to be processed. The data storage structure can be established in the cache, for example. In this data storage structure, the computer device creates a group for each text block respectively, and uses the text block as the index (or primary key) of the group, and stores one or more text lines belonging to the same text block into the group corresponding to the text block. For example, for the title A, the computer device stores the text line a1 corresponding to the text block A into the group A; for the text block B composed of the text paragraph B, the computer device stores the text lines b1 and b2 in the text block B into the group B, and so on.

[0087] After grouping each text line, the computer device can process each group in parallel. For example, by running multiple threads to simultaneously adjust the line height of the text lines stored in each group, the target line height corresponding to each text block can be obtained.

[0088] In the above embodiments, by establishing a data storage structure in the form of groups and processing each group in parallel, on the one hand, it is convenient for subsequent statistics and analysis, and on the other hand, it improves the processing efficiency of the document picture.

[0089] As described above, there are various ways to determine the target line height based on the line height statistical information. In some embodiments, the line height statistical information is determined according to the initial line heights of the text lines included in the corresponding text block to be processed, and the target line height corresponding to the corresponding text block to be processed is determined based on the line height statistical information, including: calculating the average line height of the text lines in the corresponding text block to be processed according to the initial line heights of the text lines included in the corresponding text block to be processed; and determining the target line height corresponding to the corresponding text block to be processed based on the average line height of the text lines in the corresponding text block to be processed.

[0090] Specifically, for a text block to be processed, the computer device counts the number of text lines included in the text block according to one or more text lines included in the text block, and calculates the average line height of the text lines in the text block based on the initial line heights of the one or more text lines respectively. That is, the average line height is obtained based on the ratio of the sum of the initial line heights of the one or more text lines to the number of text lines. Based on the obtained average line height, the computer device can perform further calculations according to the average line height, so as to determine the target line height corresponding to the corresponding text block to be processed. Exemplarily, the computer device may use the calculated average line height as the target line height corresponding to the text block.

[0091] In the above embodiments, the heights of the text lines in the text block are unified through the line height statistical information, effectively avoiding the possible font size fluctuations in the original document image recognition process, making the heights of the text lines belonging to the same text block unified, and the restored font sizes are also unified, optimizing the overall document font size restoration effect and accuracy, and improving the readability and aesthetics of the restored document.

[0092] On this basis, in some embodiments, determining the target line height corresponding to the corresponding text block to be processed based on the average line height of the text lines in the corresponding text block to be processed includes: determining the extreme value of height change corresponding to the corresponding text block to be processed according to the initial line heights of the text lines included in the corresponding text block to be processed; determining a reference line height based on the average line heights of the respective text blocks to be processed; determining the difference between the reference line height and the average line height of the corresponding text block to be processed, and determining the target line height of the corresponding text block to be processed according to the difference and the extreme value of height change.

[0093] Specifically, for a text block to be processed, the computer device can calculate the average line height of the text lines in the text block according to the steps in the above embodiments. At the same time, for the text block, the computer device counts the maximum and minimum values of the initial line heights of the text lines in the text block according to the initial line height of each text line therein, and calculates the extreme value of height change corresponding to the text block based on the maximum and minimum values. Exemplarily, the extreme value of height change hr It can be shown by the following formula:

[0094]

[0095] The computer device determines a reference line height from the average line height of all text blocks to be processed, and uses the reference line height and the extreme value of height change as a reference to adjust the line height corresponding to the text block.

[0096] Specifically, among the average line heights of all text blocks to be processed, the computer device takes the mode of the average line heights as the reference line height. For a text block to be processed, the computer device compares the difference between the average line height of this text block and the reference line height, and then determines the target line height corresponding to this text block based on this difference and the extreme value of height change. In some embodiments, determining the target line height corresponding to the text block to be processed according to the difference and the extreme value of height change includes: when the difference is less than the extreme value of height change, taking the reference line height as the target line height corresponding to the text block to be processed; when the difference is greater than or equal to the extreme value of height change, taking the average line height of the text block to be processed as the target line height corresponding to the text block.

[0097] Specifically, for a text block to be processed, the computer device calculates the difference between the average line height of this text block and the reference line height, and compares the numerical size relationship between this difference and the extreme value of height change. When the difference between the average line height of this text block and the reference line height is less than the extreme value of height change, the computer device takes the reference line height as the target line height corresponding to this text block; conversely, when the difference between the average line height of this text block and the reference line height is greater than or equal to the extreme value of height change, the computer device takes the average line height of the text block to be processed as the target line height corresponding to the text block. Thus, the fluctuation range of the height change of each text line is measured by the extreme value of height change, and the target line height corresponding to the text block is comprehensively determined with reference to this fluctuation range combined with the reference line height, making the height adjustment of each text line more accurate, and further improving the accuracy of font size restoration.

[0098] In the above embodiments, the heights of the text lines in the text block are unified through the line height statistical information, effectively avoiding the possible font size fluctuations in the original document image recognition process, making the heights of the text lines belonging to the same text block unified, and the corresponding restored font sizes also unified, optimizing the overall document font size restoration effect and accuracy, and improving the readability and aesthetics of the restored document.

[0099] To further improve the accuracy of font size restoration, the above method further includes: determining a font size reference value mapped from a reference line height based on a pre-set font size mapping rule, where the reference line height is determined based on the line height statistical information corresponding to each text block to be processed; determining the difference between the font size reference value and a pre-set basic font size, and correcting the target font size corresponding to each text block according to the difference to obtain the text font size corresponding to each text block.

[0100] Specifically, the computer device searches for the font size mapped by the determined reference line height based on the pre-set font size mapping rule, so as to determine the font size reference value corresponding to the reference line height. The computer device corrects the target font size corresponding to all text blocks according to the difference between the font size reference value and the pre-set basic font size, so as to obtain the corrected target font size. In some embodiments, the computer device determines the font size level by which the font size reference value differs from the pre-set basic font size according to the difference between the font size reference value and the pre-set basic font size, and corrects the target font size corresponding to all text blocks according to the differed font size level.

[0101] Taking the layout of a Chinese document as an example, the pre-set basic font size for Chinese characters is usually 12. Assuming that the font size reference value mapped from the reference line height is 10, and the font size reference value differs from the pre-set basic font size by 2 font size levels, the computer device will correspondingly increase the target font size corresponding to all text blocks by 2 font size levels, so as to obtain the text font size corresponding to each text block. Another example is taking the layout of an English document as an example. The pre-set basic font size for English characters is usually 10. Assuming that the font size reference value mapped from the reference line height is 13, and the font size reference value differs from the pre-set basic font size by 3 font size levels, the computer device will decrease the target font size corresponding to all text blocks by 3 font size levels.

[0102] In the above embodiments, by using the font size of the commonly used document layout as a reference to further correct the target font size to obtain the text font size, the document is closer to and conforms to the document specification during restoration, not only the font size restoration is more accurate, but also the obtained document is more beautiful and has strong readability.

[0103] In a typical application scenario, after obtaining the target font size corresponding to a text block, the computer device can perform processing such as document restoration and reconstruction, for example, restoring a document picture to obtain an editable format document, etc. The font sizes of each text line in the document restored by the above method are unified, and it has strong readability and beauty.

[0104] To this end, in some embodiments, the above method further includes: based on the text font sizes respectively corresponding to the text blocks in the document picture, converting the text lines included in each text block according to the corresponding text font sizes to obtain a target document corresponding to the document picture and belonging to the text format.

[0105] Specifically, the computer device obtains the target font sizes respectively corresponding to the text blocks in the document picture according to the above processing, and further corrects the target font sizes according to the preset document layout and the preset basic font size to obtain the text font sizes that meet the aesthetics and specifications of the document layout. The computer device converts the document picture according to the information of the text font sizes and by using methods such as OCR recognition to obtain a target document belonging to the text format, and the font size of the text in the target document is the processed text font size. The text format is, for example, the doc (document) format, etc.

[0106] In the above embodiments, the heights of the text lines in the text block are unified through the line height statistical information. During document restoration, the possible font size fluctuations in the original document picture recognition process are effectively avoided, so that the heights of the text lines belonging to the same text block are unified, and the corresponding restored font sizes are also unified, optimizing the overall document font size restoration effect and accuracy, and improving the readability and aesthetics of the restored document.

[0107] This application also provides an application scenario that applies the above document font size recognition method. Specifically, the application of the document font size recognition method in this application scenario is as follows: after obtaining a document picture through various legal or authorized methods (such as downloading, taking pictures, screenshotting, etc.), the computer device executes the above document font size recognition method through the website or application program it runs, obtains the font sizes of the text blocks in the document picture, and performs processing such as document restoration to convert the document picture into a document in an editable text format, such as a word document, etc. Of course, it is not limited to this. The document font size recognition method provided by this application can also be applied in other application scenarios, such as e-book conversion, map display, etc.

[0108] To facilitate a better understanding of this application, a specific example is given below. As Figure 8As shown, taking the processing of the title section category and the paragraph section category as an example, first, each text line and its corresponding initial line height are obtained through detection, and the document layout information of the document picture is recognized. According to the detected text lines and the document layout information, the computer device traverses each text line in the document picture to determine whether it belongs to the section categories to be processed (i.e., titles and paragraphs). If so, it enters the subsequent steps; otherwise, it continues to traverse the next text line. To improve the processing efficiency, when traversing the text lines, the computer device, based on the document layout information, adds a new data storage structure to store the information of the text lines of the same paragraph or title in the same group, which is convenient for subsequent statistical analysis.

[0109] For each group, the computer device counts the number of text lines and the average line height of the group, and counts the extreme value h of the height change of the groups with the number of lines greater than 1 r , and at the same time counts the mode of the average line height of each group as the reference line height. After obtaining these line height statistical information, the computer device calculates the difference between the average line height of each group and the reference line height. If the difference between the average line height and the reference line height is less than the extreme value h of the height change r , then the height of the group is unified to the reference line height, that is, the target line height. If the difference between the average line height and the reference line height is less than the extreme value h of the height change r , then the height of the group remains unchanged. To ensure that the font sizes of the same-level chart titles are the same, for the text block of the chart title, the height of the inner text lines is unified to the average line height.

[0110] After obtaining the target line heights of all groups, the computer device maps the target line heights according to the font size mapping rule to obtain the target font sizes. For example, map the target line height greater than 11 and less than or equal to 13 to font size 9, map the target line height greater than 13 and less than or equal to 15 to font size 10.5, and so on.

[0111] On this basis, in order to further improve readability and optimize the document visualization effect, the computer device determines the font size reference value mapped by the reference line height based on the font size mapping rule, and offsets and corrects the target font size of all text blocks according to the difference between the font size reference value and the preset base font size. For example, for a Chinese document layout, the usual preset base font size is 12; for an English document layout, the usual preset base font size is 10.5. Therefore, the computer device compares the font size reference value with the preset base font size, and offsets and corrects the target font size of all text blocks according to the comparison result and the font size mapping rule. The offset correction rule is, for example: if the font size reference value is greater than or less than the preset base font size by n levels, then the target font size of all text blocks is also adjusted down or up by n levels, but the adjustment range cannot exceed the upper and lower limits in the font size mapping rule. For example, the upper limit is the maximum font size of 42, and the lower limit is the minimum font size of 6.5. Finally, the computer device uses the offset-corrected target font size as the target font size for the final output of the text block and stores it. Exemplarily, the computer device stores the target font size of the text block as a structure containing document layout information for subsequent use.

[0112] Exemplarily, the computer device can also display the processed target font size together with each text block to present a visual processing result. As Figure 9 shown, the displayed figure is on the original input document picture, with the detection frames of the detected text lines marked, and the font sizes of the text lines are marked around each detection frame. The font size of a text line is the font size of each character in the text line. For example, there is only one text line in text block 901, and the font size of this text block 901 is 24. Similarly, the font size of the text line in text block 902 is 18. Another example is that text lines 911 and 912 belong to the same text block, and their font sizes are both 14. Another example is that the corresponding font size of text block 921 is 8. According to the font sizes of each text block and text line, the computer device can further perform document restoration and other processing.

[0113] Through the above method, not only can the font size of each text line in the document picture be accurately identified, but also through font size mapping and correction, the effect of identifying the same font size for paragraph texts with the same original font size is achieved, solving the problem of inconsistent identified font sizes and improving the readability of the document. Moreover, through the above method, the font size differences between text titles, chart titles, and body paragraphs can be effectively distinguished, effectively enhancing the aesthetics of document restoration.

[0114] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of the steps or stages in other steps or other steps.

[0115] Based on the same inventive concept, an embodiment of the present application further provides a document font size recognition device for implementing the above-mentioned document font size recognition method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the document font size recognition device provided below can refer to the limitations on the document font size recognition method in the above text, and will not be repeated here.

[0116] In some embodiments, as Figure 10 shown, a document font size recognition device 1000 is provided, including: an acquisition module 1001, a processing module 1002, and an output module 1003, where:

[0117] The acquisition module 1001 is configured to acquire a document picture to be processed and detect the initial line height of each text line in the document picture.

[0118] The processing module 1002 is configured to recognize the document layout information of the document picture and determine at least one text block to be processed according to the document layout information. Each text block to be processed includes at least one consecutive text line.

[0119] The processing module 1002 is further configured to, for any text block to be processed, determine line height statistical information according to the initial line height of the text lines included in the corresponding text block to be processed, and determine the target line height corresponding to the corresponding text block to be processed based on the line height statistical information.

[0120] The output module 1003 is configured to obtain the target font size corresponding to each text block in the document picture based on the pre-set font size mapping rule and the target line height corresponding to each text block to be processed.

[0121] In some embodiments, the acquisition module is further configured to perform feature extraction on the document image to obtain multiple feature maps of different scales; perform feature fusion on the multiple feature maps of different scales to obtain a fused feature map; perform text line detection processing based on the fused feature map to obtain a target detection box including the surrounded text line, and use the height of the target detection box as the initial line height of the text line.

[0122] In some embodiments, the processing module is further configured to perform document layout recognition processing based on the document image to obtain multiple text blocks included in the document image and the corresponding layout category of each text block; obtain a target layout category, where the target layout category includes at least one of a title layout category and a paragraph layout category; and use the text block corresponding to the target layout category as the text block to be processed.

[0123] In some embodiments, the above apparatus further includes a grouping module, configured to establish a data storage structure indexed by the text block to be processed based on the document layout information of the document image, where the data storage structure includes multiple groups, and each group corresponds to a text block to be processed; store all the text lines belonging to the same text block to be processed in the group corresponding to the text block to be processed in the data storage structure; and each group is used to adjust the line height of the text lines stored therein in parallel to obtain the target line height corresponding to the corresponding text block.

[0124] In some embodiments, the document layout information includes multiple text blocks included in the document image and the corresponding layout category of each text block, and the layout category includes a margin layout category; the processing module is further configured to use the preset text font size corresponding to the margin layout category as the text font size corresponding to the text block corresponding to the margin layout category.

[0125] In some embodiments, the processing module is further configured to calculate the average line height of the text lines in the corresponding text block to be processed according to the initial line height of the text lines included in the corresponding text block to be processed; and determine the target line height corresponding to the corresponding text block to be processed based on the average line height of the text lines in the corresponding text block to be processed.

[0126] In some embodiments, the processing module is further configured to determine the extreme value of height change corresponding to the corresponding text block to be processed according to the initial line height of the text lines included in the corresponding text block to be processed; determine a reference line height based on the average line height of each text block to be processed; determine the difference between the reference line height and the average line height of the corresponding text block to be processed, and determine the target line height of the corresponding text block to be processed according to the difference and the extreme value of height change.

[0127] In some embodiments, the processing module is further configured to, when the difference is less than the extreme value of the height change, use the reference line height as the target line height corresponding to the corresponding text block to be processed; when the difference is greater than or equal to the extreme value of the height change, use the average line height of the corresponding text block to be processed as the target line height corresponding to the corresponding text block.

[0128] In some embodiments, the above device further includes a conversion module, configured to convert the initial line height corresponding to each unprocessed text block in the document picture based on a pre-set font size mapping rule, to obtain the target font size corresponding to each unprocessed text block.

[0129] In some embodiments, the above device further includes a correction module, configured to determine a font size reference value mapped from the reference line height based on a pre-set font size mapping rule, where the reference line height is determined based on the line height statistical information corresponding to all text blocks to be processed; determine the difference between the font size reference value and the preset basic font size, and correct the target font size corresponding to each text block according to the difference, to obtain the text font size corresponding to each text block.

[0130] In some embodiments, the above device further includes a restoration module, configured to convert the text lines included in each text block according to the corresponding text font size based on the text font size corresponding to each text block in the document picture, to obtain a target document corresponding to the document picture and in text format.

[0131] Each module in the above document font size recognition device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0132] In some embodiments, a computer device is provided. The computer device can be a terminal or a server, and its internal structure diagram can be as Figure 11As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for identifying document font sizes. The display unit of the computer device is used to form a visually visible image, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0133] Those skilled in the art can understand that Figure 11 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0134] In some embodiments, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0135] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0136] In some embodiments, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0137] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memories can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0138] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0139] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for identifying the font size of a document, characterized in that, The method includes: Obtain a document picture to be processed, and detect the initial line height of each text line in the document picture; Identify the document layout information of the document picture, and determine at least one text block to be processed according to the document layout information. Each text block to be processed includes at least one continuous text line; For any text block to be processed, calculate the average line height of the text lines in the corresponding text block to be processed according to the initial line height of the text lines included in the corresponding text block to be processed; Determine a reference line height based on the average line heights corresponding to each text block to be processed; the reference line height is the mode among the average line heights corresponding to each text block to be processed; Determine the target line height corresponding to the corresponding text block to be processed according to the magnitude relationship between the difference between the reference line height and the average line height of the corresponding text block to be processed and the extreme value of the height change corresponding to the corresponding text block to be processed; Based on a pre-set font size mapping rule and the target line height corresponding to each text block to be processed, obtain the target font size corresponding to each text block in the document picture.

2. The method according to claim 1, wherein The detecting the initial line height of each text line in the document picture includes: Extract features from the document picture to obtain multiple feature maps of different scales; Perform feature fusion on the multiple feature maps of different scales to obtain a fused feature map; Perform text line detection processing based on the fused feature map to obtain a target detection frame enclosing the text line, and use the height of the target detection frame as the initial line height of the text line.

3. The method according to claim 1, characterized in that, The identifying the document layout information of the document picture and determining at least one text block to be processed according to the document layout information includes: Perform document layout recognition processing based on the document picture to obtain multiple text blocks included in the document picture and the respective layout categories corresponding to each text block; Obtain a target layout category, where the target layout category includes at least one of a title layout category and a paragraph layout category; Use the text block corresponding to the target layout category as the text block to be processed.

4. The method according to claim 1, wherein The method further includes: Based on the document layout information of the document picture, establish a data storage structure indexed by the text blocks to be processed. The data storage structure includes multiple groups, and each group corresponds to a text block to be processed; Store all the text lines belonging to the same text block to be processed into the group corresponding to the corresponding text block to be processed in the data storage structure; each group is used to parallelly adjust the line height of the text lines stored therein to obtain the target line height corresponding to the corresponding text block.

5. The method according to claim 1, wherein The document layout information includes multiple text blocks included in the document picture and the respective layout categories corresponding to each text block. The layout category includes a margin layout category; the method further includes: Use a preset text font size corresponding to the margin layout category as the text font size corresponding to the text block corresponding to the margin layout category.

6. The method according to claim 1, wherein The method further includes: Determine the extreme value of height change corresponding to the text block to be processed according to the initial line height of the text lines included in the corresponding text block to be processed; the extreme value of height change is obtained based on the maximum and minimum values of the initial line heights within the text block.

7. The method according to claim 6, wherein The determining of the target line height corresponding to the text block to be processed according to the relationship between the difference between the reference line height and the average line height of the text block to be processed and the extreme value of height change corresponding to the text block to be processed includes: When the difference is less than the extreme value of height change, use the reference line height as the target line height corresponding to the text block to be processed; When the difference is greater than or equal to the extreme value of height change, use the average line height of the text block to be processed as the target line height corresponding to the text block.

8. The method according to claim 1, wherein The method further includes: Based on the pre-set font size mapping rule, convert the initial line height corresponding to each unprocessed text block in the document picture to obtain the target font size corresponding to each unprocessed text block.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Based on the pre-set font size mapping rule, determine the font size reference value mapped by the reference line height; Determine the difference between the font size reference value and the pre-set basic font size, and correct the target font size corresponding to each text block according to the difference to obtain the text font size corresponding to each text block.

10. The method according to claim 9, wherein The method further includes: Based on the text font sizes corresponding to the text blocks in the document picture, convert the text lines included in each text block according to the corresponding text font size to obtain a target document corresponding to the document picture and belonging to the text format.

11. An apparatus for recognizing the font size of a document, characterized in that, The device includes: An acquisition module, configured to acquire a document picture to be processed and detect the initial line height of each text line in the document picture; A processing module, configured to identify the document layout information of the document picture and determine at least one text block to be processed according to the document layout information, where each text block to be processed includes at least one consecutive text line; The processing module is further configured to, for any text block to be processed, calculate the average line height of the text lines in the text block to be processed according to the initial line height of the text lines included in the text block to be processed; determine the reference line height based on the average line height corresponding to each text block to be processed; the reference line height is the mode of the average line heights corresponding to each text block to be processed; determine the target line height corresponding to the text block to be processed according to the relationship between the difference between the reference line height and the average line height of the text block to be processed and the extreme value of height change corresponding to the text block to be processed; An output module, configured to obtain the target font size corresponding to each text block in the document picture based on the pre-set font size mapping rule and the target line height corresponding to each text block to be processed.

12. The document font size recognition device according to claim 11, wherein The acquisition module is further configured to perform feature extraction on the document picture to obtain multiple feature maps of different scales; perform feature fusion on the multiple feature maps of different scales to obtain a fused feature map; perform text line detection processing based on the fused feature map to obtain a target detection box including the surrounded text line, and use the height of the target detection box as the initial line height of the text line.

13. The document font size recognition device according to claim 11, characterized in that, The processing module is further configured to perform document layout recognition processing based on the document picture to obtain multiple text blocks included in the document picture and the corresponding layout category of each text block; obtain a target layout category, where the target layout category includes at least one of a title layout category and a paragraph layout category; and use the text block corresponding to the target layout category as the text block to be processed.

14. The document font size recognition device according to claim 11, characterized in that, The apparatus further includes a grouping module, configured to establish a data storage structure indexed by the text block to be processed based on the document layout information of the document picture, where the data storage structure includes multiple groups, and each group corresponds to a text block to be processed. All text lines belonging to the same text block to be processed are stored in the group corresponding to the corresponding text block to be processed in the data storage structure; each group is used to parallelly adjust the line height of the text lines stored therein to obtain the target line height corresponding to the corresponding text block.

15. The document font size recognition device according to claim 11, characterized in that, The processing module is further configured to use the preset text font size corresponding to the margin layout category as the text font size corresponding to the text block corresponding to the margin layout category.

16. The document font size recognition device according to claim 11, characterized in that, The processing module is further configured to determine the extreme value of height change corresponding to the text block to be processed according to the initial line height of the text lines included in the corresponding text block to be processed; the extreme value of height change is obtained based on the maximum and minimum values of the initial line height within the text block.

17. The document font size recognition device according to claim 16, characterized in that, The processing module is further configured to, when the difference is less than the extreme value of height change, use the reference line height as the target line height corresponding to the corresponding text block to be processed; when the difference is greater than or equal to the extreme value of height change, use the average line height of the corresponding text block to be processed as the target line height corresponding to the corresponding text block.

18. The document font size recognition device according to claim 11, wherein, The apparatus further includes a conversion module, configured to convert the initial line height corresponding to each unprocessed text block in the document picture based on a preset font size mapping rule to obtain the target font size corresponding to each unprocessed text block.

19. The document font size recognition device according to any one of claims 11 to 18, characterized in that, The apparatus further includes a correction module, configured to determine the font size reference value mapped from the reference line height based on a preset font size mapping rule; determine the difference between the font size reference value and a preset basic font size, and correct the target font size corresponding to each text block according to the difference to obtain the text font size corresponding to each text block.

20. The document font size recognition device according to claim 19, wherein The apparatus further includes a restoration module, configured to convert the text lines included in each text block according to the corresponding text font size based on the text font size corresponding to each text block in the document picture to obtain a target document in text format corresponding to the document picture.

21. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

22. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 10 is implemented.

23. A computer program product, comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, the method according to any one of claims 1 to 10 is implemented.

Images