Document image binarization method and device, computing equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of machine learning, in particular to a document image binarization method and device, computing equipment and a storage medium. The method comprises the following steps: acquiring a digital image of a target document; the document image is input into a binarization model, a text binarization image and a table binarization image corresponding to the document image are obtained through model output, the binarization model is obtained through deep learning training by using multiple sets of data, and the binarization model is obtained through deep learning training. The plurality of sets of data include a document image, a desired text binarization tag image, and a desired table binarization tag image. According to the scheme, information irrelevant to the text can be removed while the document image is binarized, so that the binarization model pays more attention to the text and table content in the document image, and the improvement of the attractiveness of document image binarization and the performance of subsequent document analysis and recognition is facilitated.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of machine learning technology, and in particular to a document image binarization method. Background Art

[0002] When a camera takes a photo of a document, problems such as uneven lighting are likely to occur. In order to facilitate reading, printing, or OCR recognition of text content in the document image, the document image needs to be binarized.

[0003] Traditional binarization methods, such as the Otsu method and adaptive binarization, have poor adaptability and often produce large areas of black shadows in the resulting binarized images. Current deep learning-based binarization methods are unable to remove non-text information, affecting the aesthetics of printed documents and the accuracy of optical character recognition (OCR). Summary of the Invention

[0004] To address the problem that existing document image binarization methods do not provide special processing for text and tables, embodiments of the present invention provide a document image binarization method, apparatus, computing device, and storage medium to improve the above problem. To achieve the above object, the technical solutions adopted by the present invention are as follows:

[0005] In a first aspect, a document image binarization method is provided, comprising:

[0006] obtaining a digital image of a target document;

[0007] The document image is input into the binarization model, and the model outputs the text binarization image and the table binarization image corresponding to the document image, wherein the binarization model is obtained by deep learning training using multiple sets of data, and the multiple sets of data include the document image and the expected text binarization label image and the expected table binarization label image.

[0008] Furthermore, before inputting the document image into the model, the method further includes: normalizing the size of the document image, wherein the step of normalizing the size of the document image includes:

[0009] Get the width W of the input document image i and height H i ;

[0010] If W i If it is less than 512, the width W of the normalized image is set to 512; otherwise, the width W of the normalized image is set to the value closest to W i 2 n where n is an integer.

[0011] If H iIf it is less than 512, the height H of the normalized image is set to 512; otherwise, the height H of the normalized image is set to the value closest to H i 2 n where n is an integer.

[0012] Furthermore, the width and height of the text binary image output by the binarization model are respectively normalized to the width W and height H of the image, and the text binary image only contains the text information in the document image; the width and height of the table binary image output by the binarization model are respectively normalized to the width W and height H of the image, and the table binary image only contains the table line information in the document image.

[0013] Furthermore, the binarization model includes a feature extraction module and a classification module; wherein the feature extraction module adopts a deep neural network structure that first downsamples and then upsamples, and the classification module includes multiple convolutional layers, and the number of output channels of the last convolutional layer is 2;

[0014] Furthermore, the binarization model includes a feature extraction module and a classification module; wherein the feature extraction module adopts a feature extraction module of a Unet, Trans-Unet, or Mamba-Unet network, and the classification module includes two convolutional layers, the convolution kernel size of the first convolutional layer is 3, the convolution step size is 1, the padding is 1, and the number of output channels is the same as the number of feature channels output by the feature extraction module; the convolution kernel size of the second convolutional layer is 3, the convolution step size is 1, the padding is 1, and the number of output channels is 2;

[0015] Furthermore, the text binarized image output by the model only contains the text content in the document image, and the table binarized image output by the model only contains the table line information in the document image.

[0016] In a second aspect, an embodiment of the present invention provides a document image binarization device, comprising:

[0017] An acquisition unit, configured to acquire a document image to be processed;

[0018] An input unit, configured to input the document image into a pre-trained binarization model;

[0019] An output unit is used to output the text binarized image and the table binarized image.

[0020] In a third aspect, an embodiment of the present invention further provides a computing device, including:

[0021] A memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0022] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0023] The beneficial effects of the present invention are:

[0024] The present invention processes the document image to obtain a text binarized image and a table binarized image, which can remove text-irrelevant information and make the binarization model focus more on the text and table content in the document image, which is beneficial to improving the aesthetics of the document image binarization and the performance of subsequent document analysis and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 This is a flow chart of a document image binarization method provided by one embodiment of the present invention;

[0027] Figure 2 This is a model architecture diagram of a document image binarization method provided by one embodiment of the present invention;

[0028] Figure 3 This is a hardware architecture diagram of an electronic device provided by one embodiment of the present invention;

[0029] Figure 4 This is a structural diagram of a document image binarization device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0031] As mentioned above, in the relevant technologies, the existing binarization methods based on traditional Otsu's method, adaptive binarization method and deep learning-based binarization method all give a single binarized image based on the input image, without considering the text and table characteristics in the input image. The binarization results contain other interference signals, which is not conducive to text recognition and table recognition.

[0032] In order to solve the above technical problems, the inventors considered using a binarization model to learn text and table features. The new model can output the binarization results corresponding to the text and tables in the document image, and filter out other interference information such as non-text and tables. The output of the model is conducive to the realization of tasks such as text recognition, table recognition and layout analysis.

[0033] The specific implementation of the above concept is described below.

[0034] refer to Figure 1 , an embodiment of the present invention provides a text image binarization method, the method comprising:

[0035] Step 100: Acquire a digital image of a target document;

[0036] Step 101: Normalize the size of the document image to obtain a normalized document image I. The normalization method is: obtain the width W of the input document image i and height H i , if W i If W is less than 512, the width of the normalized image is set to 512. Otherwise, the width of the normalized image is set to the value closest to W. i 2 n Where n is an integer. If H i If it is less than 512, the height H of the normalized image is set to 512, otherwise the height H of the normalized image is set to the value closest to H i 2 n where n is an integer.

[0037] Step 102: Input the normalized image I into the binarization model, the binarization model includes a feature extraction module and a classification module; wherein the feature extraction module adopts the feature extraction module of the Unet network, the classification module includes 2 convolutional layers, the convolution kernel size of the first convolutional layer is 3, the convolution step is 1, the padding is 1, the number of output channels is the same as the number of feature channels output by the feature extraction module, the convolution kernel size of the second convolutional layer is 3, the convolution step is 1, the padding is 1, and the number of output channels is 2. The text binarization image output by the model only contains text-related content, and the table binarization image output by the model only contains table line-related information. In this embodiment of the present invention, the network of the binarization model is as follows Figure 2 shown.

[0038] Figure 2 The box represents the input image, feature tensor, or output image. The three numbers separated by commas above or to the side of the box (e.g., H, W, 3) represent the height, width, and number of channels of the image or tensor. The arrowed line "conv" represents a convolution operation with a kernel size of 3, a stride of 1, and a padding of 1. The number of channels is determined by the input and output of the convolution. maxpool represents a 2x2 maximum pooling operation, upconv represents an upsampling operation, and cat represents the concatenation of two tensors in the channel dimension. The network outputs a binary image of text and a binary image of table after encoding the input image through five downsampling operations and decoding it through five upsampling operations, followed by two convolution operations.

[0039] like Figure 3 、 Figure 4 As shown, the embodiment of the present invention provides a document image binarization device. The device embodiment can be implemented by software, hardware, or a combination of software and hardware. From the hardware level, Figure 3 As shown in FIG. 1 , a hardware architecture diagram of a computing device where a document image binarization device is provided in an embodiment of the present invention is provided. Figure 3 In addition to the processor, memory, and network interface shown, the computing device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 4 As shown, as a device in a logical sense, it is formed by the CPU of the computing device in which it is located reading the corresponding computer program in the non-volatile memory into the internal memory and running it.

[0040] like Figure 4 As shown, this embodiment provides a document image binarization device, including:

[0041] An acquisition unit 301 is configured to acquire a document image to be processed;

[0042] An input unit 302 is used to normalize the document image and input it into a pre-trained binarization model;

[0043] The output unit 303 is configured to output the text binarized image and the table binarized image.

[0044] It should be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a document image binarization method. In other embodiments of the present invention, a document image binarization method may include more or fewer modules than illustrated, or may combine or separate certain modules, or may have different module arrangements. The illustrated modules may be implemented in hardware, software, or a combination of software and hardware.

[0045] The information interaction, execution process, etc. between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention and will not be repeated here.

[0046] An embodiment of the present invention further provides a computing device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a document image binarization method according to any embodiment of the present invention is implemented.

[0047] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes a document image binarization method according to any embodiment of the present invention.

[0048] Specifically, a system or device equipped with a storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program codes stored in the storage medium.

[0049] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0050] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0051] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0052] In addition, it can be understood that the program code read from the storage medium is written into a memory provided in an expansion board inserted into the computer or into a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0053] It should be noted that, in this article, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical factors in the process, method, article or device comprising the elements.

[0054] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A document image binarization method, characterized in that: include: obtaining a digital image of a target document; The document image is input into the binarization model, and the model outputs the text binarization image and the table binarization image corresponding to the document image, wherein the binarization model is obtained through deep learning training using multiple sets of data, and the multiple sets of data include the document image and the expected text binarization label image and the expected table binarization label image.

2. The method according to claim 1, characterized in that Before inputting the document image into the model, the method further includes: normalizing the size of the document image; The document image size normalization processing step includes: Get the width W of the input document image i and height H i ; If W i If it is less than 512, the width W of the normalized image is set to 512; otherwise, the width W of the normalized image is set to the value closest to W i 2 n where n is an integer. If H i If it is less than 512, the height H of the normalized image is set to 512; otherwise, the height H of the normalized image is set to the value closest to H i 2 n where n is an integer.

3. The method according to claim 1, characterized in that The width and height of the text binary image are W and H as described in claim 2, respectively, and the text binary image only contains text information in the document image; The width and height of the table binary image are W and H as described in claim 2, respectively. The table binary image only contains table line information in the document image.

4. The method according to claim 1, wherein The binarization model includes a feature extraction module and a classification module; wherein the feature extraction module adopts a deep neural network structure that first downsamples and then upsamples, and the classification module includes multiple convolutional layers, and the number of output channels of the last convolutional layer is 2.

5. The method according to claim 1, wherein The binarization model includes a feature extraction module and a classification module; wherein the feature extraction module adopts the feature extraction module of the Unet, Trans-Unet or Mamba-Unet network, and the classification module includes two convolutional layers, the convolution kernel size of the first convolution layer is 3, the convolution step size is 1, the padding is 1, and the number of output channels is the same as the number of feature channels output by the feature extraction module; the convolution kernel size of the second convolution layer is 3, the convolution step size is 1, the padding is 1, and the number of output channels is 2.

6. The method according to claim 1, characterized in that The text binarized image output by the model only contains the text content in the document image, and the table binarized image output by the model only contains the table line information in the document image.

7. A document image binarization device, characterized in that: include: An acquisition unit, configured to acquire a document image to be processed; An input unit, configured to normalize the document image and input the normalized document image into a pre-trained binarization model; An output unit is used to output the text binarized image and the table binarized image.

8. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 6.

Images