A document image binarization method and system
By combining a progressive multi-scale window self-attention encoder and a frequency domain feature enhancement module with a Transformer decoder, the problem of insufficient segmentation accuracy in document image binarization under complex backgrounds is solved, achieving efficient and accurate binarization processing and improving the model's adaptability and robustness.
Patent Information
- Application Number
- CN202611040032.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-14
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2046-07-14
AI Technical Summary
Existing document image binarization methods suffer from insufficient segmentation accuracy when faced with complex backgrounds and text deformations. Furthermore, they have a large number of model parameters and slow inference speed, making it difficult to achieve efficient and accurate binarization processing in different scenarios.
A progressive multi-scale window self-attention encoder module and a frequency domain feature enhancement module are adopted, combined with a Transformer decoder. Feature extraction, encoding and decoding are performed through a multi-scale window self-attention mechanism and fast Fourier convolution to achieve local and global frequency domain feature enhancement and generate binarized results.
It significantly improves the accuracy and robustness of document image binarization, and can handle complex degraded images, balancing adaptability and speed.
Smart Images

Figure CN122574409B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and document image binarization, and in particular to a document image binarization method and system. Background Technology
[0002] With the rapid development of information technology, the digitization and information processing of document images has become a crucial part of modern society. Document image segmentation technology plays a vital role in practical tasks such as building digital libraries, managing electronic documents, and automatically parsing documents. In particular, document image thresholding segmentation technology can improve the accuracy and efficiency of document information extraction and effectively optimize image content, thus possessing significant application value. However, document images are often affected by various factors such as noise, image blurring, uneven lighting, and cluttered backgrounds. Although traditional image thresholding algorithms can achieve relatively ideal results under certain conditions, the segmentation accuracy is often insufficient when dealing with images with complex backgrounds or text distortion. In contrast, the Transformer-based document image binarization method can automatically extract and learn deep features of the image through multi-layer neural networks, thereby obtaining more accurate segmentation results than traditional algorithms.
[0003] For example, the patent document titled "A Binarization Method and System for Degenerate Document Images Based on Adaptive Convolution" (Publication No.: CN119444582A) uses an adaptive convolution module that dynamically adjusts the weights of the convolution kernel through a dynamic weight subnet to better distinguish between text and background. A guided attention module is used in the encoder to further refine features. A gated pyramid module in the decoder is used for multi-scale feature fusion to better recover image details. Five loss functions are optimized simultaneously during training to make the binarization result closer to the true label from different dimensions. However, because the model integrates multiple specialized modules and uses five loss functions, it may result in a large number of model parameters and slow inference speed, which is not conducive to deployment in practical application scenarios. Another example is the patent document titled "An Adaptive Image Binarization Method Based on Edge Information" (Publication No.: CN116363158A). This method calculates the threshold by weighted average based on image edge information. The process is simple and direct, and does not involve model training. However, this method is highly sensitive to the results of edge detection. If the text in the image is blurred or contains a lot of noise, it will directly lead to errors in subsequent threshold calculations, resulting in binarization failure. A patent document titled "Document Image Binarization Method, System, Computer Device, and Readable Storage Medium" (Publication No.: CN116798043A) treats image grayscale values as terrain elevation, i.e., text strokes are raised and the background is the ground, and uses a ground filter to separate the text from the background. However, it requires manual adjustment of filter parameters and has limited generalization ability for non-smooth backgrounds. Therefore, there is an urgent need to design a new method that can accurately binarize complex images while also considering adaptability and robustness. Summary of the Invention
[0004] In view of the above-mentioned shortcomings of current image binarization, the present invention provides a document image binarization method and system, which can efficiently process document images in different scenarios and finally generate binarized document images.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: A document image binarization method, the method includes: Step S1: Obtain document image data; Step S2: Construct a document image binarization model consisting of a progressive multi-scale window self-attention encoder module, a frequency domain feature enhancement module, and a Transformer decoder module cascaded sequentially. Data flows through the progressive multi-scale window self-attention encoder module, the frequency domain feature enhancement module, and the Transformer decoder module in sequence to achieve feature extraction, encoding, frequency domain feature enhancement, decoding, and reconstruction, and outputs the binarized document image. Step S3: Input the acquired document image data into the document image binarization model to obtain the reconstructed document image. After thresholding, the reconstructed document image is used to obtain the binarized document image.
[0006] According to one aspect of the present invention, a progressive multi-scale window self-attention encoder module is used to perform feature extraction and encoding processing on the input binarized document image to obtain a feature sequence; a frequency domain feature enhancement module is used to perform global frequency domain feature enhancement processing on the feature sequence based on fast Fourier convolution to obtain an enhanced feature sequence; and a Transformer decoder module is used to decode and reconstruct the enhanced feature sequence to obtain a reconstructed document image, which is then thresholded to obtain a binarized document image.
[0007] According to one aspect of the present invention, a progressive multi-scale window self-attention encoder module includes m sequentially connected improved Transformer encoders. Each improved Transformer encoder includes a first normalization layer, a first progressive window multi-head self-attention layer, a second normalization layer, and a first feedforward neural network layer. The binarized document image is sequentially passed through the first normalization layer, the first progressive window multi-head self-attention layer, the second normalization layer, and the first feedforward neural network layer to obtain the encoded feature sequence.
[0008] According to one aspect of the present invention, a frequency domain feature enhancement module includes a sequence-to-two-dimensional mapping layer, L dynamically learnable Fourier convolutional residual blocks, and a two-dimensional-to-sequence mapping layer; the feature sequence output by the progressive multi-scale window self-attention encoder module is first rearranged into a two-dimensional feature map by the sequence-to-two-dimensional mapping layer, and then sequentially input into the L dynamically learnable Fourier convolutional residual blocks for local and global joint frequency domain enhancement processing; finally, the enhanced two-dimensional feature map is input into the two-dimensional-to-sequence mapping layer to map back to sequence features, thereby obtaining an enhanced feature sequence; The dynamically learnable Fourier convolutional residual block includes a first dynamically learnable Fourier convolutional layer and a second dynamically learnable Fourier convolutional layer. The input of the dynamically learnable Fourier convolutional residual block is connected to the input of the first dynamically learnable Fourier convolutional layer, the output of the first dynamically learnable Fourier convolutional layer is connected to the input of the second dynamically learnable Fourier convolutional layer, and the output of the second dynamically learnable Fourier convolutional layer after concatenating the input residual of the dynamically learnable Fourier convolutional residual block is the output of the dynamically learnable Fourier convolutional residual block.
[0009] According to one aspect of the present invention, a dynamically learnable Fourier convolutional layer includes local branches, global branches, and dynamic routing branches, specifically: The local branch includes a first convolutional layer, a third normalization layer, and a first activation function layer. The input of the first convolutional layer is the input of the dynamically learnable Fourier convolutional layer. The input of the third normalization layer is the output of the first convolutional layer. The input of the first activation function layer is the output of the third normalization layer. The output of the first activation function layer is the output of the local branch. The global branch includes a second convolutional layer, a fourth normalization layer, a second activation function layer, a first real-valued FFT2D layer, a third convolutional layer, a fifth normalization layer, a third activation function layer, a first learnable frequency domain mask, and a first inverse real-valued FFT2D layer. The input of the second convolutional layer is the input of the dynamically learnable Fourier convolutional layer. The input of the fourth normalization layer is the output of the second convolutional layer. The input of the second activation function layer is the output of the fourth normalization layer. The input of the first real-valued FFT2D layer is the output of the second activation function layer. The input of the third convolutional layer is the output of the first real-valued FFT2D layer. The input of the fifth normalization layer is the output of the third convolutional layer. The input of the third activation function layer is the output of the fifth normalization layer. The input of the first learnable frequency domain mask is the output of the third activation function layer. The input of the first inverse real-valued FFT2D layer is the output of the first learnable frequency domain mask. The output of the first inverse real-valued FFT2D layer is the output of the global branch. The dynamic routing branch is the first lightweight gated network layer. The input of the first lightweight gated network layer is the input of the dynamically learnable Fourier convolutional layer. The output of the first lightweight gated network layer is the routing weight. The outputs of the local branch, the global branch, and the routing weights are weighted and fused to obtain the final output of the dynamically learnable Fourier convolutional layer. The first lightweight gated network layer includes a first global average pooling layer, a first fully connected layer, a second fully connected layer, and a fourth activation function layer. The input of the first global average pooling layer is the input of the dynamically learnable Fourier convolutional layer, the input of the first fully connected layer is the output of the first global average pooling layer, the input of the second fully connected layer is the output of the first fully connected layer, and the input of the fourth activation function layer is the output of the second fully connected layer. The output of the second fully connected layer is used as the routing weight.
[0010] According to one aspect of the present invention, a Transformer decoder module includes n decoders connected in sequence. Each Transformer decoder includes a sixth normalization layer, a first multi-head self-attention layer, a seventh normalization layer, and a second feedforward neural network layer. The feature sequence passes through the sixth normalization layer, the first multi-head self-attention layer, the seventh normalization layer, and the second feedforward neural network layer in sequence to obtain a decoded feature sequence. The decoded feature sequence is then linearly mapped and rearranged to obtain a reconstructed document image. The input to the sixth normalization layer is the enhanced feature sequence; the input to the first multi-head self-attention layer is the output of the sixth normalization layer; the input to the seventh normalization layer includes the output of the first multi-head self-attention layer and the enhanced feature sequence; and the input to the second feedforward neural network layer is the output of the seventh normalization layer.
[0011] According to one aspect of the present invention, a document image binarization method includes: generating training data based on acquired document image data to train a document image binarization model, specifically including: Obtain ground truth images from public datasets as label images; The document image data is combined with the corresponding tag images to construct the first document image dataset; The document image binarization model was trained using the first document image dataset.
[0012] According to one aspect of the present invention, a document image binarization method includes: preprocessing a first document image dataset.
[0013] According to one aspect of the present invention, a document image binarization method includes: preprocessing a first document image dataset, specifically including: Step S91: According to the preset dataset partitioning rules, divide the first document image dataset... The dataset is divided into training, validation, and test sets. The training subset, training ground truth subset, validation subset, validation ground truth subset, test subset, and test ground truth subset are all initialized as empty sets. An image iteration variable k is defined and initialized to 1. The dataset n is set to the first document's image dataset. Total number of images; Set training set slice size The size of the validation and test sets is [size missing]. Image cutting sliding step size ,in Set the base cutting size ; Step S92: Read the first document image dataset The k-th image and its corresponding label image ,make For the first document image dataset The k-th image The height, For the first document image dataset The k-th image The width of the sample; determine whether the current sample belongs to the training set, validation set, or test set based on the dataset partitioning results; Step S93: When the current sample belongs to the training set, use overlapping segmentation and set the first vertical iteration variable. The initial value is 0, and the vertical coordinate is taken by the following formula: ; Step S94: Set the first horizontal iteration variable j, with an initial value of 0, and take the horizontal coordinate as follows: ; in, It is the horizontal starting coordinate of the j-th slice. It is the vertical starting coordinate of the i-th slice; Step S95: For each ,by Window segmentation; if the window goes out of bounds, perform boundary padding to obtain the input training block. With the corresponding label training block The training block region is defined as: ; ; Step S96: When or When filling missing areas, the block size remains unchanged after filling. The missing dimensions in the height and width directions are calculated as follows: ; ; in, The first cut piece is missing dimensions in the height direction. It refers to the missing dimension in the width direction of the first cut piece; Step S97: Input training blocks Write to the training subset and add the corresponding label training blocks. Write into the training truth subset; Step S98: When the current sample belongs to the validation set or the test set, non-overlapping blocks are used, with a stride of [value missing]. Set the second vertical iteration variable u and the second horizontal iteration variable v, both with an initial value of 0; the vertical and horizontal coordinates are respectively taken as follows: ; ; in, It is the horizontal starting coordinate of the v-th slice. It is the vertical starting coordinate of the u-th slice; Step S99: For each ,by The window is segmented; if the window goes out of bounds, boundary padding is performed to obtain either an input validation block or a test block. With the corresponding label verification block or label test block The verification block or test block region is defined as follows: ; ; Step S910: When or When filling in the missing areas, the dimensions after filling are maintained. The missing dimensions in the height and width directions are calculated as follows: ; ; in, The second cut piece is missing dimensions in the height direction. It refers to the missing dimension in the width direction of the second cut piece; Step S911: If the current sample is a validation set, obtain the validation block. Write to the validation subset and include the corresponding tag validation block. Write the verification truth subset; if the current sample is the test set, the resulting test block... Write the test subset and the corresponding tag test block. Write a subset of the test truth values; Step S912: Add the training subset, training ground truth subset, validation subset, validation ground truth subset, test subset, and test ground truth subset to the second document image dataset. ; Step S913: Increment the value of the image iteration variable k by 1, if If so, proceed to step S92.
[0014] A document image binarization system, applied to the aforementioned document image binarization method, includes: The data acquisition module acquires document image data; The model building module constructs a document image binarization model consisting of a progressive multi-scale window self-attention encoder module, a frequency domain feature enhancement module, and a Transformer decoder module, which are sequentially cascaded. Data flows through each module in sequence to achieve feature extraction, encoding, frequency domain feature enhancement, decoding, and reconstruction, so as to output the binarized document image. The result generation module inputs the acquired document image data into the document image binarization model to obtain the reconstructed document image, and then obtains the binarized result document image after thresholding.
[0015] The advantages of this invention are as follows: It employs a progressive multi-scale window self-attention encoder module to gradually expand the receptive range at different layers. During the encoding stage, it simultaneously captures local details such as character strokes and cross-scale structures such as paragraph structures through a self-attention mechanism using windows of different scales. Furthermore, it introduces a frequency domain feature enhancement module based on Fast Fourier Convolution (FFT). FFT enhances the global branch in the frequency domain and merges it with the local branch, performing joint local and global frequency domain modeling of features to compensate for the insufficient utilization of spectral information by pure spatial domain attention. The combined effect of these two approaches results in superior performance in processing complex degraded images, thereby significantly improving the accuracy and robustness of document image binarization. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a schematic flowchart of a document image binarization method according to the present invention; Figure 2 This is a structural diagram of a document image binarization model for a document image binarization method according to the present invention; Figure 3 This is a structural diagram of the progressive multi-scale window self-attention encoder module of a document image binarization method according to the present invention; Figure 4 This is a structural diagram of the frequency domain feature enhancement module of a document image binarization method according to the present invention; Figure 5 This is a diagram of the dynamically learnable Fourier convolution residual block structure of a document image binarization method according to the present invention. Figure 6 This is a diagram of the dynamically learnable Fourier convolutional layer structure of a document image binarization method according to the present invention. Figure 7 This is a diagram of a lightweight gated network layer structure for a document image binarization method according to the present invention. Figure 8 This is a structural diagram of the Transformer decoder module of a document image binarization method according to the present invention; Figure 9 This is a structural diagram of a document image binarization system according to the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Example 1
[0020] like Figure 1 As shown, a document image binarization method includes the following steps: Step S1: Obtain document image data; The specific steps for obtaining document image data in step S1 are as follows: The merged DIBCO (Document Image Binarization Contest Dataset) public dataset (including 2009–2014 and 2016–2019) was used as the data source for binarized document images. As one of the most authoritative benchmark datasets in the field of document image binarization, the DIBCO public dataset contains degraded document images and their corresponding pixel-level ground truth binary images, covering various degradation types such as uneven lighting, background contamination, and paper aging, making it suitable for evaluating the robustness and generalization ability of models.
[0021] Step S2: Construct a document image binarization model consisting of a progressive multi-scale window self-attention encoder module, a frequency domain feature enhancement module, and a Transformer decoder module cascaded sequentially. Data flows through each module in turn to achieve feature extraction, encoding, frequency domain feature enhancement, decoding, and reconstruction, so as to output a binarized document image. In this embodiment, the progressive multi-scale window self-attention encoder module is used to extract and encode features from the input binarized document image to obtain a feature sequence; the frequency domain feature enhancement module is used to perform global frequency domain feature enhancement processing based on fast Fourier convolution on the feature sequence to obtain an enhanced feature sequence; the Transformer decoder module is used to decode the enhanced feature sequence and perform image reconstruction processing to obtain a reconstructed document image, which is then thresholded to obtain a binarized document image. Wherein, as Figure 2 As shown, the modules are cascaded in the order of data flow: the output of the progressive multi-scale window self-attention encoder module is used as the input of the frequency domain feature enhancement module, the output of the frequency domain feature enhancement module is connected to the Transformer decoder module, and the output of the Transformer decoder module is post-processed to obtain the final binarized document image.
[0022] In this embodiment, the progressive multi-scale window self-attention encoder module consists of 6 improved Transformer encoders, and the Transformer decoder module consists of 6 Transformer decoders. In practical applications, the appropriate module can be selected based on the specific circumstances.
[0023] like Figure 2 As shown, the connection order of the six improved Transformer encoders in the progressive multi-scale window self-attention encoder module is as follows: The input of the progressive multi-scale window self-attention encoder module is connected to the input of the first improved Transformer encoder; the output of the first improved Transformer encoder is connected to the input of the second improved Transformer encoder, the output of the second improved Transformer encoder is connected to the input of the third improved Transformer encoder, the output of the third improved Transformer encoder is connected to the input of the fourth improved Transformer encoder, the output of the fourth improved Transformer encoder is connected to the input of the fifth improved Transformer encoder, the output of the fifth improved Transformer encoder is connected to the input of the sixth improved Transformer encoder, and the output of the sixth improved Transformer encoder is the output of the progressive multi-scale window self-attention encoder module.
[0024] like Figure 3 As shown, any of the six improved Transformer encoders includes one progressive window multi-head self-attention layer, one feedforward neural network layer, and two normalization layers, specifically: a first normalization layer, a first progressive window multi-head self-attention layer, a second normalization layer, and a first feedforward neural network layer.
[0025] The specific connection sequence is as follows: the input of the improved Transformer encoder is connected to the input of the first normalization layer; the output of the first normalization layer is connected to the input of the first asymptotic window multi-head self-attention layer; the output of the first asymptotic window multi-head self-attention layer is connected to the input residual of the improved Transformer encoder and then connected to the input of the second normalization layer; the output of the second normalization layer is connected to the input of the first feedforward neural network; the output of the first feedforward neural network is connected to the input residual of the second normalization layer, and the output of the improved Transformer encoder is the output of the first normalization layer.
[0026] The binarized document images in the first document image dataset are sequentially processed by the improved Transformer encoder through a first normalization layer, a multi-head self-attention layer based on a progressive window, a feedforward neural network layer, and a second normalization layer for feature extraction and encoding, resulting in a feature sequence.
[0027] like Figure 4 As shown, the frequency domain feature enhancement module includes one sequence-to-two-dimensional mapping layer, L dynamically learnable Fourier convolutional residual blocks, and one two-dimensional-to-sequence mapping layer. In this embodiment, four dynamically learnable Fourier convolutional residual blocks are selected, but the specific number can be chosen according to the actual application. The specific connection order is as follows: The input of the frequency domain feature enhancement module is connected to the input of the sequence-to-two-dimensional mapping layer. The output of the sequence-to-two-dimensional mapping layer is connected to the input of the first dynamically learnable Fourier convolutional residual block. The output of the first dynamically learnable Fourier convolutional residual block is connected to the input of the second dynamically learnable Fourier convolutional residual block. The output of the second dynamically learnable Fourier convolutional residual block is connected to the input of the third dynamically learnable Fourier convolutional residual block. The output of the third dynamically learnable Fourier convolutional residual block is connected to the input of the fourth dynamically learnable Fourier convolutional residual block. The output of the fourth dynamically learnable Fourier convolutional residual block is connected to the input of the two-dimensional-to-sequence mapping layer. The output of the two-dimensional-to-sequence mapping layer is the output of the frequency domain feature enhancement module.
[0028] like Figure 5 As shown, any one of the four dynamically learnable Fourier convolutional residual blocks includes two dynamically learnable Fourier convolutional layers, namely the first dynamically learnable Fourier convolutional layer and the second dynamically learnable Fourier convolutional layer, with the specific connection order as follows: The input of the dynamically learnable Fourier convolutional residual block is connected to the input of the first dynamically learnable Fourier convolutional layer. The output of the first dynamically learnable Fourier convolutional layer is connected to the input of the second dynamically learnable Fourier convolutional layer. The output of the second dynamically learnable Fourier convolutional layer is the output of the dynamically learnable Fourier convolutional residual block after being concatenated with the input residual of the dynamically learnable Fourier convolutional residual block.
[0029] like Figure 6 As shown, any dynamically learnable Fourier convolutional layer includes local branches, global branches, and dynamic routing branches, specifically: The local branch includes a first convolutional layer, a third normalization layer, and a first activation function layer. The input of the first convolutional layer is connected to the input of the dynamically learnable Fourier convolutional layer. The input of the third normalization layer is connected to the output of the first convolutional layer. The input of the first activation function layer is connected to the output of the third normalization layer. The output of the first activation function layer is the output of the local branch.
[0030] The global branch includes a second convolutional layer, a fourth normalization layer, a second activation function layer, a first real-valued FFT2D layer, a third convolutional layer, a fifth normalization layer, a third activation function layer, a first learnable frequency domain mask, and a first inverse real-valued FFT2D layer. The input of the second convolutional layer is connected to the input of the dynamically learnable Fourier convolutional layer. The input of the fourth normalization layer is connected to the output of the second convolutional layer. The input of the second activation function layer is connected to the output of the fourth normalization layer. The input of the first real-valued FFT2D layer is connected to the output of the second activation function layer. The input of the third convolutional layer is connected to the output of the first real-valued FFT2D layer. The input of the fifth normalization layer is connected to the output of the third convolutional layer. The input of the third activation function layer is connected to the output of the fifth normalization layer. The input of the first learnable frequency domain mask is connected to the output of the third activation function layer. The input of the first inverse real-valued FFT2D layer is connected to the output of the first learnable frequency domain mask. The output of the first inverse real-valued FFT2D layer is the output of the global branch.
[0031] The dynamic routing branch is the first lightweight gated network layer. The input of the first lightweight gated network layer is the input of the dynamically learnable Fourier convolutional layer, and the output of the first lightweight gated network layer is the routing weight. ; Output of local branch, output of global branch, and routing weight Weighted fusion yields the final output of the dynamically learnable Fourier convolutional layer. The formula for weighted fusion is defined as follows: ; In the formula, H is the final output of the dynamically learnable Fourier convolutional layer. For local branch output, For global branch output, This represents the routing weight.
[0032] like Figure 7 As shown, the first lightweight gated network layer includes one global average pooling layer, two fully connected layers, and one activation function layer, specifically: a first global average pooling layer, a first fully connected layer, a second fully connected layer, and a fourth activation function layer. The specific connection order is as follows: The input of the first global average pooling layer is connected to the input of the first lightweight gated network layer. The input of the first fully connected layer is connected to the output of the first global average pooling layer. The input of the second fully connected layer is connected to the output of the first fully connected layer. The input of the fourth activation function layer is connected to the output of the second fully connected layer, and its output is used as the routing weight. .
[0033] The feature sequence output by the progressive multi-scale window self-attention encoder module is processed by the frequency domain feature enhancement module to output an enhanced feature sequence.
[0034] like Figure 2As shown, the connection order of the six Transformer decoders in the Transformer decoder module is as follows: The input of the Transformer decoder module is connected to the input of the first Transformer decoder; the output of the first Transformer decoder is connected to the input of the second Transformer decoder; the output of the second Transformer decoder is connected to the input of the third Transformer decoder; the output of the third Transformer decoder is connected to the input of the fourth Transformer decoder; the output of the fourth Transformer decoder is connected to the input of the fifth Transformer decoder; the output of the fifth Transformer decoder is connected to the input of the sixth Transformer decoder; and the output of the sixth Transformer decoder is the output of the Transformer decoder module.
[0035] like Figure 8 As shown, any one of the six Transformer decoders in the Transformer decoder module includes one multi-head self-attention layer, one feedforward neural network layer, and two normalization layers. Specifically, it consists of the sixth normalization layer, the first multi-head self-attention layer, the seventh normalization layer, and the second feedforward neural network layer, with the following connection order: The input of the Transformer decoder is connected to the input of the sixth normalization layer. The output of the sixth normalization layer is connected to the input of the first multi-head self-attention layer. The output of the first multi-head self-attention layer is concatenated with the input residual of the Transformer decoder and then connected to the input of the seventh normalization layer. The output of the seventh normalization layer is connected to the input of the second feedforward neural network layer. The output of the second feedforward neural network layer is concatenated with the input residual of the seventh normalization layer and then connected to the input of the seventh normalization layer. This output is the output of the Transformer decoder.
[0036] The enhanced feature sequence output by the frequency domain feature enhancement module is processed by six Transformer decoders in sequence to gradually restore the spatial structure information and output a binarized reconstructed image.
[0037] In this embodiment, the document image binarization method further includes: generating training data based on the acquired document image data to train the document image binarization model, specifically including: Step S21: Construct the training dataset Read the ground truth images from the public dataset as label images; combine the document image data with the corresponding label images to construct the first document image dataset. Then, the first document image dataset... Preprocessing to generate training set Validation set and test set .
[0038] Step S22: Initialize the document image binarization model Set the embedding dimension of either the improved encoder or decoder in the progressive multiscale window self-attention encoder module and the Transformer decoder module to... The number of heads for multi-head attention is set to The improved Transformer encoder's embedding block size is set to The remaining model parameters are randomly initialized to obtain the second document image binarization model. .
[0039] Step S23: Train the initialized document image binarization model based on the training dataset. Using the training set and verification set Binarization model of the second document image Training is performed to obtain the third document image binarization model. The specific steps are as follows: Step S211: Set the number of training rounds for the model. Set the training data batch size Set the learning rate for the AdamW optimizer. Set AdamW's momentum parameters Set the parameters for L2 regularization Set the optimal verification evaluation value. Optimal verification evaluation value The initial value is set to 0, and the optimal document image binarization model is set. Optimal document image binarization model The parameters are initialized to the second document image binarization model. The parameters are: let epoch represent the current training epoch, and initialize the current training epoch to 1.
[0040] Step S212: Load all necessary training hyperparameters from the configuration file, including image patch size. Number of dynamically learnable Fourier convolution residual blocks and the initial size of the progressive window And use the data loader to load the first document image dataset. training set and verification set .
[0041] Step S213: Load the model onto the corresponding device according to the hardware environment.
[0042] Step S214: Begin training the model for each epoch, with each epoch covering the entire training set. For any training batch, the input image block and its corresponding label image block are obtained respectively. The input image block is fed into the document image binarization model for forward inference to obtain the prediction block. The training loss is calculated using the mean squared error loss function. Then, the loss is backpropagated and the model parameters are updated to minimize the training loss.
[0043] Step S215: After each round of training, use the validation set. The validation process is performed, the validation loss is calculated, and the visualization results are generated. The visualization results include the degraded input image, the corresponding ground truth image, and the thresholded binary prediction image of the model output.
[0044] Step S216: Calculate the PSNR index on the validation results to obtain the validation set. Average PSNR value in the current training round .
[0045] Step S217, if Then the optimal verification evaluation value will be The value is set to And the optimal document image binarization model The parameters are updated to the binarization model of the first document image in the current training round. The parameters.
[0046] Step S218: Increment the value of the current training epoch by 1; if epoch < epochs, proceed to step S214; otherwise, output the optimal model corresponding to the optimal validation evaluation value. And the optimal document image binarization model Save as a third-party document image binarization model .
[0047] Step S3: Input the acquired document image data into the document image binarization model to obtain the reconstructed document image, and then obtain the binarized document image after thresholding; specifically: Organize document image data according to image block size The feature sequence is segmented, mapped to a feature embedding sequence, and positional encoding is added. Then, it is input into a progressive multi-scale window self-attention encoder module consisting of 6 improved Transformer encoders. Multi-scale context modeling is achieved through the progressive multi-scale window self-attention mechanism, and the encoded feature sequence is output. The encoded feature sequence is input into the frequency domain feature enhancement module. The frequency domain feature enhancement module first rearranges the feature sequence into a two-dimensional feature map, and then inputs it into the frequency domain enhancement network composed of multiple dynamically learnable Fourier convolution residual blocks in sequence to perform local + global joint frequency domain modeling and enhancement of the features. Finally, the enhanced two-dimensional feature map is mapped back to the enhanced feature sequence. The enhanced feature sequence is input to a Transformer decoder module consisting of six Transformer decoders. The decoders decode and reconstruct the enhanced feature sequence through a multi-head self-attention mechanism and a feedforward neural network, and output the reconstructed feature sequence. After linear mapping and rearrangement, the reconstructed document image is obtained. The reconstructed document image is then thresholded to obtain the final binarized document image.
[0048] Example 2
[0049] A document image binarization method is provided. The only difference between this embodiment and Embodiment 1 is that the document image binarization method includes: preprocessing a first document image dataset.
[0050] Specifically, the first document image dataset Image segmentation and extraction were performed to obtain the second document image dataset. The specific steps are as follows: Step S21: According to the preset dataset partitioning rules (e.g., 6:2:2), divide the first document image dataset... Divided into training set Validation set and test set Initialize the training subset, training ground truth subset, validation subset, validation ground truth subset, test subset, and test ground truth subset to be empty sets; define the image iteration variable k, with an initial value of 1; and set n to the first document image dataset. Total number of images; Set training set slice size The size of the validation and test sets is [size missing]. Image cutting sliding step size ,in Set the base cutting size ; Step S22: Read the first document image dataset The k-th image and its corresponding label image ,make For the first document image dataset The k-th image The height, For the first document image dataset The k-th image Width; determine whether the current sample belongs to the training set based on the dataset partitioning results. Validation set or test set ; Step S23: The current sample belongs to the training set. At that time, overlapping blocks are used, and the first vertical iteration variable is set. The initial value is 0, and the vertical coordinate is taken by the following formula: ; Step S24: Set the first horizontal iteration variable j, with an initial value of 0, and take the horizontal coordinate as follows: ; in, It is the horizontal starting coordinate of the j-th slice. It is the vertical starting coordinate of the i-th slice; Step S25: For each ,by Window segmentation; if the window goes out of bounds, perform boundary padding to obtain the input training block. With label training blocks The training block region is defined as follows: ; ; Step S26: When or When the window overflows (i.e., the window goes out of bounds), the missing area is filled, and the block size is maintained after filling. The missing dimensions in the height and width directions are calculated as follows: ; ; in, The first cut piece is missing dimensions in the height direction. It refers to the missing dimension in the width direction of the first cut piece; Step S27: Input training blocks obtained Write to the training subset, corresponding label block Write into the training truth subset; Step S28: The current sample belongs to the validation set. or test set When using non-overlapping blocks, the step size is taken as follows: Set the second vertical iteration variable u and the second horizontal iteration variable v, both with an initial value of 0; the vertical and horizontal coordinates are respectively taken as follows: ; ; in, It is the horizontal starting coordinate of the v-th slice. It is the vertical starting coordinate of the u-th slice; Step S29: For each ,by The window is segmented; if the window goes out of bounds, boundary padding is performed to obtain either an input validation block or a test block. With label verification block or label test block The verification block or test block region is defined as follows: ; ; Step S210: When or When the window goes out of bounds, the missing areas in both the input and label images are uniformly filled with white pixels, and the size of the filled area is maintained. The missing dimensions in the height and width directions are calculated as follows: ; ; in, The second cut piece is missing dimensions in the height direction. It refers to the missing dimension in the width direction of the second cut piece; Step S211: If the current sample is a validation set The obtained verification block Write to the validation subset and include the corresponding tag validation block. Write to the validation truth subset; if the current sample is the test set. The obtained test block Write the test subset and the corresponding tag test block. Write a subset of the test truth values; Step S212: Add the training subset, training ground truth subset, validation subset, validation ground truth subset, test subset, and test ground truth subset to the second document image dataset. ; Step S213: Increment the value of the image iteration variable k by 1, if If so, proceed to step S22.
[0051] Example 3
[0052] like Figure 9 As shown, a document image binarization system is provided. This embodiment applies the document image binarization method of Embodiment 1 and / or Embodiment 2. The system includes: The data acquisition module acquires document image data; The model building module constructs a document image binarization model consisting of a progressive multi-scale window self-attention encoder module, a frequency domain feature enhancement module, and a Transformer decoder module, which are sequentially cascaded. By passing data through each module in sequence, feature extraction, encoding, frequency domain feature enhancement, decoding, and reconstruction are achieved to output a binarized document image. The result generation module inputs the acquired document image data into the document image binarization model to obtain the reconstructed document image, and then obtains the binarized result document image after thresholding.
[0053] The beneficial effects of this invention are as follows: A progressive multi-scale window self-attention encoder module is employed to gradually expand the receptive range at different layers. During the encoding stage, the self-attention mechanism of different scale windows simultaneously captures local details such as character strokes and cross-scale structures such as paragraph structures. Furthermore, a frequency domain feature enhancement module based on Fast Fourier Convolution is introduced. Fast Fourier Convolution enhances the global branch in the frequency domain and merges it with the local branch, performing joint local and global frequency domain modeling of features to compensate for the insufficient utilization of spectral information by pure spatial domain attention. The combination of these two approaches ultimately results in superior performance in processing complex degraded images, thereby significantly improving the accuracy and robustness of document image binarization.
[0054] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A document image binarization method, characterized in that: The methods include: Step S1: Obtain document image data; Step S2: Construct a document image binarization model consisting of a progressive multi-scale window self-attention encoder module, a frequency domain feature enhancement module, and a Transformer decoder module cascaded sequentially. Data flows through the progressive multi-scale window self-attention encoder module, the frequency domain feature enhancement module, and the Transformer decoder module in sequence to achieve feature extraction, encoding, frequency domain feature enhancement, decoding, and reconstruction, and outputs the binarized document image. Step S3: Input the acquired document image data into the document image binarization model to obtain the reconstructed document image. After thresholding, the reconstructed document image is used to obtain the binarized document image. The frequency domain feature enhancement module includes a sequence-to-2D mapping layer, L dynamically learnable Fourier convolutional residual blocks, and a 2D-to-sequence mapping layer. The feature sequence output by the progressive multi-scale window self-attention encoder module is first rearranged into a 2D feature map through the sequence-to-2D mapping layer, and then sequentially input into the L dynamically learnable Fourier convolutional residual blocks for local and global joint frequency domain enhancement processing. Finally, the enhanced 2D feature map is input into the 2D-to-sequence mapping layer to map back to the sequence features, resulting in an enhanced feature sequence. The dynamically learnable Fourier convolutional residual block includes a first dynamically learnable Fourier convolutional layer and a second dynamically learnable Fourier convolutional layer. The input of the dynamically learnable Fourier convolutional residual block is connected to the input of the first dynamically learnable Fourier convolutional layer, the output of the first dynamically learnable Fourier convolutional layer is connected to the input of the second dynamically learnable Fourier convolutional layer, and the output of the second dynamically learnable Fourier convolutional layer after concatenating the input residual of the dynamically learnable Fourier convolutional residual block is the output of the dynamically learnable Fourier convolutional residual block. Dynamically learnable Fourier convolutional layers include local branches, global branches, and dynamic routing branches, specifically: The local branch includes a first convolutional layer, a third normalization layer, and a first activation function layer. The input of the first convolutional layer is the input of the dynamically learnable Fourier convolutional layer. The input of the third normalization layer is the output of the first convolutional layer. The input of the first activation function layer is the output of the third normalization layer. The output of the first activation function layer is the output of the local branch. The global branch includes a second convolutional layer, a fourth normalization layer, a second activation function layer, a first real-valued FFT2D layer, a third convolutional layer, a fifth normalization layer, a third activation function layer, a first learnable frequency domain mask, and a first inverse real-valued FFT2D layer. The input of the second convolutional layer is the input of the dynamically learnable Fourier convolutional layer. The input of the fourth normalization layer is the output of the second convolutional layer. The input of the second activation function layer is the output of the fourth normalization layer. The input of the first real-valued FFT2D layer is the output of the second activation function layer. The input of the third convolutional layer is the output of the first real-valued FFT2D layer. The input of the fifth normalization layer is the output of the third convolutional layer. The input of the third activation function layer is the output of the fifth normalization layer. The input of the first learnable frequency domain mask is the output of the third activation function layer. The input of the first inverse real-valued FFT2D layer is the output of the first learnable frequency domain mask. The output of the first inverse real-valued FFT2D layer is the output of the global branch. The dynamic routing branch is the first lightweight gated network layer. The input of the first lightweight gated network layer is the input of the dynamically learnable Fourier convolutional layer. The output of the first lightweight gated network layer is the routing weight. The outputs of the local branch, the global branch, and the routing weights are weighted and fused to obtain the final output of the dynamically learnable Fourier convolutional layer. The first lightweight gated network layer includes a first global average pooling layer, a first fully connected layer, a second fully connected layer, and a fourth activation function layer. The input of the first global average pooling layer is the input of the dynamically learnable Fourier convolutional layer, the input of the first fully connected layer is the output of the first global average pooling layer, the input of the second fully connected layer is the output of the first fully connected layer, and the input of the fourth activation function layer is the output of the second fully connected layer. The output of the second fully connected layer is used as the routing weight.
2. The document image binarization method according to claim 1, characterized in that: The progressive multi-scale window self-attention encoder module is used to extract and encode features from the input binary document image to obtain a feature sequence. The frequency domain feature enhancement module is used to perform global frequency domain feature enhancement processing on the feature sequence based on fast Fourier convolution to obtain the enhanced feature sequence; The Transformer decoder module is used to decode the enhanced feature sequence and perform image reconstruction to obtain the reconstructed document image, which is then thresholded to obtain the binarized document image.
3. The document image binarization method according to claim 2, characterized in that: The progressive multi-scale window self-attention encoder module contains m sequentially connected improved Transformer encoders. Each improved Transformer encoder includes a first normalization layer, a first progressive window multi-head self-attention layer, a second normalization layer, and a first feedforward neural network layer. The binarized document image is sequentially passed through the first normalization layer, the first progressive window multi-head self-attention layer, the second normalization layer, and the first feedforward neural network layer to obtain the encoded feature sequence.
4. The document image binarization method according to claim 3, characterized in that: The Transformer decoder module contains n decoders connected in sequence. Each Transformer decoder includes a sixth normalization layer, a first multi-head self-attention layer, a seventh normalization layer, and a second feedforward neural network layer. The feature sequence passes through the sixth normalization layer, the first multi-head self-attention layer, the seventh normalization layer, and the second feedforward neural network layer in sequence to obtain the decoded feature sequence. The reconstructed document image is obtained by linear mapping and image rearrangement of the feature sequence; The input to the sixth normalization layer is the enhanced feature sequence; the input to the first multi-head self-attention layer is the output of the sixth normalization layer; the input to the seventh normalization layer includes the output of the first multi-head self-attention layer and the enhanced feature sequence; and the input to the second feedforward neural network layer is the output of the seventh normalization layer.
5. A document image binarization method according to claim 4, characterized in that: Document image binarization methods include: generating training data from acquired document image data to train a document image binarization model, specifically including: Obtain ground truth images from public datasets as label images; The document image data is combined with the corresponding tag images to construct the first document image dataset; The document image binarization model was trained using the first document image dataset.
6. The document image binarization method according to claim 5, characterized in that: The document image binarization method includes: preprocessing the first document image dataset.
7. A document image binarization method according to claim 6, characterized in that: The document image binarization method includes: preprocessing the first document image dataset, specifically including the following steps: Step S91: According to the preset dataset partitioning rules, divide the first document image dataset... The dataset is divided into training, validation, and test sets. The training subset, training ground truth subset, validation subset, validation ground truth subset, test subset, and test ground truth subset are all initialized as empty sets. An image iteration variable k is defined and initialized to 1. The dataset n is set to the first document's image dataset. Total number of images; Set training set slice size The size of the validation and test sets is [size missing]. Image cutting sliding step size ,in Set the base cutting size ; Step S92: Read the first document image dataset The k-th image and its corresponding label image ,make For the first document image dataset The k-th image The height, For the first document image dataset The k-th image The width of the sample; determine whether the current sample belongs to the training set, validation set, or test set based on the dataset partitioning results; Step S93: When the current sample belongs to the training set, use overlapping segmentation and set the first vertical iteration variable. The initial value is 0, and the vertical coordinate is taken by the following formula: ; Step S94: Set the first horizontal iteration variable j, with an initial value of 0, and take the horizontal coordinate as follows: ; in, It is the horizontal starting coordinate of the j-th slice. It is the vertical starting coordinate of the i-th slice; Step S95: For each ,by Window segmentation; if the window goes out of bounds, perform boundary padding to obtain the input training block. With the corresponding label training block The training block region is defined as: ; ; Step S96: When or When filling missing areas, the block size remains unchanged after filling. The missing dimensions in the height and width directions are calculated as follows: ; ; in, The first cut piece is missing dimensions in the height direction. It refers to the missing dimension in the width direction of the first cut piece; Step S97: Input training blocks Write to the training subset and add the corresponding label training blocks. Write into the training truth subset; Step S98: When the current sample belongs to the validation set or the test set, non-overlapping blocks are used, with a stride of [value missing]. Set the second vertical iteration variable u and the second horizontal iteration variable v, both with an initial value of 0; the vertical and horizontal coordinates are respectively taken as follows: ; ; in, It is the horizontal starting coordinate of the v-th slice. It is the vertical starting coordinate of the u-th slice; Step S99: For each ,by The window is segmented; if the window goes out of bounds, boundary padding is performed to obtain either an input validation block or a test block. With the corresponding label verification block or label test block The verification block or test block region is defined as follows: ; ; Step S910: When or When filling in the missing areas, the dimensions after filling are maintained. The missing dimensions in the height and width directions are calculated as follows: ; ; in, The second cut piece is missing dimensions in the height direction. It refers to the missing dimension in the width direction of the second cut piece; Step S911: If the current sample is a validation set, obtain the validation block. Write to the validation subset and include the corresponding tag validation block. Write the verification truth subset; if the current sample is the test set, the resulting test block... Write the test subset and the corresponding tag test block. Write a subset of the test truth values; Step S912: Add the training subset, training ground truth subset, validation subset, validation ground truth subset, test subset, and test ground truth subset to the second document image dataset. ; Step S913: Increment the value of the image iteration variable k by 1, if If so, proceed to step S92.
8. A document image binarization system, applied to the document image binarization method of claim 7, characterized in that: include: The data acquisition module acquires document image data; The model building module constructs a document image binarization model consisting of a progressive multi-scale window self-attention encoder module, a frequency domain feature enhancement module, and a Transformer decoder module, which are sequentially cascaded. Data flows through each module in sequence to achieve feature extraction, encoding, frequency domain feature enhancement, decoding, and reconstruction, so as to output the binarized document image. The result generation module inputs the acquired document image data into the document image binarization model to obtain the reconstructed document image, and then obtains the binarized result document image after thresholding.
Citation Information
Patent Citations
Image adaptive binarization method based on edge information
CN116363158A
Document image binarization method and system, computer equipment and readable storage medium
CN116798043A
Degenerated document image binarization method and system based on adaptive convolution
CN119444582A
Monocular depth prediction algorithm based on multi-scale progressive interaction and aggregation cross attention features
CN116485860A
Document image binarization method and system based on deep learning
CN120411982A
