Document tampering detection method based on sliding window frequency domain feature fusion
The document image is processed through the sliding window and frequency domain feature fusion model, which solves the problems of error detection and poor robustness in document tamper detection, and achieves more accurate tampering area identification and performance improvement.
Patent Information
- Application Number
- CN202210317167.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-29
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-03-29
AI Technical Summary
Existing document tamper detection methods are prone to error detection when processing document images, and shape adjustment and filling interference caused by different text lines lengths affect the recognition of tamper features. It is poorly robust and difficult to identify smaller tampering. area.
The text lines of the document image are extracted using a sliding window, and the tamper prediction is performed through the frequency domain feature fusion model. The low, medium and high frequency coefficient features are extracted using two-dimensional DCT transformation and filters, and further extracted with the MobileNetV3 small model. , form a score list corresponding to the text line, reduce shape adjustment and fill interference, and improve robustness.
Through the fusion of sliding windows and frequency domain features, error detection is reduced, model performance and robustness are improved, and document tampering areas can be more accurately identified.
Smart Images

Figure CN114677690B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a document tampering detection method based on sliding window frequency domain feature fusion, which is applicable to the document tampering detection field. Background Art
[0002] There are currently few public document tampering detection solutions, and most of them treat this problem as a common image tampering detection problem.
[0003] For common image tampering detection solutions, the entire document image is input into the model for tampering area detection, which often leads to too many false detections. And too many false detections will increase labor costs.
[0004] Of course, there are also some solutions that extract text lines in key areas based on document types and use the text lines as input for tampering detection. This method can effectively reduce false detections because it excludes some areas with no tampering value. However, due to the different lengths of text lines, this solution usually requires additional shape adjustments during training to make all text lines the same size, which will cause text deformation and introduce additional noise to interfere with the identification of tampering features. Alternatively, all text lines are filled with black to a uniform size during training. However, when the length difference of text lines is too large, the proportion of text in the short text line after filling will be too small, which is also not conducive to the identification of tampering features.
[0005] In addition, decisions based on a single image or text line often have poor robustness, and the tampered areas of many documents are very small, so the larger input makes it difficult to identify tampering. Summary of the invention
[0006] The technical problem to be solved by the present invention is: in view of the above-mentioned problems, a document tampering detection method based on sliding window frequency domain feature fusion is provided.
[0007] The technical solution adopted by the present invention is: a document tampering detection method based on sliding window frequency domain feature fusion, characterized in that:
[0008] S1. Extract image blocks from the text lines of the document image using a sliding window, where the sliding window has a length of W', a height of H, and a sliding step of S, wherein W' is less than the length W of the corresponding text line, H is consistent with the height of the corresponding text line, and S is less than W';
[0009] S2, inputting the image block obtained through the sliding window into the frequency domain feature fusion model to obtain the tampering prediction score corresponding to the image block;
[0010] S3. Divide the text line into multiple sub-regions according to the sliding step size S, and use the average of the tampering prediction scores of the image block where the sub-region is located as the regional tampering score of the sub-region, to form a score list corresponding to the text line, and each regional tampering score on the score list corresponds one-to-one to each sub-region of the text line.
[0011] Before step S1, the method further includes:
[0012] Input the document image to be detected into the document classifier for classification, obtain the corresponding document type, and select the corresponding key area segmentor based on the document type;
[0013] The document image to be detected is input into the corresponding key area segmenter to extract the image key area of the document image, and the image key area is used to extract the image block by using the sliding window in step S1.
[0014] The step of inputting the image block obtained through the sliding window into the frequency domain feature fusion model to obtain the tampering prediction score corresponding to the image block includes:
[0015] The image block is transformed into two-dimensional DCT, and the frequency domain coefficient characteristics after DCT transformation are filtered using three adaptive fine-tuning low, medium and high filters to obtain low-frequency, medium-frequency and high-frequency coefficient characteristics, which are set as F L , F M , F H ;
[0016] Then, the three frequency domain coefficient features are subjected to inverse DCT transformation to obtain three filtered images, which are set as f L , f M , f H ;
[0017] The three inverse DCT transformed images and the original image blocks are channel-wise concatenated and input into the MobileNetV3 small model to further extract features and output the tampering prediction score of the corresponding image block.
[0018] The step of performing a two-dimensional DCT transformation on the image block comprises:
[0019] The DCT transform formula is as follows:
[0020] F=AfA T
[0021]
[0022]
[0023] Among them, F is the frequency domain coefficient obtained by DCT transformation; f is the pixel value of the image block; A is the transformation matrix; i is the horizontal frequency of the two-dimensional wave, j is the vertical frequency of the two-dimensional wave, and the value range is [0, N-1], and N is the size of the image block.
[0024] The inverse DCT transform comprises:
[0025] f L =AF L A T
[0026] f M =AF M A T
[0027] f H =AF H A T
[0028] A document tampering detection device based on sliding window frequency domain feature fusion, characterized by comprising:
[0029] An image block extraction module is used to extract image blocks from text lines of a document image using a sliding window, wherein the sliding window has a length of W', a height of H, and a sliding step of S, wherein W' is less than the length W of the corresponding text line, H is consistent with the height of the corresponding text line, and S is less than W';
[0030] An image block score prediction module is used to input the image block obtained through the sliding window into a frequency domain feature fusion model to obtain a tampering prediction score corresponding to the image block;
[0031] The regional score calculation module is used to divide the text line into multiple sub-regions according to the sliding step size S, and take the average of the tampering prediction scores of the image block where the sub-region is located as the regional tampering score of the sub-region, forming a score list corresponding to the text line, and each regional tampering score on the score list corresponds one-to-one to each sub-region of the text line.
[0032] A storage medium stores a computer program that can be executed by a processor, characterized in that: when the computer program is executed, the steps of the document tampering detection method based on sliding window frequency domain feature fusion are implemented.
[0033] A computer device has a memory and a processor, wherein the memory stores a computer program that can be executed by the processor, characterized in that when the computer program is executed, the steps of the document tampering detection method based on sliding window frequency domain feature fusion are implemented.
[0034] The beneficial effects of the present invention are as follows: the present invention utilizes the characteristics of the sliding window to divide the text lines, extracts multiple image blocks of the same size in each text line for training and reasoning, reduces the degradation of tampering detection performance due to shape adjustment or additional padding, reduces additional changes and interference to the text lines, improves the quality of training and reasoning data, and thus improves model performance.
[0035] The present invention adds frequency domain feature extraction to the model, making the tampering features more obvious and improving the model tampering detection capability.
[0036] The present invention alleviates prediction jitter by fusing the image block scores at the corresponding positions of each sub-region of the text line, making the overall solution more robust. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 Schematic diagram of the process of the embodiment.
[0038] Figure 2 It is a structural block diagram of the frequency domain feature fusion model in the embodiment.
[0039] Figure 3 This is a schematic diagram of the fusion of tampering prediction scores in the embodiment. DETAILED DESCRIPTION
[0040] This embodiment is a document tampering detection method based on sliding window frequency domain feature fusion, which specifically includes the following steps:
[0041] S0. Input the document image to be detected into the document classifier for classification, obtain the corresponding document type, and select the corresponding key area segmentor based on the document type; input the document image to be detected into the corresponding key area segmentor to extract the image key area of the document image.
[0042] In this embodiment, different document types correspond to different key area segmenters, for example Figure 1 , the input image type is a screenshot of the travel code. The key area output by the key area segmentor of this type of document is the white area in the middle. For other types of document images, there are also specific key areas.
[0043] S1. Use a sliding window to extract image blocks from the text line of the document image. Assuming that the length of the text line is W and the height is H, the length of the sliding window is W', satisfying W'<W, the height of the sliding window is the same as the height of the text line, which is H, and the sliding step size of the sliding window is S, S<W'.
[0044] S2, the image block obtained by sliding window is input into the frequency domain feature fusion model (see Figure 2 ), and obtain the tampering prediction score corresponding to the image block.
[0045] S21, perform a two-dimensional DCT transformation on the input image block. The DCT transformation formula is as follows:
[0046] F=AfA T
[0047]
[0048]
[0049] Among them, F is the frequency domain coefficient obtained by DCT transformation; f is the pixel value of the image; A is the conversion matrix, i is the horizontal frequency of the two-dimensional wave; j is the vertical frequency of the two-dimensional wave, and the value range is [0, N-1], and N is the size of the image block.
[0050] S22, use three adaptively fine-tuned low, medium and high filters to filter the frequency domain coefficient characteristics after DCT transformation to obtain low-frequency, medium-frequency and high-frequency coefficient characteristics, which are set as F L , F M , F H .
[0051] S23, F L , F M , F H The three frequency domain coefficient features are subjected to inverse DCT transformation to obtain three filtered images, which are set as f L , f M , f H The DCT inverse transform process formula is as follows:
[0052] f L =AF L A T
[0053] f M =AF M A T
[0054] f H =AF H A T
[0055] Finally, three inverse DCT transformed images are obtained.
[0056] S24. Channel-wise splicing of the three inverse DCT transformed images and the original image blocks is performed, and the images are input into the MobileNetV3small (Howard, Andrew, et al. "Searching for mobilenetv3." Proceedings of the IEEE / CVF International Conference on Computer Vision. 2019.) model to further extract features and output the tampering prediction score of the corresponding image block.
[0057] S3, divide the text line into multiple sub-regions according to the sliding step size S, and take the average of the tampering prediction scores of the image blocks where the sub-regions are located as the regional tampering scores of the sub-regions, forming a score list corresponding to the text line, and each regional tampering score on the score list corresponds to each sub-region of the text line. This step is used to fuse the obtained image block scores, so as to make the tampering prediction more stable and improve the robustness of the overall solution.
[0058] Figure 3 Image blocks 1 to 7 represent 7 image blocks extracted based on the sliding window, and the decimal on the right represents the score corresponding to each image block. Each wireframe frames the image block involved in the sub-region of the sliding step size in the text line. The final prediction result of the sub-region of the sliding step size is the average of the tampering prediction scores of all image blocks involved. For example, the first wireframe on the left only involves image block 1, so the tampering score corresponding to the text line of the sliding step size is the score of image block 1, the second wireframe on the left involves image block 1 and image block 2, so the tampering score corresponding to this part of the text line is the average of the scores of image block 1 and image block 2, and so on. Here, the remaining image blocks smaller than the sliding step size are not included in the mean calculation. The above method can be used to obtain a score list for this part of the text line. Each value in the list represents the tampering score of the text line in the sub-region of the sliding step size S.
[0059] When there is a part greater than 0.5 in the score list, the text line is judged to be tampered. And because each score in the list represents the tampering of a sub-text line with a different length, the specific tampering position in the text line can be located based on the text line score list.
[0060] This embodiment is a document tampering detection device based on sliding window frequency domain feature fusion, including: an image block extraction module, an image block score prediction module and a region score calculation module.
[0061] In this example, the image block extraction module is used to extract image blocks from text lines of document images using a sliding window, where the sliding window has a length of W', a height of H, and a sliding step of S, wherein W' < the length W of the corresponding text line, H is consistent with the height of the corresponding text line, and S < W'; the image block score prediction module is used to input the image block obtained through the sliding window into a frequency domain feature fusion model to obtain a tampering prediction score corresponding to the image block; the regional score calculation module is used to divide the text line into multiple sub-regions according to the sliding step S, and use the average of the tampering prediction scores of the image blocks in the sub-region as the regional tampering score of the sub-region, to form a score list corresponding to the text line, and each regional tampering score on the score list corresponds one-to-one to each sub-region of the text line.
[0062] This embodiment also provides a storage medium on which a computer program executable by a processor is stored. When the computer program is executed, the steps of the document tampering detection method based on sliding window frequency domain feature fusion in this example are implemented.
[0063] This embodiment also provides a computer device having a memory and a processor, wherein the memory stores a computer program that can be executed by the processor, and when the computer program is executed, the steps of the document tampering detection method based on sliding window frequency domain feature fusion in this example are implemented.
Claims
1. A document tampering detection method based on sliding window frequency domain feature fusion, characterized by: S1. Extract image blocks from the text lines of the document image using a sliding window, where the sliding window has a length of W', a height of H, and a sliding step of S, wherein W' is less than the length W of the corresponding text line, H is consistent with the height of the corresponding text line, and S is less than W'; S2, inputting the image block obtained through the sliding window into the frequency domain feature fusion model to obtain the tampering prediction score corresponding to the image block; The step of inputting the image block obtained through the sliding window into the frequency domain feature fusion model to obtain the tampering prediction score corresponding to the image block includes: The image block is transformed into two-dimensional DCT, and the frequency domain coefficient characteristics after DCT transformation are filtered using three adaptive fine-tuning low, medium and high filters to obtain low-frequency, medium-frequency and high-frequency coefficient characteristics, which are set as F L ,F M ,F H ; Then, the three frequency domain coefficient features are subjected to inverse DCT transformation to obtain three filtered images, which are set as f L ,f M ,f H ; The three inverse DCT transformed images and the original image blocks are channel-joined and input into the MobileNetV3small model to further extract features and output the tampering prediction score of the corresponding image block; S3. Divide the text line into multiple sub-regions according to the sliding step size S, and use the average of the tampering prediction scores of the image block where the sub-region is located as the regional tampering score of the sub-region, to form a score list corresponding to the text line, and each regional tampering score on the score list corresponds one-to-one to each sub-region of the text line.
2. The document tampering detection method based on sliding window frequency domain feature fusion according to claim 1 is characterized in that: Before step S1, the method further includes: Input the document image to be detected into the document classifier for classification, obtain the corresponding document type, and select the corresponding key area segmentor based on the document type; The document image to be detected is input into the corresponding key area segmenter to extract the image key area of the document image, and the image key area is used to extract the image block by using the sliding window in step S1.
3. The document tampering detection method based on sliding window frequency domain feature fusion according to claim 1 is characterized in that: The step of performing a two-dimensional DCT transformation on the image block comprises: The DCT transform formula is as follows: F=AfA T Among them, F is the frequency domain coefficient obtained by DCT transformation; f is the pixel value of the image block; A is the transformation matrix; i is the horizontal frequency of the two-dimensional wave, j is the vertical frequency of the two-dimensional wave, and the value range is [0, N-1], and N is the size of the image block.
4. The document tampering detection method based on sliding window frequency domain feature fusion according to claim 3 is characterized in that: The inverse DCT transform comprises: f L =OFF L A T f M =OFF M A T f H =OFF H A T 。 5. A document tampering detection device based on sliding window frequency domain feature fusion, characterized in that: include: An image block extraction module is used to extract image blocks from text lines of a document image using a sliding window, wherein the sliding window has a length of W', a height of H, and a sliding step of S, wherein W' is less than the length W of the corresponding text line, H is consistent with the height of the corresponding text line, and S is less than W'; The image block obtained through the sliding window is input into the frequency domain feature fusion model to obtain the tampering prediction score corresponding to the image block, including: The image block is transformed into two-dimensional DCT, and the frequency domain coefficient characteristics after DCT transformation are filtered using three adaptive fine-tuning low, medium and high filters to obtain low-frequency, medium-frequency and high-frequency coefficient characteristics, which are set as F L ,F M ,F H ; Then, the three frequency domain coefficient features are subjected to inverse DCT transformation to obtain three filtered images, which are set as f L ,f M ,f H ; The three inverse DCT transformed images and the original image blocks are channel-joined and input into the MobileNetV3small model to further extract features and output the tampering prediction score of the corresponding image block; An image block score prediction module is used to input the image block obtained through the sliding window into a frequency domain feature fusion model to obtain a tampering prediction score corresponding to the image block; The regional score calculation module is used to divide the text line into multiple sub-regions according to the sliding step size S, and take the average of the tampering prediction scores of the image block where the sub-region is located as the regional tampering score of the sub-region, forming a score list corresponding to the text line, and each regional tampering score on the score list corresponds one-to-one to each sub-region of the text line.
6. A storage medium having stored thereon a computer program executable by a processor, characterized in that: When the computer program is executed, the steps of the document tampering detection method based on sliding window frequency domain feature fusion as described in any one of claims 1 to 4 are implemented.
7. A computer device comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, wherein: When the computer program is executed, the steps of the document tampering detection method based on sliding window frequency domain feature fusion as described in any one of claims 1 to 4 are implemented.
Citation Information
Cited By
Identity document tampering detection method and system based on geometric gating and field reference distribution
CN123094513A
