Document tampering detection method, device and equipment for network document content security

Abstract

The invention discloses a document tampering detection method, a document tampering detection device and document tampering detection equipment for network document content security. The method comprises the following steps: generating a background amplitude domain component, a background phase domain component, a text amplitude domain component and a text phase domain component according to an initial document image; respectively adding a prompt word to each component, and generating a target document image according to each component after the prompt word is added; and taking the target document image as a training image, training the pre-training model to obtain a document tampering detection model, and performing document tampering detection through the document tampering detection model. According to the technical scheme, the target document image is obtained by processing the components of the background image and the text image of the initial document image in the frequency domain after the prompt words are added, the model is trained through the target document image, the document tampering detection model is obtained, and document tampering detection is carried out. And the generalization performance of the document tampering detection method for network document content security is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and network security, and in particular to a document tampering detection method, device and equipment for network document content security. Background Art

[0002] With the rapid development of information technology, digital documents have been widely used in various application scenarios, especially in the fields of business, law and medicine, where digital documents have become the main form of information exchange and storage. However, compared with paper documents, digital documents are more modifiable. The content of documents can be easily modified through image editing tools, or the key information of document images can be tampered with using deep networks. These operations are often difficult to detect, making it extremely difficult to identify forgeries. Tampering seriously affects the authenticity and integrity of document content.

[0003] Existing document image tampering detection methods are mainly based on text features, document content structured features and texture features, and can achieve effective document image tampering detection under certain conditions. However, when facing document images with different quality and content, stronger robustness and cross-domain generalization capabilities are required. Existing document image tampering detection methods are sensitive to the type and content of document images, and the generalization performance still needs to be improved and cannot meet the needs. Summary of the invention

[0004] The present invention provides a document tampering detection method, device and equipment for network document content security, so as to realize tampering detection of document images.

[0005] In a first aspect, an embodiment of the present invention provides a document tampering detection method for network document content security, the method comprising:

[0006] Generate a background amplitude domain component, a background phase domain component, a text amplitude domain component, and a text phase domain component according to the initial document image;

[0007] respectively adding prompt words to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component, and generating a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after the prompt words are added;

[0008] The target document image is used as a training image to train the pre-trained neural network model to obtain a document tampering detection model, and the document tampering detection is performed using the document tampering detection model.

[0009] In a second aspect, an embodiment of the present invention further provides a document tampering detection device for network document content security, the device comprising:

[0010] A component determination module, used to generate a background amplitude domain component, a background phase domain component, a text amplitude domain component and a text phase domain component according to an initial document image;

[0011] A target document image determination module is used to add prompt words to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component respectively, and generate a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after the prompt words are added;

[0012] The document tampering detection module is used to use the target document image as a training image to train the pre-trained neural network model to obtain a document tampering detection model, and perform document tampering detection through the document tampering detection model.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a document tampering detection method for network document content security as described in any one of the embodiments of the present invention is implemented.

[0014] In a fourth aspect, an embodiment of the present invention further provides a storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute a document tampering detection method for network document content security as described in any one of the embodiments of the present invention.

[0015] The technical solution of the embodiment of the present invention obtains a target document image by adding prompt words to the background amplitude domain component, background phase domain component, text amplitude domain component and text phase domain component of the initial document image and then processing them. The target document image is used as a training image to train a neural network model, and a document tampering detection model is obtained and document tampering detection is performed, thereby realizing tampering detection of the document image and improving the document tampering detection performance.

[0016] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 This is a flow chart of a document tampering detection method for network document content security provided by Embodiment 1 of the present invention;

[0019] Figure 2 This is a flow chart of a document tampering detection method for network document content security provided by Embodiment 2 of the present invention;

[0020] Figure 3 is a schematic diagram of a text image and a background image provided by Embodiment 2 of the present invention;

[0021] Figure 4 It is a structural schematic diagram of a document tampering detection device for network document content security provided by Embodiment 3 of the present invention;

[0022] Figure 5 It is a structural schematic diagram of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In the embodiments of the present application, some existing solutions in the industry such as certain software, components, models, etc. may be mentioned, and they should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or must use the solution.

[0025] The acquisition, transmission, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0026] Embodiment 1

[0027] Figure 1 A flowchart of a document tampering detection method for network document content security is provided for the first embodiment of the present invention. This embodiment is applicable to document image tampering detection. The method can be executed by a document tampering detection device for network document content security. The document tampering detection device for network document content security can be implemented in the form of hardware and / or software. The document tampering detection device for network document content security can be configured in any electronic device with network communication and computing. Figure 1 As shown, the method includes:

[0028] S110, generating a background amplitude domain component, a background phase domain component, a text amplitude domain component, and a text phase domain component according to the initial document image.

[0029] In this embodiment, the initial document image is an image obtained by photographing or scanning the document, the background amplitude domain component is the amplitude component of the background area in the frequency domain, the background phase domain component is the phase component of the background area in the frequency domain, the text amplitude domain component is the amplitude component of the text area in the frequency domain, and the text phase domain component is the phase component of the text area in the frequency domain.

[0030] In this embodiment, the initial document image can be obtained by photographing a paper document or an electronic document with a camera, or by scanning a paper document with a scanner, or can be directly obtained from an existing document image database. This embodiment does not limit the method for obtaining the initial document image.

[0031] Furthermore, by processing the initial document image through an image processing algorithm, the amplitude domain component and phase domain component of the initial document image can be obtained, wherein the amplitude domain component of the initial document image contains more style information, and the phase domain component contains more semantic content. By obtaining the background amplitude domain component, the background phase domain component, the text amplitude domain component, and the text phase domain component, it is helpful to improve the accuracy of subsequent document tampering detection.

[0032] S120, respectively adding prompt words to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component, and generating a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after adding the prompt words.

[0033] In this embodiment, the prompt words represent a set of adjustable learnable parameters, and the target document image represents the document image after the prompt words are added.

[0034] It should be noted that compared with manually designed fixed prompt words, learnable prompt words are more flexible and can better capture the requirements and characteristics of the task.

[0035] In practical applications, learnable prompt words are added to the background amplitude domain component, background phase domain component, text amplitude domain component, and text phase domain component. Learnable prompt words can learn information from different areas of document images. By training document images with learnable prompt words added to update and optimize the model, the model can more effectively capture the distortion features of tampered text, thereby improving the generalization performance of existing tampering detection methods.

[0036] In addition, the addition of learnable prompt words is a prompt word or input template that is automatically adjusted in an optimized manner during the training process of the neural network model. By introducing a small number of learnable prompt words at the input end, the document tampering detection method for network document content security can be adapted to various network architectures and achieve plug-and-play.

[0037] Furthermore, the target document image is obtained through image processing according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after the prompt word is added.

[0038] Optionally, prompt words are added to the background amplitude domain component, the background phase domain component, the text amplitude domain component, and the text phase domain component, respectively, including:

[0039] Prompt words are added to the central areas of the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component respectively.

[0040] In this embodiment, the amplitude domain reflects the strength information of the signal, and the central area of ​​the amplitude domain component can be an area where the signal amplitude is highly focused, an area with significant amplitude characteristics, or a core range of concentrated amplitude distribution. The phase domain contains the phase information of the signal, and the central area of ​​the phase domain component can be an area of ​​stable phase, an area with dominant phase characteristics, or a position where the phase is prominently displayed.

[0041] Specifically, the central regions of the amplitude domain components and the phase domain components can be determined by analyzing the signal and finding the key positions of the amplitude and phase. For example, in multi-resolution image analysis, the central regions of different frequencies can be determined by performing spectral decomposition on the full phase system function.

[0042] Furthermore, learnable prompt words are added to the central areas of the determined background amplitude domain component, background phase domain component, text amplitude domain component, and text phase domain component. The prompt words can be specific numerical values, vectors, or functions, and their values ​​are adjusted through training to adapt to different task requirements. For example, learnable prompt words can be used as a way to directly replace text embedding to activate task-related features.

[0043] It should be noted that the central areas of the amplitude domain component and the phase domain component contain the most information and features. Therefore, adding prompt words to the central areas of the background amplitude domain component, background phase domain component, text amplitude domain component and text phase domain component, respectively, is beneficial to improving the document tampering detection model to capture the distortion features of tampered text, and improving the accuracy and generalization performance of the document tampering detection method used for network document content security.

[0044] S130: Use the target document image as a training image to train the pre-trained neural network model to obtain a document tampering detection model, and perform document tampering detection using the document tampering detection model.

[0045] In this embodiment, the document tampering detection model refers to a model used for document tampering detection. The document tampering detection model is obtained by training a neural network model using a target document image as a training image.

[0046] Among them, commonly used neural network models include convolutional neural network models, residual network models, and Transformer network models.

[0047] It should be noted that in this embodiment, the target document image is obtained by processing the initial document image, which is used to train the neural network model to obtain a document tampering detection model, and then the document tampering detection model is used to perform document tampering detection on the document image, which is beneficial to improving the accuracy of the document tampering detection model in detecting tampering of document images and the generalization performance of the document tampering detection method for network document content security.

[0048] Optionally, performing document tampering detection by using the document tampering detection model includes:

[0049] The document image to be identified is input into the document tampering detection model to obtain the document tampering detection result output by the document tampering detection model.

[0050] In this embodiment, the document image to be identified represents an image to be identified that contains text content, and the document tampering detection result is whether the document image to be identified is a document tampering image.

[0051] In actual applications, the document tampering detection results can be returned to the terminal for visual display. Specifically, the document tampering detection results can be displayed in the terminal in the form of plain text, directly listing the key information of the detection, such as whether it has been tampered with, the tampering location or the tampering type (such as text replacement, deletion, addition, etc.), so that the document tampering detection results are clear. The original image of the document or the preview of the electronic document can also be displayed on the terminal, and the tampered area can be highlighted with marks of different colors and shapes. There is no specific limitation on the visual display of the document tampering detection results in this embodiment.

[0052] The technical solution of the embodiment of the present invention obtains a target document image by adding prompt words to the background amplitude domain component, background phase domain component, text amplitude domain component and text phase domain component of the initial document image and then processing them. The target document image is used as a training image to train a pre-trained neural network model to obtain a document tampering detection model and perform document tampering detection, thereby realizing tampering detection of document images and improving the generalization performance of the document tampering detection method for network document content security.

[0053] Embodiment 2

[0054] Figure 2 A flowchart of a document tampering detection method for network document content security provided in the second embodiment of the present invention, this embodiment can be applied to document image tampering detection, the method can be executed by a document tampering detection device for network document content security, the document tampering detection device for network document content security can be implemented in the form of hardware and / or software, and the document tampering detection device for network document content security can be configured in any electronic device with network communication and computing. The embodiment of the present invention is further specified on the basis of the above embodiment.

[0055] like Figure 2 As shown, the method includes:

[0056] S210: Perform character recognition on the initial document image to obtain a text area mask.

[0057] In this embodiment, the text area mask represents a binary image, which is composed of a white area and a black area.

[0058] It should be noted that the text area mask can be obtained by marking the text area in the document image and covering the background area through a specific character recognition method, forming a "mask" only for the text area. The mask can be binary or have different transparency or color values ​​to distinguish between text and background.

[0059] Among them, commonly used character recognition methods include character recognition methods based on template matching, character recognition methods based on feature extraction, and character recognition methods based on deep learning. In this embodiment, the character recognition method used for character recognition of the initial document image is not specifically limited.

[0060] Specifically, the existing deep learning-based optical character recognition method can be used to detect the text area and background area (non-text area) in the initial document image, set the pixel value of the text area to all 1 to display white, and set the pixel value of the background area to all 0 to display black, so as to obtain the text area mask of the corresponding binary image.

[0061] S220 : Divide the initial document image into a text area and a background area according to the text area mask.

[0062] It should be noted that the white area in the text area mask of the binary image represents the text area, and the black area represents the background area. This is because the pixel values ​​of the text area in the text area mask are all 1 and displayed in white, and the pixel values ​​of the background area are all 0 and displayed in black.

[0063] Therefore, according to the black area and the white area in the text area mask, the text area and the background area of ​​the original document image can be determined.

[0064] S230: Generate a text image according to the text area, and generate a background image according to the background area.

[0065] In this embodiment, according to the text area mask, the text area in the initial document is retained, and the pixel value of the background area is set to 0 to display black, thereby obtaining a text image. At the same time, according to the text area mask, the pixel value of the text area is set to 0 to display black, and the background area in the initial document is retained to obtain a background image.

[0066] It should be noted that, considering that the tampering of the document image occurs in the text area, the text area mask generated by the optical character recognition algorithm distinguishes different areas in the initial document image. The text image is generated according to the text area, and the background image is generated according to the background area, so as to better distinguish the text and the background, so that the document tampering detection model pays more attention to the text, which is conducive to improving the accuracy of the document tampering detection model in detecting document image tampering.

[0067] S240, converting the background image into the background frequency domain, and splitting the background frequency domain into a background amplitude domain component and a background phase domain component, and converting the text image into the text frequency domain, and splitting the text frequency domain into a text amplitude domain component and a text phase domain component.

[0068] It should be noted that the background frequency domain and the text frequency domain are complex matrices, wherein the background frequency domain contains the amplitude and phase information of the signal in the background image, and the text frequency domain contains the amplitude and phase information of the signal in the text image.

[0069] In this embodiment, converting the background image to the background frequency domain means converting the signal in the background image from the spatial domain to the frequency domain, thereby revealing the distribution of different frequency components in the background image. Similarly, converting the text image to the text frequency domain means converting the signal in the text image from the spatial domain to the frequency domain, thereby revealing the distribution of different frequency components in the text image.

[0070] Furthermore, the background frequency domain may be split into a background amplitude domain component and a background phase domain component, and the text frequency domain may be split into a text amplitude domain component and a text phase domain component.

[0071] Optionally, converting the background image into a background frequency domain, and splitting the background frequency domain into a background amplitude domain component and a background phase domain component, includes steps A1-A2:

[0072] Step A1: Perform discrete Fourier transform on the background image to obtain the background frequency domain.

[0073] Step A2: perform amplitude calculation on the background frequency domain to obtain a background amplitude domain component, and perform phase calculation on the background frequency domain to obtain a background phase domain component.

[0074] It should be noted that commonly used frequency conversion methods include discrete Fourier transform and wavelet transform, etc. Among them, commonly used amplitude calculation methods include vector amplitude calculation method and amplitude spectrum calculation method based on frequency domain signal, etc., and phase calculation methods include phase calculation based on phase difference and phase calculation based on trigonometric function, etc.

[0075] In this embodiment, by performing discrete Fourier transform on the background image, the background frequency domain is obtained, which is expressed by the following formula:

[0076]

[0077] in, is the background frequency domain, I b For the background image, H is the image height, W is the image width, and C is the number of RGB channels of the image.

[0078] Furthermore, the amplitude spectrum of the background frequency domain can be determined by using an amplitude spectrum calculation method based on frequency domain signals to determine the background amplitude component. The background phase domain component can be obtained by calculating the phase of the complex number in the background frequency domain.

[0079] The text image is converted into the text frequency domain, and the text frequency domain is split into a text amplitude domain component and a text phase domain component, including steps C1-C2:

[0080] Step C1: Perform discrete Fourier transform on the text image to obtain the text frequency domain.

[0081] Step C2: perform amplitude calculation on the text frequency domain to obtain the text amplitude domain component, and perform phase calculation on the text frequency domain to obtain the text phase domain component.

[0082] In this embodiment, a discrete Fourier transform is performed on the text image to obtain a text frequency domain, which is expressed by the following formula:

[0083]

[0084] in, is the text frequency domain, I t For text images, H is the image height, W is the image width, and C is the number of RGB channels of the image.

[0085] Furthermore, the amplitude spectrum of the text frequency domain can be determined by using an amplitude spectrum calculation method based on frequency domain signals to determine the background amplitude component. The phase domain component of the text can be obtained by calculating the phase of the complex number in the text frequency domain.

[0086] It should be noted that the text amplitude domain component can reflect the intensity information of the text in the frequency domain. By analyzing the amplitude domain, features such as the thickness change of the text strokes can be obtained. The text amplitude domain component can be used to distinguish texts of different fonts. The stroke structures and distributions of different fonts are different, and they will show different amplitude distribution patterns in the frequency domain. For example, Song font and bold font, Song font has thin strokes and more decorative strokes, and may have richer amplitude information in the high-frequency part; bold font has uniform strokes, and the amplitude of the low-frequency part may be relatively more stable.

[0087] In addition, the text phase domain component contains the position information of the text strokes and structure. For text deformation (such as tilt and rotation), the text phase domain component can provide more stable features. Since the phase information is related to the geometric structure of the text to a certain extent, when the text is deformed, the phase domain may better reflect the original structural characteristics of the text than the amplitude domain. The text phase domain component helps to improve the robustness of recognizing deformed text.

[0088] Since the document tampering area is more concentrated in the text area, the background amplitude domain component, background phase domain component, text amplitude domain component and text phase domain component are obtained respectively, which is conducive to better distinguishing the amplitude domain features and phase domain features of the text area and the background area, so that the document tampering model pays more attention to the text. When faced with document images with different quality and content, the document tampering detection method used for network document content security has stronger robustness and cross-domain generalization ability.

[0089] S250, respectively adding prompt words to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component, and generating a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after adding the prompt words.

[0090] The specific process of adding prompt words has been described in the above embodiment, and will not be repeated in this embodiment.

[0091] Optionally, generating a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after adding the prompt word includes steps B1-B2:

[0092] Step B1, perform inverse Fourier transform on the background amplitude domain component and the background phase domain component after adding the prompt word to obtain a background prompt image, and perform inverse Fourier transform on the text amplitude domain component and the text phase domain component after adding the prompt word to obtain a text prompt image.

[0093] Step B2: Fusing the background prompt image and the text prompt image to obtain a target prompt image.

[0094] In this embodiment, the background prompt image represents an image including the background area obtained by inverse Fourier transforming the background amplitude domain component and the background phase domain component after adding the prompt word. The text prompt image represents an image including the text area obtained by inverse Fourier transforming the text amplitude domain component and the text phase domain component after adding the prompt word. The target prompt image represents an image including the background area and the text area after the background prompt image and the text prompt image are fused.

[0095] Among them, the inverse Fourier transform is the inverse process of the Fourier transform. In image processing, the inverse Fourier transform can be used for image reconstruction. By performing Fourier transform on the image, the frequency domain representation of the image is obtained, and then the frequency domain representation is processed, and then the processed frequency domain signal is converted back to the spatial domain representation of the image through the inverse Fourier transform to achieve image reconstruction.

[0096] In this embodiment, a background prompt image and a text prompt image are obtained by inverse Fourier transform, and the background prompt image and the text prompt image are fused to obtain a complete target prompt image.

[0097] Specifically, the background amplitude domain component and the background phase domain component after adding the prompt word are subjected to inverse Fourier transform to obtain a background prompt image, which is expressed by the following formula:

[0098]

[0099] in, Represents the background hint image, represents the inverse Fourier transform, represents the background amplitude domain component, represents the background phase domain component, and Indicates the prompt words added to the background amplitude domain component and the prompt words added to the background phase domain component.

[0100] Specifically, the text amplitude domain component and the text phase domain component after adding the prompt word are subjected to inverse Fourier transform to obtain a text prompt image, which is expressed by the following formula:

[0101]

[0102] in, Represents a text hint image, represents the inverse Fourier transform, represents the amplitude domain component of the text, represents the phase domain component of the text, and Indicates the prompt words added to the amplitude domain component of the text and the prompt words added to the phase domain component of the text.

[0103] Specifically, the background prompt image and the text prompt image are fused to obtain the target prompt image, which is expressed by the following formula:

[0104]

[0105] Among them, I P represents the target prompt image, Represents the background hint image, Represents a text hint image.

[0106] S260: Use the target document image as a training image to train the pre-trained neural network model to obtain a document tampering detection model, and perform document tampering detection using the document tampering detection model.

[0107] The technical solution of the embodiment of the present invention generates a text area mask by performing character recognition on an initial document image, determines a text image and a background image according to the document area mask, and respectively adds prompt words to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component for processing to obtain a target document image, uses the target document image as a training image, trains a pre-trained neural network model, obtains a document tampering detection model and performs document tampering detection, thereby realizing tampering detection of the document image and improving the generalization performance of the document tampering detection method for network document content security.

[0108] In order to verify the effectiveness of the document tampering detection method for network document content security of the present invention, in this embodiment, data set A and data set B are used as test sets, and the document tampering detection method for network document content security is extended to image detection and semantic segmentation models for cross-database experiments, including the existing a model, b model and c model based on deep learning architecture, and the existing d model based on convolutional neural network. Among them, both data sets contain high-resolution certificate documents with rich document content and use multiple operations to simulate the manual tampering process to cover more practical application scenarios.

[0109] The experimental results all use intersection over union (IoU), precision, recall and F1-score as evaluation indicators.

[0110] Table 1 shows the results of the cross-database experiment. If the model is not fine-tuned (model a, model b, model c, model d), the cross-database performance is poor.

[0111] If only the decoder in each model network architecture is fine-tuned (a(F) model, b(F) model, c(F) model, d(F) model), the performance is improved, but the gain is not obvious. Among them, the fine-tuning is performed using dataset C, which contains document images of certificates and documents with rich background colors and samples with large tampered areas. During the fine-tuning process, the size of the prompt word is 64×64 pixels, the input size of the model is 512×512 pixels, and the optimizer is used with a learning rate of 3e -5 , the batch size is 12, and the cross entropy loss and loss function are used for training for 20 epochs.

[0112] However, by expanding the document tampering method of the present application to various models (a+MFVP(F) model, b+MFVP(F) model, c+MFVP(F) model, d+MFVP(F) model), the performance gain is the greatest in most cases, proving the effectiveness and scalability of the document tampering detection method of the present application for network document content security.

[0113] Table 1 Cross-database experimental results (the best performance under each backbone network is marked in bold)

[0114]

[0115]

[0116] Among them, -F means using the dataset C to fine-tune the decoder of the network; -MFVP(F) means adding learnable prompt words in the frequency domain space of different areas of the initial document image, that is, expanding the document tampering detection method for network document content security of this application on the original model.

[0117] In addition, the document tampering detection method for network document content security is expanded on each original model to visualize the prompt words learned in the amplitude domain of the text image and the background image.

[0118] See also Figure 3 The figure shows a schematic diagram of a text image and a background image. The information learned from the text image and the background image is different, which proves that the prompt word can capture the difference between the two and more effectively capture the distortion traces introduced by the tampering operation.

[0119] Furthermore, in order to prove the effectiveness of introducing text region mask to distinguish the text region and background region in the document image, and adding prompt words in the frequency domain of the text image and the background image respectively.

[0120] Model a with good cross-database performance is selected for ablation experiments. Table 2 shows the results of the ablation experiment, where FVP means Fourier transforming the input image and adding learnable prompt words in its frequency domain without considering the division of regions. Since the tampering of document images mainly occurs in the text area, the gain brought by the division of regions is more significant, which proves that it is crucial to introduce text region masks to distinguish the text area and background area in the document image.

[0121] Table 2 Ablation experiment results (the best performance is marked in bold)

[0122]

[0123] Embodiment 3

[0124] Figure 4 This is a schematic diagram of the structure of a document tampering detection device for network document content security provided by the third embodiment of the present invention. This embodiment can be applied to the situation of document image tampering detection. The document tampering detection device for network document content security can be implemented in the form of hardware and / or software. The document tampering detection device for network document content security can be configured in any electronic device with network communication and computing. Figure 4 As shown, the device comprises:

[0125] The component determination module 310 is used to generate a background amplitude domain component, a background phase domain component, a text amplitude domain component and a text phase domain component according to the initial document image;

[0126] The target document image determination module 320 is used to add prompt words to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component respectively, and generate a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after the prompt words are added;

[0127] The document tampering detection module 330 is used to use the target document image as a training image to train the pre-trained neural network model to obtain a document tampering detection model, and perform document tampering detection through the document tampering detection model.

[0128] Optionally, the component determination module 310 includes:

[0129] generating a background image and a text image according to an initial document image;

[0130] The background image is converted into the background frequency domain, and the background frequency domain is split into a background amplitude domain component and a background phase domain component; and the text image is converted into the text frequency domain, and the text frequency domain is split into a text amplitude domain component and a text phase domain component.

[0131] Optionally, the component determination module 310 includes:

[0132] Perform character recognition on the initial document image to obtain a text area mask;

[0133] According to the text region mask, the initial document image is divided into a text region and a background region;

[0134] A text image is generated according to the text region, and a background image is generated according to the background region.

[0135] Optionally, the component determination module 310 includes:

[0136] Perform discrete Fourier transform on the background image to obtain the background frequency domain;

[0137] Performing amplitude calculation on the background frequency domain to obtain a background amplitude domain component, and performing phase calculation on the background frequency domain to obtain a background phase domain component;

[0138] Convert the text image to the text frequency domain, and split the text frequency domain into the text amplitude domain component and the text phase domain component, including:

[0139] Perform discrete Fourier transform on the text image to obtain the text frequency domain;

[0140] The amplitude of the text is calculated in the frequency domain to obtain the amplitude domain component of the text, and the phase of the text is calculated in the frequency domain to obtain the phase domain component of the text.

[0141] Optionally, the target document image determination module 320 includes:

[0142] Prompt words are added to the central areas of the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component respectively.

[0143] Optionally, the target document image determination module 320 includes:

[0144] Performing inverse Fourier transformation on the background amplitude domain component and the background phase domain component after the prompt word is added to obtain a background prompt image, and performing inverse Fourier transformation on the text amplitude domain component and the text phase domain component after the prompt word is added to obtain a text prompt image;

[0145] The background prompt image and the text prompt image are fused to obtain the target prompt image.

[0146] Optionally, the document tampering detection module 330 includes:

[0147] Inputting the document image to be identified into the document tampering detection model to obtain the document tampering detection result output by the document tampering detection model;

[0148] The document tampering detection result is used to indicate whether the document image to be identified is a document tampering image.

[0149] The technical solution of the embodiment of the present invention obtains a target document image by adding prompt words to the background amplitude domain component, background phase domain component, text amplitude domain component and text phase domain component of the initial document image and then processing them. The target document image is used as a training image to train a pre-trained neural network model to obtain a document tampering detection model and perform document tampering detection, thereby realizing tampering detection of document images and improving the generalization performance of the document tampering detection method for network document content security.

[0150] The document tampering detection device for network document content security provided by the embodiment of the present invention can execute the document tampering detection method for network document content security provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0151] Embodiment 4

[0152] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0153] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0154] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0155] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 performs the various methods and processes described above, such as a document tampering detection method for network document content security.

[0156] In some embodiments, the document tampering detection method for network document content security may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the document tampering detection method for network document content security described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the document tampering detection method for network document content security in any other appropriate manner (e.g., by means of firmware).

[0157] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0159] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0160] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0161] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0162] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0163] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0164] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A document tampering detection method for network document content security, characterized in that: include: Generate a background amplitude domain component, a background phase domain component, a text amplitude domain component, and a text phase domain component according to the initial document image; respectively adding prompt words to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component, and generating a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after the prompt words are added; The target document image is used as a training image to train the pre-trained neural network model to obtain a document tampering detection model, and the document tampering detection is performed using the document tampering detection model.

2. The method according to claim 1, characterized in that According to the initial document image, a background amplitude domain component, a background phase domain component, a text amplitude domain component and a text phase domain component are generated, including: generating a background image and a text image according to an initial document image; The background image is converted into the background frequency domain, and the background frequency domain is split into a background amplitude domain component and a background phase domain component; and the text image is converted into the text frequency domain, and the text frequency domain is split into a text amplitude domain component and a text phase domain component.

3. The method according to claim 2, characterized in that Generate a background image and a text image based on an initial document image, including: Perform character recognition on the initial document image to obtain a text area mask; According to the text region mask, the initial document image is divided into a text region and a background region; A text image is generated according to the text region, and a background image is generated according to the background region.

4. The method according to claim 2, characterized in that: The background image is converted to the background frequency domain, and the background frequency domain is split into the background amplitude domain component and the background phase domain component, including: Perform discrete Fourier transform on the background image to obtain the background frequency domain; Performing amplitude calculation on the background frequency domain to obtain a background amplitude domain component, and performing phase calculation on the background frequency domain to obtain a background phase domain component; Convert the text image to the text frequency domain, and split the text frequency domain into the text amplitude domain component and the text phase domain component, including: Perform discrete Fourier transform on the text image to obtain the text frequency domain; The amplitude of the text is calculated in the frequency domain to obtain the amplitude domain component of the text, and the phase of the text is calculated in the frequency domain to obtain the phase domain component of the text.

5. The method according to claim 1, characterized in that Prompt words are added to the background amplitude domain component, background phase domain component, text amplitude domain component and text phase domain component respectively, including: Prompt words are added to the central areas of the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component respectively.

6. The method according to claim 1, characterized in that Generate a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after adding the prompt word, including: Performing inverse Fourier transformation on the background amplitude domain component and the background phase domain component after the prompt word is added to obtain a background prompt image, and performing inverse Fourier transformation on the text amplitude domain component and the text phase domain component after the prompt word is added to obtain a text prompt image; The background prompt image and the text prompt image are fused to obtain the target prompt image.

7. The method according to claim 1, characterized in that Document tampering detection is performed using the document tampering detection model, including: Inputting the document image to be identified into the document tampering detection model to obtain the document tampering detection result output by the document tampering detection model; The document tampering detection result is used to indicate whether the document image to be identified is a document tampering image.

8. A document tampering detection device for network document content security, characterized in that: include: A component determination module, used to generate a background amplitude domain component, a background phase domain component, a text amplitude domain component and a text phase domain component according to an initial document image; A target document image determination module is used to add prompt words to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component respectively, and generate a target document image according to the background amplitude domain component, the background phase domain component, the text amplitude domain component and the text phase domain component after the prompt words are added; The document tampering detection module is used to use the target document image as a training image to train the pre-trained neural network model to obtain a document tampering detection model, and perform document tampering detection through the document tampering detection model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the document tampering detection method for network document content security as described in any one of claims 1-7 is implemented.

10. A storage medium storing computer executable instructions, characterized in that: The computer executable instructions are used to execute the document tampering detection method for network document content security as described in any one of claims 1-7 when executed by a computer processor.

Images