Document data cleaning method, system, electronic device, and storage medium

Abstract

The present invention provides a document data cleaning method, system, electronic device, and storage medium. The method comprises: obtaining an image version of a document to be cleaned, splitting the image version of the document into multiple examples; converting the examples into image vectors and comparing them with preset sample image vectors to determine whether the examples need to be removed; removing the examples that need to be removed and concatenating the remaining examples to obtain a cleaned document. The present invention can clean image versions of most documents in various formats, reducing data cleaning costs and improving data production efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of data cleaning, and in particular to a document data cleaning method, system, electronic device, and storage medium. Background Art

[0002] There is a widespread demand for cleaning redundant information (including privacy information, watermark information, etc.) in image versions of documents in many actual production scenarios. In recent years, with the vigorous development of related algorithm technologies for computer vision tasks based on artificial intelligence, using AI technology to assist in various data cleaning is a very valuable process that can save a lot of time and reduce costs for enterprises or individuals.

[0003] When processing and parsing image versions of documents, we often encounter unwanted, redundant information mixed in with the required text, such as headers and footers, watermarks, and QR codes. However, it is difficult to find a convenient and effective way to remove this useless information all at once.

[0004] Currently, image versions of documents are usually cleaned by using programming languages ​​to read the text in the image versions of documents and then cleaning them by specifying specific rules. This type of method is relatively simple to implement and can only be used for documents with a relatively regular format and less information to be removed. Summary of the Invention

[0005] In response to the problems existing in the prior art, the present invention provides a document data cleaning method, system, electronic device, and storage medium.

[0006] The present invention provides a document data cleaning method, the method comprising:

[0007] Obtain an image version of the document to be cleaned, and split the image version of the document into multiple examples;

[0008] Convert the example into an image vector and compare it with the preset sample image vector to determine whether the example needs to be removed;

[0009] Remove the examples that need to be removed, and splice the remaining examples to obtain the cleaned document.

[0010] According to a document data cleaning method provided by the present invention, splitting the image version of a document into multiple examples includes:

[0011] Based on multi-instance learning, the image version of the document is split into multiple multi-instance bags;

[0012] Each multi-instance contains multiple examples without classification labels.

[0013] According to a document data cleaning method provided by the present invention, converting examples into image vectors includes:

[0014] Based on the Transformer network, the examples are first split into sequences, and then the sequences are encoded to form image vectors corresponding to the examples.

[0015] According to a document data cleaning method provided by the present invention, converting examples into image vectors and comparing them with preset sample image vectors includes:

[0016] Convert examples into image vectors;

[0017] The Euclidean distance between the transformed image vector of the example and the preset sample image vector is calculated, and the Euclidean distance is used as the similarity between the two image vectors;

[0018] Determine examples whose similarity is greater than a predetermined threshold as examples that need to be removed;

[0019] Among them, examples and preset samples are converted into image vectors through the same Transformer network.

[0020] According to a document data cleaning method provided by the present invention, before obtaining the cleaned document, or before splitting the image version of the document into multiple examples, the method further includes:

[0021] Perform pigment detection on the image version of the document and remove spots that are inconsistent with the overall appearance of the current image.

[0022] According to a document data cleaning method provided by the present invention, converting the example into an image vector and comparing it with a preset sample image vector includes:

[0023] Identify the types of information that need to be removed from the document to be cleaned;

[0024] Select the type of sample image that corresponds to the type of information to be removed.

[0025] The present invention also provides a document data cleaning system, the system comprising:

[0026] an acquisition module, wherein the acquisition module acquires an image version of a document to be cleaned and splits the image version of the document into multiple examples;

[0027] a judgment module, which converts the example into an image vector and compares it with a preset sample image vector to determine whether the example needs to be removed;

[0028] A reconstruction module removes examples that need to be removed and splices the remaining examples to obtain a cleaned document.

[0029] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned document data cleaning methods when executing the program.

[0030] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned document data cleaning methods.

[0031] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned document data cleaning methods when executed by a processor.

[0032] The document data cleaning method, system, electronic device, and storage medium provided by the present invention can clean most image versions of documents in various forms, greatly saving data cleaning costs and improving data production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 A flowchart of a document data cleaning method provided by the present invention;

[0035] Figure 2 A schematic diagram of the structure of a document data cleaning system provided by the present invention;

[0036] Figure 3 This is a schematic diagram of the physical structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0038] The document data cleaning method provided in the embodiment of the present application is described in detail below with reference to specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0039] Figure 1 A flowchart of a document data cleaning method provided by the present invention is shown as follows: Figure 1 As shown, the present invention provides a document data cleaning method, the method comprising:

[0040] Step 100: Obtain an image version of the document to be cleaned, and split the image version of the document into multiple examples.

[0041] This method can be applied to medical PDF documents. The implementation of this method is based on a trained neural network model. By manually processing 5,000 PDF documents, some non-critical information (such as watermarks, headers, footers, QR codes) and key text content information of the documents (such as titles, abstracts, main text, keywords, etc.) are coordinate-marked and classified into two categories: non-critical information and key information, which are used as training set data for the neural network model.

[0042] In the neural network model, each image is split into multiple image blocks, and each image block is further split into multiple examples. Here, an example refers to a unit that is finer-grained than an image block and constitutes an image block. Preferably, an example refers to an instance that constitutes a multi-instance bag in multi-instance learning. It can also be understood as an example. For example, in the field of medical imaging, CT images are labeled as having symptoms, and a person has a disease because a certain area of ​​the CT image is detected as a lesion. In this case, the multi-instance bag is the CT image, and the example is a small area within the CT image.

[0043] Step 200: Convert the example into an image vector and compare it with a preset sample image vector to determine whether the example needs to be removed.

[0044] The neural network model is pre-set with sample images that need to be removed. Through an image vector conversion module, both the examples and the pre-set samples are converted into image vectors, which facilitates mathematical comparison.

[0045] The preset sample images that need to be removed can be achieved by uploading text or pictures.

[0046] If the similarity between the converted image vector of the example and the preset sample image vector is greater than a preset threshold, it means that the example needs to be deleted.

[0047] Step 300: Remove the examples that need to be removed, and splice the remaining examples to obtain a cleaned document.

[0048] Examples are obtained through image blocks according to predetermined rules. Therefore, there is a connection or mapping relationship between each example based on the predetermined rules. They are encoded through a neural network. The encoding is global. After removing the examples that need to be removed, the reverse operation of the encoding is performed to restore the mapping relationship between the examples, complete the splicing, and obtain the cleaned document.

[0049] This embodiment uses this method to accurately and seamlessly clean watermark information, QR code information, and other custom information in image versions of documents, thereby meeting the needs for cleaning image versions of documents in daily work and life.

[0050] Furthermore, in another embodiment, this embodiment provides a document data cleaning method, which splits the image version of a document into multiple instances, including:

[0051] Based on multi-instance learning, the image version of the document is split into multiple multi-instance bags;

[0052] Each multi-instance contains multiple examples without classification labels.

[0053] Multi-instance learning is used in the neural network model. Each image is split into multiple image blocks, which are combined into a set of multi-instance packages with classification labels. Each multi-package (i.e., multi-instance package) contains multiple examples without classification labels to build a multi-instance learning classifier.

[0054] This embodiment splits the image version of the document into multiple examples through multi-instance learning to achieve smaller-granularity cleaning.

[0055] Furthermore, in another embodiment, this embodiment provides a document data cleaning method, which converts examples into image vectors, including:

[0056] Based on the Transformer network, the examples are first split into sequences, and then the sequences are encoded to form image vectors corresponding to the examples.

[0057] The neural network model uses a Transformer network combined with a multi-instance learning classifier to further split each example in the multi-instance package and form an embedding sequence, similar to tokens in NLP. They are encoded using the Transformer network. The encoding focuses on the local area, and the corresponding mapping forms an image vector of the example transformation.

[0058] This embodiment combines multi-instance learning with the Transformer architecture to build a target detection algorithm, improving the accuracy and efficiency of the data cleaning model.

[0059] Furthermore, in another embodiment, this embodiment provides a document data cleaning method, which converts examples into image vectors and compares them with preset sample image vectors, including:

[0060] Convert examples into image vectors;

[0061] The Euclidean distance between the transformed image vector of the example and the preset sample image vector is calculated, and the Euclidean distance is used as the similarity between the two image vectors;

[0062] Examples whose similarity is greater than a predetermined threshold are determined to be examples that need to be removed.

[0063] Among them, examples and preset samples are converted into image vectors through the same Transformer network.

[0064] Preferably, the preset cleaning information (such as some logos, QR codes, text information, etc.) is first converted into a custom image of 20*20 pixels, and then converted into a custom image vector by the Transformer network. Then, the custom image vector is used to perform the Euclidean distance between the image vector and the image vector corresponding to each example in the multi-example package split from the image version of the document, and the distance between the two vectors (E a , E b ) similarity d(E a ,E b ), the formula is as follows:

[0065] E a =(a1,a2,...,a n ) (1)

[0066] E b =(b1,b2,...,b n ) (2)

[0067]

[0068] By setting a threshold, the coordinates in the multi-example package that are close to the custom image vector are recorded, and the information of the corresponding image page is removed.

[0069] This embodiment applies the calculation of image vector similarity to the field of literature data cleaning.

[0070] Furthermore, in another embodiment, this embodiment provides a document data cleaning method, which, before obtaining the cleaned document, or before splitting the image version of the document into multiple instances, further includes:

[0071] Perform pigment detection on the image version of the document and remove spots that are inconsistent with the overall appearance of the current image.

[0072] Through the OCR module of the model, pigment detection is performed on each image to remove spots that do not match the overall outline of the current image. Through small-grained recognition, the spots in each image block are removed. Finally, the corresponding information is cleared through pigment differences, which can achieve the cleaning of watermarks, QR codes and other information.

[0073] This embodiment uses two channels, text and image, to clean the content that needs to be cleaned, and integrates the OCR model with image similarity and image processing technology based on pigment separation. Ultimately, it achieves the classification and removal of spot data with obvious pigment differences in the image version of the document and redundant information input by the user in the block detection.

[0074] Furthermore, in another embodiment, this embodiment provides a document data cleaning method, which converts examples into image vectors and compares them with preset sample image vectors, including:

[0075] Identify the types of information that need to be removed from the document to be cleaned;

[0076] Select the type of sample image that corresponds to the type of information to be removed.

[0077] It should be noted that the information categories that need to be removed include logos, QR codes, watermarks, headers, footers, etc.

[0078] In this embodiment, by selecting the information category that needs to be cleaned or uploading the text or image that needs to be cleaned, the system uses AI technology to perform dual-channel automatic cleaning or customized cleaning based on this information.

[0079] The document data cleaning system provided by the present invention is described below. The document data cleaning system described below and the document data cleaning method described above can be referenced to each other.

[0080] Figure 2 The present invention also provides a structural diagram of a document data cleaning system, as shown in FIG. Figure 2 As shown, the present invention also provides a document data cleaning system, the system comprising:

[0081] An acquisition module, which acquires an image version of the document to be cleaned and splits the image version of the document into multiple examples;

[0082] The judgment module converts the example into an image vector and compares it with the preset sample image vector to determine whether the example needs to be removed;

[0083] Reconstruction module,The reconstruction module removes the examples that need to be removed, and splices the remaining examples to obtain the cleaned documents.

[0084] By uploading 100 PDF documents of various types for verification, 91 of the results were in line with expectations, and a small number of expected elements were not removed in the remaining 9 documents, which can be improved by fine-tuning the model, and the overall accuracy can reach more than 90%.

[0085] This embodiment applies AI computer vision algorithms to the cleaning of image version document data, which can clean batches of image versions of documents more quickly and accurately.

[0086] Figure 3 A schematic diagram of the physical structure of an electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the document data cleaning method, which includes:

[0087] Obtain an image version of the document to be cleaned, and split the image version of the document into multiple examples;

[0088] Convert the example into an image vector and compare it with the preset sample image vector to determine whether the example needs to be removed;

[0089] Remove the examples that need to be removed, and splice the remaining examples to obtain the cleaned document.

[0090] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0091] On the other hand, the present invention further provides a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer is capable of performing the document data cleaning method provided by the above methods, wherein the method comprises:

[0092] Obtain an image version of the document to be cleaned, and split the image version of the document into multiple examples;

[0093] Convert the example into an image vector and compare it with the preset sample image vector to determine whether the example needs to be removed;

[0094] Remove the examples that need to be removed, and splice the remaining examples to obtain the cleaned document.

[0095] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the document data cleaning method provided above, the method comprising:

[0096] Obtain an image version of the document to be cleaned, and split the image version of the document into multiple examples;

[0097] Convert the example into an image vector and compare it with the preset sample image vector to determine whether the example needs to be removed;

[0098] Remove the examples that need to be removed, and splice the remaining examples to obtain the cleaned document.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A document data cleaning method, characterized in that: This method is applied to medical PDF documents and is based on a trained neural network model. By manually processing the PDF documents, some non-critical information and key text content information of the documents are classified into two categories: non-critical information and key information. This information is used as training data for the neural network model. In the neural network model, each image is split into multiple image blocks, and each image block is further split into multiple examples. An example refers to a unit that is finer-grained than an image block and constitutes an image block. The method includes: Obtain an image version of the document to be cleaned, and split the image version of the document into multiple examples; Convert the example into an image vector and compare it with the preset sample image vector to determine whether the example needs to be removed; Remove the examples that need to be removed, and splice the remaining examples to obtain the cleaned document; Examples are obtained from image blocks according to predetermined rules. Therefore, there is a connection or mapping relationship between each example based on the predetermined rules. The examples are encoded through a neural network. The encoding is global. After removing the examples that need to be removed, the reverse operation of the encoding is performed to restore the mapping relationship between the examples, complete the splicing, and obtain the cleaned document; The image version of the document is split into multiple samples, including: Based on multi-instance learning, the image version of the document is split into multiple multi-instance bags; Among them, each multi-example contains multiple examples without classification labels; The example is converted into an image vector, including: Based on the Transformer network, the examples are first split into sequences, and then the sequences are encoded to form image vectors corresponding to the examples; The neural network model uses a Transformer network combined with a multi-instance learning classifier to further split each example in the multi-instance package and form an embedding sequence, forming tokens similar to those in NLP. The Transformer network is used for encoding, and the encoding focuses on the local area. The corresponding mapping forms an image vector of the example transformation.

2. The document data cleaning method according to claim 1, characterized in that: The converting of the example into an image vector and comparing it with a preset sample image vector includes: Convert examples into image vectors; The Euclidean distance between the transformed image vector of the example and the preset sample image vector is calculated, and the Euclidean distance is used as the similarity between the two image vectors; Determine examples whose similarity is greater than a predetermined threshold as examples that need to be removed; Among them, examples and preset samples are converted into image vectors through the same Transformer network.

3. The document data cleaning method according to any one of claims 1-2, characterized in that: Before obtaining the cleaned document, or before splitting the image version of the document into multiple examples, the method further includes: Perform pigment detection on the image version of the document and remove spots that are inconsistent with the overall appearance of the current image.

4. The document data cleaning method according to claim 3, characterized in that: The example is converted into an image vector and compared with a preset sample image vector, which previously included: Identify the types of information that need to be removed from the document to be cleaned; Select the type of sample image that corresponds to the type of information to be removed.

5. A document data cleaning system, characterized in that: Applied to medical PDF documents, this system is based on a trained neural network model. By manually processing PDF documents, some non-critical information and key text content are classified into two categories: non-critical information and key information. This information is used as training data for the neural network model. In the neural network model, each image is split into multiple image blocks, and each image block is further split into multiple examples. An example refers to a unit that is finer-grained than an image block and constitutes an image block. The system includes: an acquisition module, wherein the acquisition module acquires an image version of a document to be cleaned and splits the image version of the document into multiple examples; a judgment module, which converts the example into an image vector and compares it with a preset sample image vector to determine whether the example needs to be removed; A reconstruction module, which removes the examples that need to be removed and splices the remaining examples to obtain a cleaned document; Examples are obtained from image blocks according to predetermined rules. Therefore, there is a connection or mapping relationship between each example based on the predetermined rules. The examples are encoded through a neural network. The encoding is global. After removing the examples that need to be removed, the reverse operation of the encoding is performed to restore the mapping relationship between the examples, complete the splicing, and obtain the cleaned document; The image version of the document is split into multiple samples, including: Based on multi-instance learning, the image version of the document is split into multiple multi-instance bags; Among them, each multi-example contains multiple examples without classification labels; The example is converted into an image vector, including: Based on the Transformer network, the examples are first split into sequences, and then the sequences are encoded to form image vectors corresponding to the examples; The neural network model uses a Transformer network combined with a multi-instance learning classifier to further split each example in the multi-instance package and form an embedding sequence, forming tokens similar to those in NLP. The Transformer network is used for encoding, and the encoding focuses on the local area. The corresponding mapping forms an image vector of the example transformation.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the document data cleaning method according to any one of claims 1 to 4 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the document data cleaning method according to any one of claims 1 to 4 are implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the document data cleaning method according to any one of claims 1 to 4 are implemented.

Images