Sensitive data detection and replacement

By identifying and replacing sensitive data in preprocessing, initial processing and processing stages in the document processing system, the problem of sensitive information protection in paper documents is solved, and the privacy protection and automated data processing of machine learning systems are realized.

CN113228019BActive Publication Date: 2024-11-26SERVICENOW INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980082500.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-10-26
Filing Date
2019-10-25
Publication Date
2024-11-26
Estimated Expiration
2039-10-25

AI Technical Summary

Technical Problem

The prior art is difficult to effectively protect and process sensitive information in paper documents, and privacy issues are difficult to solve when used in training data sets for machine learning systems.

Method used

A system and method are designed to receive document images through preprocessing, initial processing and processing stages, apply image preprocessing, locate and replace sensitive data in documents, generate appropriate replacement data and insert them into images, and remove sensitive information. . The system includes a preprocessing module, a clustering module, a data type determination module and a data replacement module. It uses the database to generate replacement data to ensure that the replacement data is seamlessly integrated with the original data.

Benefits of technology

It realizes effective identification and replacement of sensitive data in the document, ensures that the machine learning system does not expose user privacy during the training process, reduces manual intervention, and improves the degree of automation of data processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113228019B_ABST
    Figure CN113228019B_ABST
Patent Text Reader

Abstract

Systems and methods for privacy and sensitive data protection. An image of a document is received in a pre-processing stage and image pre-processing is applied to the image to ensure that the resulting image is adequate for further processing. The pre-processing may include processing related to image quality and image orientation. The image is then passed to an initial processing stage. In the initial processing stage, relevant data in the document is located and a bounding box is placed around the data. The resulting image is then passed to a processing stage. In this stage, the type of data within the bounding box is determined and appropriate replacement data is generated. The replacement data is then inserted into the image, thereby removing and replacing sensitive data in the image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to document processing. More particularly, the present invention relates to systems and methods for replacing sensitive information in documents. Background Art

[0002] The early 21st century was marked by the rise and growing popularity of machine learning across multiple industries. However, in order for machine learning to achieve higher levels of penetration in the business world, machine learning must be able to handle the thousands of various tasks that make a business run. In order to be able to do this, a machine learning system must learn, perfect, and ultimately perform these tasks. And, in order to be able to do this properly, a machine learning system will need access to large amounts of data. This large amount of data can then be used to properly train such a machine learning system to implement various business processes.

[0003] It should be clear that such a large amount of data is available. Even with the increasing use of automation in business, the business world still revolves around paper and its representations. For example, business is still conducted with letters (on paper), with invoices (printed on paper), and with receipts (printed on paper). Although the paperless society has been touted, the fact remains that paper is still widely used, and even if attempts are being made to phase it out, the form of data on paper is unlikely to change any time soon. As an example of this, even paperless invoices and receipts still look the same as if they were printed on paper.

[0004] Such paper records, invoices, and documents can be used to train machine learning systems to perform such business processes. However, this path is fraught with problems stemming from another major concern of the information technology revolution: privacy. Real-world documents (such as invoices, credit card receipts, medical reports, etc.) all have information that people do not want others to have access to. For this reason, there is a need for systems and methods that can protect the privacy of data and users while still allowing real-world documents to be used as training data sets for machine learning systems. Preferably, once these systems and methods are properly initiated, they will require little or no human intervention. Summary of the invention

[0005] The present invention relates to systems and methods for privacy and sensitive data protection. An image of a document is received in a pre-processing stage and image pre-processing is applied to the image to ensure that the resulting image is sufficient for further processing. The pre-processing may involve processing related to image quality and image orientation. The image is then passed to an initial processing stage. In the initial processing stage, relevant data in the document is located and a bounding box is placed around the data. The resulting image is then passed to a processing stage. In this stage, the type of data within the bounding box is determined and appropriate replacement data is generated. The replacement data is then inserted into the image, thereby removing and replacing sensitive data in the image.

[0006] In a first aspect, the present invention provides a system for replacing sensitive data in a document, the system comprising:

[0007] - a pre-processing module, configured to receive an image of at least one document;

[0008] - a clustering module for subdividing said image of each of said at least one document into different fields, each field containing data;

[0009] - a data type determination module, configured to identify specific data types in the different fields, thereby determining whether the data in the different fields includes sensitive data; and

[0010] - A data replacement module, configured to replace sensitive data of at least one specific data type in the different fields with replacement data.

[0011] In a second aspect, the invention provides a method for conditioning an image of a document before the image is used as a data point in a data set, the method comprising:

[0012] a) receiving the image;

[0013] b) determining the location of sensitive data on the document in the image;

[0014] c) determining the type of the sensitive data determined in step b);

[0015] d) determining replacement data based on the type of the sensitive data determined in step c);

[0016] e) inserting the replacement data into the image at the location, thereby replacing the sensitive data with the replacement data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Embodiments of the present invention will now be described with reference to the following drawings, in which like reference numerals in different figures refer to like elements, and in which:

[0018] Figure 1 is a block diagram of a system according to one aspect of the present invention;

[0019] Figure 2 is a block diagram of a configuration of a system according to another aspect of the present invention;

[0020] Figure 3 yes Figure 2 A variation of the system in ; and

[0021] Figure 4 It is detailed description Figure 2 and Figure 3 A block diagram of a sub-module within one of the components in a system. DETAILED DESCRIPTION

[0022] refer to Figure 1 , a block diagram according to one aspect of the present invention is shown. This aspect of the present invention includes a system 10 having a pre-processing stage 20, an initial processing stage 30 and a processing stage 40.

[0023] The pre-processing stage 20 receives an image containing at least one document. The pre-processing stage 20 determines whether the image has sufficient image quality for further processing. To this end, the pre-processing stage may analyze the image's contrast, color, image size, orientation, and sharpness, as well as the presence or absence of any artifacts that may affect the clarity of the image. If these image quality parameters are insufficient for further processing (e.g., color balance and / or contrast make it impossible for OCR to determine the characters), the pre-processing stage will adjust these parameters as needed. In addition to adjusting the image's contrast, color, geometry, and sharpness, the pre-processing stage may also crop the image and / or remove artifacts that may affect the clarity of the image.

[0024] In addition to the above, the pre-processing stage can also adjust the orientation of the image to ensure that the data therein is easier to process. Specifically, the pre-processing stage can rotate the image to ensure that the image is square on the screen. As needed, this rotation can be a full 180-degree rotation (for example, if the document is upside down in the image) or a partial rotation (for example, if the document is at an angle in the image). Similarly, the image can be geometrically translated (i.e., adjusted or moved to the left / right as needed) to again ensure that the image is square on the screen. This ensures that the resulting image is easier to process.

[0025] It should be clear that the adjustment of various parameters of the image can be performed by various suitable image adjustment modules according to predetermined limits and / or criteria. Such modules are well known to those skilled in the art of image processing and / or image quality adjustment.

[0026] It should also be clear that, depending on the implementation, the pre-processing stage may also process the image to ensure that the image only shows / references a single document. If the image is a collection of images of multiple documents, the pre-processing stage may separate the original image into multiple images, each of which only shows a single document.

[0027] After the pre-processing stage, the conditioned image is then sent to the initial processing stage. The initial processing stage analyzes the conditioned image of the document to locate relevant data within the image and place an appropriate bounding box around the data. In addition, the initial processing stage determines the type of data within the bounding box.

[0028] The initial processing phase determines the data clusters in the document. The data extraction phase determines the data clusters within the document, determines the type of data within each of the data clusters, and places bounding boxes around the various data clusters (where necessary). The position (within the document) of each data cluster is also determined, and where necessary, adjacent data clusters are concatenated to form larger clusters. This may be necessary if the data in one cluster is part of a data set (e.g., one cluster has the first part of an address, and its adjacent cluster has the second part of the address - the two clusters are concatenated to form a single data cluster containing the address).

[0029] For clarity, a clustering algorithm or process is applied to the document so that the pixels are segmented into logically consistent chunks of text / concept data. For example, pixels containing a patient's name are grouped together and clustered separately from the cluster of pixels containing that patient's address (in the line below the patient's name).

[0030] It should be clear that the initial processing stage determines the type of data within each data cluster in the document. This includes determining whether the data in the cluster is an image, text (i.e., alphanumeric characters), a table, a signature, or any other data type that may be encountered in a business document. In addition to determining this, the initial processing stage also determines the parameters of the cluster and the parameters of the data within the cluster. Thus, the initial processing stage can determine the size of the cluster in pixels. This may mean determining the length, width, and height of the cluster in pixels or any other suitable unit of measurement. For data clusters containing text data, the initial processing stage can determine the font type of the text data as well as the font size. For image data, the size of the image, the lines within the image, the shapes in the image, and the presence or absence of text in the image can also be determined. For table data (i.e., the cluster data contains a table), the initial processing stage can determine the parameters of the data within the table. In addition to the above, the initial processing stage also identifies the font, size, style, spacing, etc. of the data (especially text data).

[0031] It should be clear that the initial processing stage not only determines the clusters, but also places bounding boxes where necessary to define the various clusters. Of course, the term "bounding box" also includes different shapes, sizes, and configurations for devices and methods that define, depict, or otherwise indicate data clusters and are machine-readable on the image. Such bounding boxes also include boxes or shapes that are irregular in shape, size, or orientation (e.g., at a certain angle).

[0032] It should also be clear that the initial processing stage may also determine the type of content in the cluster data containing the image. The initial processing stage may also be configured to determine whether the image data indicates that the cluster contains a logo, a signature, or some kind of picture. As described above, any text in the image may be determined and, if necessary, extracted and / or marked for replacement / removal.

[0033] According to the above, the initial processing stage needs to perform OCR functions on various clusters to determine the text in the clusters. For this function, a suitable OCR module can be used. Similarly, other modules that are particularly suitable for the above tasks can also be used. Therefore, dedicated modules for cluster determination, cluster cascading, image recognition, text parameter determination, image parameter determination and other functions can be used. Of course, depending on the configuration of the system, multiple functions can be performed by one or more modules. Those skilled in the art of OCR, data analysis, text and image analysis and determination can determine which modules and which functions are necessary for implementing the various stages and modules of the present invention.

[0034] After the initial processing stage, the images with associated bounding boxes are then passed to the processing stage. For data types that are considered sensitive data (such as names and addresses), the processing stage generates similar data of the same type and replaces the sensitive data with appropriate replacement data.

[0035] The processing stage first determines which data in the bounding box contains sensitive data. Since the initial processing stage has determined which data types are in which bounding boxes, the processing stage only finds out the relevant data types. Therefore, in one embodiment, name data (such as proprietary names and company names) and address data (such as data containing at least numbers and streets) are determined. Then, suitable replacement data for each of the sensitive data is generated. This can be done with reference to a database with suitable names and / or name / address elements, which can then be used to randomly generate suitable replacement data. Once replacement data is generated for each instance of sensitive data, the replacement data is inserted into the image in a suitable manner, thereby using the replacement data to mask and / or cover or blur the original sensitive data. The replacement data is adjusted so that the replacement data matches the font, size, style, spacing, etc. of the sensitive data. This ensures that the replacement text of the replacement data is seamlessly blended into the document.

[0036] It should be clear that randomly generating replacement data can include using a database of name / address elements, wherein the elements are randomly selected from the database and then cascaded to generate replacement data. As an example, a random name can be generated by first selecting a name from a database of known / common names (e.g., Bill, William, Jane, Joe, Tom, Stacey, etc.). Then, a database of known / common surnames is accessed to randomly select common / known surnames (e.g., Smith, Kim, Park, Singh, Fletcher, Cooper, etc.). Then, these two random elements can be cascaded together to generate a random replacement name that can be used to replace one or more instances of name data in the image of the document. For addresses, a database of known / common street names is accessed to randomly select street names (e.g., Park, No. 1, No. 2, Main Street, Wall Street, River, etc.). Then another database with public street name extensions is accessed to find random street extensions (e.g., Street, Avenue, Plaza, Road, Crescent Street, etc.). Then, these address elements can be cascaded to generate street names. For street numbers, a suitable random number generator can generate street numbers to be used with street names. Random city names can also be used in conjunction with the above methods for generating addresses. Other randomly generated replacement data can be used to replace other sensitive data. For example, a randomly generated number can be used to replace an instance of a social security number or social security number. With respect to corporate identities, random company names can also be generated by randomly selecting one or more company name elements from a database and then concatenating those elements to form a random replacement company name. Phone numbers can also be randomly generated, or known fake numbers can be inserted into a document. For example, one can randomly generate an area code and use a 555 switch with a randomly generated four-digit number to produce a known fake number, such as (212) 555-4243.

[0037] Of course, it should be clear that the contents of the databases set out above need not be fake or random data. There may be situations where one desires to insert real data into a document to replace sensitive data originally present in the document. For such situations, a suitable database containing real data points to be inserted into the document may be used.

[0038] As an alternative to the above, a specific replacement name / address may be used for each instance of an address or name, rather than generating a new replacement data point for each sensitive data segment. Thus, as an example of such an alternative, each instance of a name would be replaced with a specific name (such as John Doe), while each instance of an address would be replaced with a specific replacement address (such as 742 Evergreen Terrace, Springfield). A specific fake phone number could also be used to replace all instances of a phone number in a document.

[0039] For clarity, any of the following may be considered sensitive data in a document and may be replaced by appropriate replacement data as described above: name, address, telephone number, social insurance / social security number, health insurance number, license plate number, company name, website address, email address, vehicle identification number, IP address, insurance policy number, credit card number, ID number, date of birth, blood type, bank account data, and medical prescription data. For further clarity, the term "sensitive data" may include not only text data, but also image data, audio data, or any type of data that a user, corporate entity, or organization may not want to be available to anyone other than him or itself. Therefore, the term "sensitive data" may be defined in a contextual manner, so that what is considered sensitive data in one context may not be considered sensitive data in another context. Similarly, "sensitive data" may also include a logo, a company logo, or any other image that may be present in the image of the document. Therefore, if a logo is sensitive data, the logo may be marked and / or replaced by appropriate replacement data. For a logo, appropriate replacement data may include a generic logo, a generic image, a suitable generated image, and / or a block that effectively blocks and / or removes sensitive data in the document image.

[0040] For greater clarity, in some embodiments, any data that may be considered personal data may be included by the term "sensitive data". It should therefore be clear that "personal data" or "sensitive data" may include any information relating to an identified or identifiable natural person, and an identifiable natural person is a person who is capable of being identified directly or indirectly, in particular by reference to an identifier such as a name, identification number, location data, online identifier, or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person. "Sensitive data" may also include personal data relating to inherited or acquired genetic characteristics of a natural person, which give unique information about the physiology or health of that natural person, and which is particularly generated by the analysis of a biological sample from the natural person in question. Similarly, "sensitive data" may also include "biometric data" or personal data resulting from specific technical processing related to the physical, physiological or behavioral characteristics of a natural person, which allows or confirms the unique identity of this natural person, such as facial images or fingerprint data. "Sensitive data" may also include personal data relating to the physical or mental health of a natural person, including the provision of health care services, which reveals information about the health status of this person. In addition, “sensitive data” may include data that could be used to analyse or identify or predict a natural person’s performance at work, economic situation, health, personal preferences, interests, reliability, behaviour, location or movements.

[0041] In one embodiment of the invention, the output of the system is a list of various data clusters in the document. Parameters of each cluster, content, location / position, and parameters of the content of the cluster are associated with each cluster. Thus, the machine learning system can use this output to determine desired and / or relevant document content.

[0042] For the above embodiments, sensitive data within various data clusters within a document may be replaced within the clusters and before the data is processed by the machine learning system. Thus, the system of the present invention will process, determine and locate sensitive data within an image of a document, and before the document is processed by a machine learning system that will use the data in the document, the system will replace the sensitive data with appropriate replacement data. Thus, the sensitive data will not form part of a data set for training the machine learning system. However, it should be clear that even if the sensitive data itself does not form part of a data set for training, the context or features of the original sensitive data are retained and form part of the training data set, especially after the sensitive data is replaced by the replacement data. The placement, location and properties of the sensitive data will form part of the training data set, but the actual sensitive data itself will not form part of the training data set.

[0043] In yet another embodiment of the present invention, the image of the document, the data in each of the clusters, or in each of the bounding boxes are classified by data type. For the classification embodiment, various data clusters within the document are classified into a plurality of specific categories of data clusters. As an example, clusters may be classified into address clusters (e.g., address blocks in an invoice or statement), document title / type clusters (e.g., blocks indicating an invoice or statement), data clusters, item description clusters (e.g., descriptions of items listed item by item in an invoice, receipt, or statement), signature clusters, name clusters (e.g., salutation blocks in a letter or "Note" blocks in a statement / invoice), logo clusters, symbol clusters (e.g., handwritten note blocks in annotated documents), and email address clusters (i.e., clusters containing email addresses), etc. This classification allows the data type of the data within the cluster and / or within the bounding box to be easily determined. This measure will also reduce the burden of generating suitable replacement data points to be used to replace sensitive data within the cluster and / or within the bounding box.

[0044] In an alternative, the present invention can be implemented using multiple modules similar to the above modules or stages. In this embodiment, an image of a document is received at an input module. Then, the content of the document is divided into clusters, and the location / position of these clusters in the document and the location / position relative to other clusters are determined. Based on the location / position of specific clusters within the document and / or the content of the clusters, these clusters are classified as having specific types of data, such as names, addresses, and monetary sums. In addition to cluster orientation, other considerations can also be used to determine the type of data contained in the cluster. These considerations may include the location of the cluster relative to other clusters, the size of the cluster, and the presence or absence of specific text in the cluster. Once the data type in the cluster is determined, the relevant data can be marked as sensitive data or not sensitive data. Once the cluster and its content are determined, the nature of the data can be discovered. If sensitive data is found in different clusters, suitable replacement data is generated and then inserted into the document to replace the sensitive data. Of course, it should be clear that if the document is previously known, the location and / or positioning of different types of specific data on this document can be pre-configured within the system so that the system does not need to analyze the image and / or document to determine the location of these data segments.

[0045] refer to Figure 2 , a block diagram of a system according to one aspect of the present invention is shown. The system 110 includes an image receiving module 120, a cluster determination module 130, a position determination module 140, a data type determination module 150, a replacement data generation module 160, and a data replacement module 170. The system receives an image of a document, uses the image to determine data clusters within the document, and then determines the position / location of each cluster within the document. In addition, the system determines the position / location of the cluster relative to other clusters. The system then determines the type of data present in the cluster based on the position / location of one or more clusters and / or the content of the cluster. In addition, the nature of the data in one or more clusters (i.e., whether the data is sensitive data) is determined. Based on the type of data present in one or more clusters, appropriate replacement data is generated and the replacement data is inserted into the image of the document.

[0046] exist Figure 2In a system of the type described above, an image receiving module 120 receives an image of a document and may perform pre-processing steps, such as adjusting parameters of the image, such as contrast, sharpness, color, etc., to make the image easier to process or at least less difficult to process. A cluster determination module 130 then obtains groups of data within the image and determines which data form which clusters. A position determination module 140 then determines the absolute position (or location) of each cluster within the document, as well as the position of each cluster relative to other clusters. A data type determination module 150 then analyzes one or more clusters to determine the data type within the cluster based on a number of factors that may include the location / location of the cluster. The location used to determine the data within a cluster may be the absolute position of the cluster in the document (in terms of pixel distance or size), or the position of the cluster relative to other clusters (also in terms of pixel distance or size).

[0047] Once the data types in the cluster are determined, the data is analyzed to determine whether the data is sensitive. Based on the results of this analysis, suitable replacement data can be generated and then inserted into the document. It should be clear that the sensitivity of the data may also depend solely on the nature of the data. For example, all addresses can be considered sensitive data, and similarly, all proper names can also be considered sensitive data. It should also be clear that the determination of whether a data segment is sensitive is not necessarily a binary question, but can be based on probability and / or confidence levels. Therefore, the system can identify a specific data segment and can assign a probability or confidence level that the data segment is a specific data type (e.g., a confidence level that the data segment is an address or a person's name). In addition, the system can assign a probability that the potentially identified data is of a sensitive nature (e.g., a confidence level that the data (which can be a proper name of a person) is sensitive because it identifies the subject of the document). Based on any suitable combination of confidence levels, the system can therefore proceed to remove / replace the data segment (if the confidence level is above a predetermined threshold), do nothing with the data (if the confidence level is below a predetermined threshold), or can mark the data for review by the user (if the confidence level is close to a predetermined threshold). Of course, other configurations are possible and may depend on the desired tolerances and end results of the system.

[0048] As the sensitive data is determined, the replacement data generator 160 then generates suitable replacement data for the sensitive data found in the document. Then, the generated replacement data is inserted into the image of the document by the data replacement module 170 to replace the sensitive data. In some embodiments, the replacement data can have the same features of the sensitive data being replaced. For such an embodiment, the replacement data can have the same color, font, size and / or shadow as the sensitive data being replaced. Such an embodiment can therefore generate replacement data, which is indistinguishable from the replaced sensitive data in terms of visual features. For such an embodiment, the analysis of the sensitive data will not only generate the data type of the sensitive data, but also generate the identification of the visual and / or textual features of the sensitive data (e.g., font, font size, character size, character spacing, color, character pitch, background color, foreground color, etc.). These features of the sensitive data can therefore be used to format and / or process the replacement data to generate replacement data that is visually indistinguishable from the original sensitive data.

[0049] For the purposes of this document, "position" may be considered to refer to where in an image a cluster is found, and this "position" may be quantified in terms of pixels, pixel distances, or some other form of distance measurement from one or more fixed points in an image or document. Similarly, the term "location" may be considered to refer to an area or region within an image or document where a cluster may be found. Thus, as an example, a cluster may have a location in the upper right quadrant of an image or in the lower half of an image. Similarly, as another example, a cluster may have a location that is x pixels from the right edge of the image and y pixels from the upper edge of the image. It should be clear to those skilled in the art that the terms "position" and "location" may be used interchangeably, depending on the implementation of the system.

[0050] It should be clear that Figure 2 The system shown in the figure only represents one possible implementation of the system of the present invention. Of course, other implementations are also possible. As another possible implementation, Figure 3 A variation is shown in which the image receiving module 120 and the data type determining module 150 are connected to Figure 2 In this variation, the functionality of the data replacement generator module 160 and the data replacement module 170 are combined in a single module, the replacement module 180. Figure 3In a variation of , the functionality of modules 130 and 140 is combined in a single cluster processing module 160, the output of which is processed by a data type determination module 150. In one embodiment, the output of cluster processing module 160 includes a list of clusters (including any concatenated text in each cluster) and a location indicator for each cluster. This cluster and cluster location list can then be analyzed by data type determination module 150 to determine the data within one or more of the clusters. Also in this variation, the functionality of modules 160 and 170 is combined into module 180. Module 180 thus generates replacement data and inserts this replacement data into the relevant clusters in the document.

[0051] It should be clear that the data type determination module 150 can have a variety of implementations. In one implementation, a rule-based module is used to determine the data within one or more of the clusters in the document. The rules implemented by the module can take into account the absolute position of the cluster, its relative position relative to one or more other clusters, the length of the cluster, and the content of the cluster to determine the data within the cluster. For example, if a cluster is located near the top of the document, adjacent to two smaller clusters, and has a medium size in terms of the font size of the letters or words within the cluster, then the cluster may have an address within it. To explain the above, in a business letter, the address block of the recipient of the letter is near the top of the document and between the date and the reference line. Since the date line is a small cluster, and the reference line (i.e., RE: line) is also a relatively small cluster, the cluster between these two small clusters is typically the address block.

[0052] It should be clear that if the desired outcome is to identify addresses, dates, and total dollar amounts from a document, in a machine learning system, multiple similarities and patterns can be extrapolated by the system from the sample. In a rule-based system, the extrapolations may need to be determined in advance and then hard-coded into the rules within the system. In a typical business document, the address block is usually near the top of the document. Similarly, for invoices and receipts, the total amount of the invoice or receipt is usually set separately near the bottom of the document. The date is usually near the top of the document and is usually adjacent to or in close proximity to the address cluster. Based on a sample of typical business documents, a rule-based data determination module can be constructed. Of course, given the variability in the structure of these business documents, such a rule-based module may not work as well as expected.

[0053] In another embodiment, a neural network can be used in a data type determination module. The task of the neural network can be to classify various clusters into date clusters, address clusters, or total amount clusters. Clusters larger than a certain size (i.e., clusters with more than a predetermined amount of characters) can be discarded because they will constitute the main cluster and are not date, address, or total amount clusters. The neural network can be trained using a data set that includes different business letters, invoices, receipts, purchase orders, claim forms, payrolls, and other business-type documents whose clusters have been classified as date, address, and total amount clusters. By using such a data set, the neural network can be trained to determine which clusters in the business document contain these data points (i.e., address, date, and total amount).

[0054] Return to reference Figure 3 It should also be clear that the cluster processing module 160 can also use a neural network to determine the various clusters in the document and determine the location of each cluster in the document. The results can then be fed to the data type determination module 150 equipped with a neural network to find the date, address and amount clusters (or other clusters). The data found in these clusters can then be identified and, if necessary, replaced by suitable replacement data.

[0055] It should also be clear that the various clusters in the document can be further processed to ensure that the desired cluster has been found. For example, for a date cluster, the text within the cluster can be analyzed to scan for text corresponding to the name or abbreviation of the month. Similarly, potential address clusters can be analyzed to search for text that commonly appears in addresses (e.g., "street (street or st.)", "avenue (avenue or ave.)", etc.). The relative positioning of the clusters can also be considered when determining the type of data within a cluster. For example, a medium-sized cluster near the top of the document and a cluster between two smaller-sized clusters may be an address cluster in a letter. Similarly, a medium-sized cluster at the very top of the document (adjacent to or above one or two smaller clusters) may be an address cluster in a receipt. As another example, a medium-sized cluster just above the largest cluster may be an address cluster. The absolute positioning of the clusters can also determine the data therein. For example, the cluster at the top of each receipt has an address in it. Similarly, the total amount on any invoice or receipt is at the bottom of the document (or very close to the bottom). Of course, such analysis and processing may be encoded into the rule-based data determination module, or it may be factored in when training a suitable neural network.

[0056] In yet another variation, Figure 1The output of the system can be used as an appropriate input to a neural network for data type determination as described above. The neural network can then determine which cluster contains which data, and accordingly, which data will need to be processed and / or replaced.

[0057] It should be clear that the replacement data generation module 160 can have Figure 4 Configuration shown. Figure 4 160. In the example of FIG. 160, it can be seen that the module may have an aggregation submodule 190 that receives a request for replacement data. This request will include the type of replacement data being requested (e.g., address, name, phone number, etc.). Based on this request, appropriate replacement data elements may be collected from the database submodule 200, and appropriate digital elements may be generated from the random number generator submodule 210. The various replacement data elements are then sent back to the aggregation submodule 190, which then assembles the various elements into appropriate replacement data points. This generated replacement data may then be sent out as an output of the module 160.

[0058] Although the above discussion mentions OCR, it should be clear that the present invention can also be applied to handwritten text and / or handwritten data. Thus, handwritten text can be recognized and then appropriately marked and replaced by appropriate replacement data. Such replacement data can also be processed or generated so that the replacement data inserted into the image looks like or has the characteristics of handwritten text. Of course, as needed or desired, the replacement data can also be typed text or other suitable types of text data.

[0059] Another alternative embodiment of the present invention can use a data generation module that generates data without reference to a database. Suitable name / data generation modules can be used for such an embodiment. For example, a module with a generative adversarial network can be used that generates suitable names, addresses, and other data.

[0060] As an alternative, while the above discussion details replacing sensitive data with replacement data of the same type as the sensitive data, the sensitive data may simply be removed from the image of the document. For this alternative, the sensitive data may be masked with a solid color image block to effectively delete the sensitive data from the document. For this alternative, the replacement data would simply be a digital image block that effectively removes and / or masks the sensitive data.

[0061] It should be clear that various aspects of the present invention can be implemented as software modules in the entire software system. Thus, the present invention can take the form of computer executable instructions, which implement various software modules with predefined functions when executed.

[0062] Additionally, it should be understood that any reference herein to an "image" or "images" refers to a digital image or images comprising pixels or picture elements, unless otherwise specified. Likewise, any reference to an "audio file" or "audio files" refers to digital audio files, unless otherwise specified. "Video," "video file," "data object," "data file," and all other such terms should be understood to refer to digital files and / or data objects, unless otherwise specified.

[0063] Embodiments of the present invention may be performed by a computer processor or similar device programmed in the manner of the method steps, or may be performed by an electronic system provided with means for performing these steps. Similarly, electronic memory devices such as computer floppy disks, CD-ROMs, random access memories (RAMs), read-only memories (ROMs), or similar computer software storage media known in the art may be programmed to perform these method steps. Likewise, electronic signals representing these method steps may also be transmitted over a communication network.

[0064] Embodiments of the present invention may be implemented in any conventional computer programming language. For example, preferred embodiments may be implemented in a procedural programming language (e.g., "C" or "Go") or an object-oriented language (e.g., "C++", "java", "PHP", "PYTHON" or "C#"). Alternative embodiments of the present invention may be implemented as pre-programmed hardware elements, other related components, or a combination of hardware and software components.

[0065] The embodiments may be implemented as a computer program product for use with a computer system. Such an implementation may include a series of computer instructions that are either fixed on a tangible medium (such as a computer readable medium (e.g., a disk, CD-ROM, ROM, or fixed disk) or can be transmitted to a computer system via a modem or other interface device (such as a communication adapter connected to a network through the medium). The medium may be a tangible medium (e.g., an optical or electrical communication line) or a medium implemented using wireless technology (e.g., microwave, infrared, or other transmission technology). The series of computer instructions implements all or part of the functions previously described herein. Those skilled in the art should understand that such computer instructions can be written in a variety of programming languages ​​for use with many computer architectures or operating systems. In addition, such instructions may be used to transmit the instructions to a computer system. The command can be stored in any memory device (such as semiconductor, magnetic, optical or other memory device), and can be transmitted using any communication technology (such as light, infrared, microwave or other transmission technology). It is expected that such a computer program product can be distributed as a removable medium with an accompanying printed or electronic document (e.g., compressed packaged software), preloaded with a computer system (e.g., on a system ROM or fixed disk), or distributed from a server via a network (e.g., the Internet or the World Wide Web). Of course, some embodiments of the present invention can be implemented as a combination of software (e.g., a computer program product) and hardware. Other embodiments of the present invention can be implemented as complete hardware or complete software (e.g., a computer program product).

[0066] Those who appreciate the invention may now conceive of alternative arrangements and embodiments or variations of the above, all of which are intended to fall within the scope of the invention as defined by the appended claims.

Claims

1. A system for replacing sensitive data in a document, the system comprising: - a pre-processing module, configured to receive an image of at least one document; - a clustering module for subdividing said image of each of said at least one document into different clusters, each cluster containing data, each cluster being associated with a respective cluster position; -Data type determination module, used to: determining the content of the data within the different clusters; Determining specific data types of the data within the clusters based on the corresponding cluster positions and the content of the data within the different clusters, and identifying the specific data types in the different clusters by assigning corresponding sensitivity confidence levels, thereby determining whether the data in the different clusters includes sensitive data; as well as - A data replacement module for replacing sensitive data of at least one specific data type in the different clusters with replacement data based on the corresponding sensitivity confidence level being above a threshold.

2. The system of claim 1, wherein the sensitive data of at least one specific data type comprises at least one of the following: - Name of the person; - the person's address; - the person's telephone number; -gender; - Confidential information; - physical characteristics; - Financial information; - Health-related information; -Business related information; - Name of the corporate entity; - Address of the corporate entity; -date; -Credit Card Number; - Bank account data; - Social Security / Social Insurance Number; - Personally identifiable data; and -blood type.

3. The system of claim 1, wherein the clustering module places a bounding box around each of the different clusters.

4. The system of claim 1, wherein elements of the replacement data are randomly selected from a database.

5. The system of claim 1, wherein the replacement data is created by concatenating data elements selected from a database. The system of claim 1 , wherein a portion of the replacement data is randomly generated.

7. The system of claim 1, wherein at least a portion of the replacement data is created by concatenating data elements selected from a database.

8. The system of claim 1, wherein specific types of the sensitive data are replaced by specific replacement data.

9. The system of claim 1, wherein the data type determination module identifies a specific data type based on the presence or absence of a specific word in the data.

10. The system of claim 1, wherein the sensitive data and the replacement data replacing the sensitive data are data of the same type.

11. The system of claim 1, wherein the replacement data is a block that effectively removes the sensitive data in the image.

12. The system of claim 1, wherein the replacement data is generated by a portion of the system.

13. The system of claim 1, wherein the system identifies a visual appearance of the sensitive data.

14. The system of claim 13, wherein the visual appearance comprises at least one of: a font, size, style, and spacing of the sensitive data.

15. The system of claim 13, wherein the replacement data matches the visual appearance of the sensitive data to be replaced.

16. The system of claim 15, wherein the replacement data is seamlessly blended into the document.

17. A method for replacing sensitive data in a document, the method being executed by a processor, the method comprising: a) receiving an image of the document; b) clustering the image to obtain different clusters, each cluster field containing cluster data, each cluster being associated with a corresponding cluster position; c) determining the content of the data within the different clusters; d) determining specific data types of the data in the different clusters based on the corresponding cluster positions and the content of the data in the different clusters, and identifying the specific data types in the different clusters by assigning corresponding sensitivity confidence levels, thereby determining whether the data in the different clusters includes sensitive data; e) determining replacement data based on the type of the sensitive data determined in step d); f) inserting the replacement data into the image at the corresponding position based on the corresponding sensitivity confidence level being higher than a threshold, thereby replacing the sensitive data with the replacement data.

18. The method of claim 17, wherein the document is a business document.

19. The method of claim 18, wherein the sensitive data is at least one of: - Name of the person; - the person's address; - the person's telephone number; -gender; - Confidential information; - physical characteristics; - Financial information; - Health-related information; -Business related information; - Name of the corporate entity; - Address of the corporate entity; -date; -Credit Card Number; - Bank account data; - Social Security / Social Insurance Number; - Personally identifiable data; and -blood type.

20. The method of claim 17, wherein the replacement data is a block that effectively removes the sensitive data in the image.