Method and system for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping

Abstract

This invention provides a method and system for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping. The method includes receiving a target document, identifying the document format and determining its type; extracting text content and its coordinate information in the document's logical space using a structured parsing path or visual recognition path, and constructing a mapping set between the text and corresponding spatial positioning information; converting the text to coordinates in the document's coordinate system using a preset mapping model, where the mapping model selects the corresponding type based on whether the document exhibits distortion; loading externally configured sensitive information recognition rules and performing sensitive information matching on the extracted text content; generating a masking instruction compatible with the target document format based on the sensitive information matching result and the converted document coordinates, executing the corresponding masking operation to hide the sensitive information; saving the masked document and outputting a desensitized file, thus achieving precise location and spatial masking of privacy information.

Description

Technical Field

[0001] This invention belongs to the field of document processing technology, specifically relating to a method and system for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping. Background Technology

[0002] With the implementation of the Personal Information Protection Law and the Data Security Law, companies are required to anonymize sensitive information in documents such as contracts, resumes, and medical records. Existing technologies have the following problems: 1. Limitations of content extraction: Traditional parsing libraries (such as PDFBox and POI) are only applicable to editable documents and cannot extract text from scanned documents.

[0003] 2. Missing coordinate mapping: OCR recognition results contain image coordinates, but lack an effective mechanism to map them to the PDF / Word drawing space, resulting in misalignment of the image.

[0004] 3. Rigid rule updates: De-identification rules are often hard-coded in the program, which cannot be dynamically adjusted and results in slow response to business changes.

[0005] Therefore, there is an urgent need for an automated desensitization solution that can achieve a closed-loop linkage of "identification-location-masking". Summary of the Invention

[0006] This invention aims to solve the technical contradictions in the prior art, such as the inability to extract unstructured document content, the systematic discontinuity between OCR recognition coordinates and document drawing coordinates, and the inability to dynamically evolve desensitization rules. It provides a method and system for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping, achieving a technological leap from "seeing" to "accurately targeting".

[0007] To achieve the above objectives, one or more embodiments of this application provide a method for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping, the method comprising: S1. Receive the target document, identify the document format and determine its type, and adopt the corresponding data extraction path for different document types; the document types include at least editable documents and image documents. The editable documents can be extracted by structured parsing to extract text and coordinate information, while the image documents need to be extracted by optical character recognition technology to extract text and coordinate information. S2. For editable documents, a structured parsing path is used to extract the text content and its coordinate information in the document's logical space; for image documents, a visual recognition path is used to extract the text content and its spatial positioning information in the image space, and a mapping set between the text and the corresponding spatial positioning information is constructed. S3. For the coordinate information obtained from the structured parsing path, the coordinates under its document logical coordinate system are directly used; for the image space coordinate information obtained from the visual recognition path, the coordinates are converted into coordinates under the document coordinate system through a preset mapping model, and the mapping model selects the corresponding type according to whether the document has distortion. S4. Load the externally configured sensitive information identification rules, and realize the real-time update of the sensitive information identification rules through the hot update mechanism without restarting the service. Based on the sensitive information identification rules, perform sensitive information matching on the extracted text content. S5. Based on the sensitive information matching results and the converted document coordinates, generate a masking instruction compatible with the target document format and execute the corresponding masking operation to hide the sensitive information. S6. Save the document after the masking operation and output the de-identified file. At the same time, record the full log information of the de-identification operation for audit traceability.

[0008] Based on the above technical solution of the present invention, the following improvements can also be made: Optionally, in step S2, the specific implementation process of the structured parsing path is as follows: call the structured parsing library to extract the text content and its coordinates in the document logical space.

[0009] Optionally, in step S2, the specific implementation process of the visual recognition path is as follows: S201. Convert image documents into high-resolution images with a resolution of ≥300dpi; S202. Call the optical character recognition service to obtain the text content in the image and the corresponding spatial positioning information; the spatial positioning information includes at least the center coordinates, width, height and rotation angle φ of the smallest bounding rectangle of the text area; S203. Construct a text-spatial location information mapping set and associate text content with corresponding spatial location parameters.

[0010] Optionally, in step S3, the preset mapping model includes an affine transformation model and a homography matrix mapping model; wherein, the affine transformation model is used in the absence of distortion, and the homography matrix mapping model is used in the presence of perspective distortion.

[0011] Optionally, the transformation formula of the affine transformation model is: X_doc=s_x·x_img, Y_doc=s_y·y_img; Where X_doc and Y_doc are the target coordinates in the document coordinate system, s_x and s_y are the scaling factors in the horizontal and vertical directions, x_img and y_img are the original coordinates in the image coordinate system, s_x = DocWidth / ImgWidth, s_y = DocHeight / ImgHeight, DocWidth is the document width, DocHeight is the document height, ImgWidth is the image width, and ImgHeight is the image height.

[0012] Optionally, the homography matrix mapping model is a 3×3 real matrix H, obtained by solving for at least 4 sets of corresponding document points, and the mapping relationship satisfies: [X_doc;Y_doc;1]∝H·[x_img;y_img;1], where H∈ℝ 3 × 3 This is used to implement a non-linear mapping from image coordinates to document coordinates; where X_doc and Y_doc are the target coordinate values ​​in the document coordinate system, and x_img and y_img are the original coordinate values ​​in the image coordinate system.

[0013] Optionally, in step S4, the file corresponding to the externally configured sensitive information identification rules is in JSON or YAML format, containing sensitive information type, matching mode, desensitization action and application conditions; the matching mode includes regular expression matching and keyword matching, and the desensitization action includes solid rectangle masking and mosaic masking; the rule hot update is achieved through a file monitoring mechanism to ensure the real-time effectiveness of the desensitization rules.

[0014] Optionally, in step S5, the masking instruction is adaptively generated according to the target document format, specifically including: If the target document is in PDF format, insert a sequence of drawing instructions into the content flow, and execute the rectangle path definition, fill color setting, and fill operation in sequence. If the target document is in .docx format, insert an opaque rectangle shape, or replace sensitive text with mask characters; If the target document is an image-based PDF or TIFF format, draw solid rectangles or mosaic areas on the image pixel layer to mask sensitive information.

[0015] Optionally, in step S6, the full log information of the de-identification operation includes at least the file hash, OCR request ID, de-identification location coordinates, and rule version. The full log information of the de-identification operation is stored in the audit database for full-process traceability and verification of the de-identification operation.

[0016] According to another aspect of the present invention, a multi-format document sensitive information desensitization system based on OCT coordinate mapping is provided, including a processor and a memory, wherein the memory stores program instructions, and when the program instructions are executed by the processor, the method described in any of the above embodiments is implemented.

[0017] The beneficial effects of this invention are that it provides a method and system for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping, and integrates optical character recognition (OCR) and document editing technologies to automatically desensitize sensitive information in multi-format unstructured documents. Specifically, it achieves accurate positioning and spatial masking of privacy information such as ID card numbers and mobile phone numbers in PDF, scanned images, Word and other documents.

[0018] This invention solves the semantic gap problem between computer systems by constructing a coordinate mapping engine, enabling the visual coordinates output by OCR to directly drive the document editing engine, realizing instruction-level interoperability between heterogeneous systems, and improving the automated collaboration capabilities of computer systems.

[0019] The desensitization operation generates a sequence of drawing instructions with a preset pattern (such as consecutive re and f operations) in the PDF content stream. The coordinates of these instructions have a calculable linear relationship with the OCR recognition results. This feature can be used as a technical basis for infringement comparison. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a system architecture block diagram according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the coordinate mapping model according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the de-identification rule file structure according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the masking instruction sequence in the PDF content stream according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0022] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in one or more embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0023] like Figures 1-5 As shown, one or more embodiments of this application provide a method for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping, comprising the following steps: S1. Document Type Recognition and Access: Receive the target document and identify its format by file extension or magic number; determine whether the document is an editable document or an image document. Editable documents: including .docx, .xlsx, editable .pdf, etc., whose text content and logical coordinates can be directly extracted using a structured parsing library; Image-based documents, including scanned .pdf, .jpg, .png, .tiff, etc., require OCR technology to extract text content and image coordinates.

[0024] S2. Dual-mode content extraction and spatial positioning information acquisition: Path 1: Structured parsing path (suitable for editable documents) Use a structured parsing library to extract the text content and its coordinates (X, Y, W, H) in the document's logical space: (1) Use Apache POI to parse paragraphs and text boxes in .docx files; (2) For editable .pdf files, use PDFBox or iText to parse the text drawing instructions (such as Tj, TJ) in the content stream and calculate the coordinates in combination with the text matrix.

[0025] Path 2: Visual Recognition Path (Applicable to Image Documents) (1) Convert the document to a high-resolution image (≥300dpi); (2) Call the optical character recognition service to obtain the text content in the image and its corresponding spatial positioning information; (3) The spatial positioning information includes at least the center coordinates (x_img, y_img), width w_img, height h_img, and rotation angle θ of the smallest bounding rectangle of the text area; (4) Construct a “text-spatial location information” mapping set.

[0026] S3. Mapping of spatial positioning information to document coordinates: For path one (structured parsing path), the coordinate information is already in the drawing coordinate system of the target document, so no transformation is needed, and it can directly enter the rule matching stage.

[0027] For path two (visual recognition path), establish a mapping model from the image coordinate system to the target document coordinate system: (1) For distortion-free scenes (such as standard scanned documents), an affine transformation model is used: X_doc=s_x.x_img Y_doc=s_y.y_img Among them, s_x=DocWidth / ImgWidth, s_y=DocHeight / ImgHeight.

[0028] (2) When perspective distortion exists (such as when taking a handheld photo), a nonlinear mapping is performed using the homography matrix H: [X_doc;Y_doc;1]∝H·[x_img;y_img;1], H∈ℝ 3 × 3 ; The homography matrix is ​​obtained by solving for at least four sets of corresponding points (such as the four corners of the document).

[0029] S4. Dynamic rule loading and matching Load an external rule file (JSON / YAML format), which contains sensitive information types, matching patterns (regular expressions, keywords), desensitization actions (solid rectangles, mosaics), and application conditions.

[0030] The system uses a file monitoring mechanism to achieve hot rule updates without requiring a service restart.

[0031] S5. Generation and execution of masking instructions Based on the matching results and the converted document coordinates, generate drawing instructions compatible with the target document format and perform masking operations: (1) If the target document is in PDF format: Insert a sequence of drawing instructions into the content stream: Define a rectangular path: whXYre; set the fill color: 0g; execute the fill: f.

[0032] (2) If the target document is in Word (.docx) format: Insert an opaque rectangular shape, or replace the text with mask characters (such as ****).

[0033] (3) If the target document is an image file (PDF or TIFF): Draw solid rectangles or mosaic areas on the image pixel layer.

[0034] S6. De-identified document generation and auditing Save the modified document object and output the de-identified file; record the operation log to the audit database, including information such as file hash, OCR request ID, de-identification location, and rule version.

[0035] This embodiment also provides a multi-format document sensitive information de-identification system based on OCT coordinate mapping, including a processor and a memory. The memory stores program instructions, which, when executed by the processor, implement any of the methods described above. Specifically, the system includes: a document access module for receiving and identifying document types; a dual-mode content extraction module for selecting a parsing path based on the document type; a coordinate mapping engine for performing the mapping of spatial positioning information to document coordinates; a rule execution engine for loading rules and executing de-identification actions; and an audit log module for recording data throughout the de-identification process.

[0036] Example 1: Processing scanned PDF contracts (distortion-free) (1) Enter “Purchase Contract_Scan.pdf” and the system will recognize it as an image PDF.

[0037] (2) Use the pdf2image library to convert the first page into a PNG image (300dpi, 2480×3508 pixels).

[0038] (3) Call the OCR service and get the response: The ID number is 11010119*****7231X, and its minimum bounding rectangle is center=(310,934), w=260, h=28.

[0039] (4) The PDF page is A4 (595×842pt). Calculate the scaling factor: s_x = 595 / 2480 ≈ 0.2399 s_y = 842 / 3508 ≈ 0.2400 (5) Coordinate transformation: X_doc = 310 × 0.2399 ≈ 74.37pt Y_doc = 934 × 0.2400 ≈ 224.16pt (6) Draw a black rectangle with a position of 74.37pt×224.16pt and a size of 77.6pt×6.7pt in the PDF to complete the masking.

[0040] Example 2: Processing photographed documents (with perspective distortion) (1) Input “Holding photo of contract.jpg”, which is tilted.

[0041] (2) Detect the coordinates of the four corners of the document in the image: A_img, B_img, C_img, D_img; (3) Given the logical coordinates of the four corners of the PDF: A_doc, B_doc, C_doc, D_doc; (4) Solve for the homography matrix HH such that pPdoc ∝ Hpimg (5) Apply HH to each text block identified by OCR to perform non-linear mapping and obtain accurate document coordinates.

[0042] Example 3: Rule Hot Update and Offset Adjustment The system dynamically adjusts the offset of the masking graphic based on the font size (e.g., 12pt) and the rendering engine (e.g., Adobe PDF Library) to ensure that rendering deviations at the font edges are covered.

[0043] Exception handling mechanism: If the OCR service returns an empty result, the system will automatically retry up to 3 times. If it still fails, mark the page as "OCR recognition failed", log the information, and skip it. If the coordinates are abnormal, the system will automatically correct or crop them.

[0044] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0045] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0046] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0047] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0048] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0049] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping, characterized in that, include: S1. Receive the target document, identify the document format and determine its type, and use the corresponding data extraction path for different document types; The document types include at least editable documents and image documents. The editable documents can be extracted for text and coordinate information through structured parsing, while the image documents need to be extracted for text and coordinate information through optical character recognition technology. S2. For editable documents, a structured parsing path is used to extract the text content and its coordinate information in the document's logical space; for image documents, a visual recognition path is used to extract the text content and its spatial positioning information in the image space, and a mapping set between the text and the corresponding spatial positioning information is constructed. S3. For the coordinate information obtained from the structured parsing path, the coordinates under its document logical coordinate system are directly used; for the image space coordinate information obtained from the visual recognition path, the coordinates are converted into coordinates under the document coordinate system through a preset mapping model, and the mapping model selects the corresponding type according to whether the document has distortion. S4. Load the externally configured sensitive information identification rules, and realize the real-time update of the sensitive information identification rules through the hot update mechanism without restarting the service. Based on the sensitive information identification rules, perform sensitive information matching on the extracted text content. S5. Based on the sensitive information matching results and the converted document coordinates, generate a masking instruction compatible with the target document format and execute the corresponding masking operation to hide the sensitive information. S6. Save the document after the masking operation and output the de-identified file. At the same time, record the full log information of the de-identification operation for audit traceability.

2. The method for desensitizing sensitive information in multi-format documents based on OCT coordinate mapping according to claim 1, characterized in that, In step S2, the specific implementation process of the structured parsing path is as follows: call the structured parsing library to extract the text content and its coordinates in the document logical space.

3. The method for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping according to claim 1, characterized in that, In step S2, the specific implementation process of the visual recognition path is as follows: S201. Convert image documents into high-resolution images with a resolution of ≥300dpi; S202. Call the optical character recognition service to obtain the text content in the image and the corresponding spatial positioning information; the spatial positioning information includes at least the center coordinates, width, height and rotation angle φ of the smallest bounding rectangle of the text area; S203. Construct a text-spatial location information mapping set and associate text content with corresponding spatial location parameters.

4. The method for desensitizing sensitive information in multi-format documents based on OCR coordinate mapping according to claim 1, characterized in that, In step S3, the preset mapping model includes an affine transformation model and a homography matrix mapping model; wherein, the affine transformation model is used in the absence of distortion, and the homography matrix mapping model is used in the presence of perspective distortion.

5. The method for desensitizing sensitive information in multi-format documents based on OCT coordinate mapping according to claim 4, characterized in that, The transformation formula for the affine transformation model is: X_doc=s_x·x_img, Y_doc=s_y·y_img; Where X_doc and Y_doc are the target coordinates in the document coordinate system, s_x and s_y are the scaling factors in the horizontal and vertical directions, x_img and y_img are the original coordinates in the image coordinate system, s_x = DocWidth / ImgWidth, s_y = DocHeight / ImgHeight, DocWidth is the document width, DocHeight is the document height, ImgWidth is the image width, and ImgHeight is the image height.

6. The method for desensitizing sensitive information in multi-format documents based on OCT coordinate mapping according to claim 4, characterized in that, The homography matrix mapping model is a 3×3 real matrix H, obtained by solving for at least 4 sets of corresponding document points, and the mapping relationship satisfies: [X_doc;Y_doc;1]∝H·[x_img;y_img;1], where H∈ℝ 3 × 3 It is used to implement non-linear mapping from image coordinates to document coordinates; Where X_doc and Y_doc are the target coordinates in the document coordinate system, and x_img and y_img are the original coordinates in the image coordinate system.

7. The method for desensitizing sensitive information in multi-format documents based on OCT coordinate mapping according to claim 1, characterized in that, In step S4, the file corresponding to the externally configured sensitive information identification rules is in JSON or YAML format and includes the sensitive information type, matching mode, desensitization action and application conditions; the matching mode includes regular expression matching and keyword matching, and the desensitization action includes solid rectangle masking and mosaic masking. The file monitoring mechanism enables hot updates of rules, ensuring the real-time effectiveness of the de-identification rules.

8. The method for desensitizing sensitive information in multi-format documents based on OCT coordinate mapping according to claim 1, characterized in that, In step S5, the masking instruction is adaptively generated according to the target document format, specifically including: If the target document is in PDF format, insert a sequence of drawing instructions into the content flow, and execute the rectangle path definition, fill color setting, and fill operation in sequence. If the target document is in .docx format, insert an opaque rectangle shape, or replace sensitive text with mask characters; If the target document is an image-based PDF or TIFF format, draw solid rectangles or mosaic areas on the image pixel layer to mask sensitive information.

9. The method for desensitizing sensitive information in multi-format documents based on OCT coordinate mapping according to claim 1, characterized in that, In step S6, the full log information of the de-identification operation includes at least the file hash, OCR request ID, de-identification location coordinates, and rule version. The full log information of the de-identification operation is stored in the audit database for full-process traceability and verification of the de-identification operation.

10. A multi-format document sensitive information desensitization system based on OCT coordinate mapping, comprising a processor and a memory, characterized in that, The memory stores program instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 9.

Images