Picture sensitive information automatic coding method, device and equipment and storage medium

Abstract

The application discloses a kind of picture sensitive information automatic coding method, device, equipment and storage medium, the method includes: the picture to be processed is input into pre-trained picture semantic understanding model, and the information type of the picture to be processed is identified;The picture to be processed is carried out optical character recognition, and the text content in the picture to be processed and the boundary box coordinate information of each text content in the picture to be processed are extracted;The text content, the boundary box coordinate information and the information type are input into pre-trained large language model, and whether each text content is sensitive information is judged;According to the type of sensitive information, the corresponding sensitive level is determined, according to the sensitive level, the corresponding coding mode and coding intensity are determined, the region corresponding to the boundary box coordinate of the sensitive information in the picture to be processed is coded and processed, and the coded picture is generated.The application can be applied to insurance claim, financial risk control, identity verification and other scenes in picture privacy information protection.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method, apparatus, device, and storage medium for automatically decrypting sensitive information in images. Background Technology

[0002] With the deepening of digital transformation, a large amount of sensitive information, including vehicle registration certificates, driver's licenses, ID cards, and vehicle photos, is circulating in business systems within the insurance industry and the broader financial sector. These images contain sensitive personal information such as names, ID numbers, license plate numbers, vehicle identification numbers, and mobile phone numbers. Leakage of such information could lead to serious privacy and security incidents and compliance risks.

[0003] Currently, the processing of sensitive information in images mainly relies on the following technical solutions: (1) Purely manual review mode. Most insurance companies use the traditional method of claims adjusters manually checking photos and editing and redacting them. This method relies entirely on human professional ethics and sense of responsibility, and the operation is cumbersome and time-consuming. New employees are prone to leakage of sensitive information due to lack of experience.

[0004] (2) OCR solutions based on rule engines. This type of solution can only recognize text content.

[0005] (3) End-to-end solutions driven by a single model. This type of solution has poor model generalization ability, and the accuracy of image recognition outside the training set drops sharply; it consumes huge computational resources and cannot flexibly adjust the strategy.

[0006] (4) Traditional image processing decryption tools. The mosaic function of existing image editing software tools relies on passive operation, requiring manual location of sensitive areas one by one; it lacks intelligent recognition capabilities and cannot automatically discover hidden sensitive information; it also lacks batch processing capabilities.

[0007] In summary, existing technical solutions generally suffer from a broken "identification-judgment-execution" chain, failing to construct a complete closed loop from image type recognition to sensitive information location and then to intelligent decryption. Summary of the Invention

[0008] This invention provides a method, apparatus, device, and storage medium for automatically decrypting sensitive information in images, in order to solve the technical defects of existing technologies, such as low efficiency due to reliance on manual operation for processing sensitive information in images, lack of semantic understanding capability in OCR schemes, poor generalization ability and lack of interpretability of single model schemes, and lack of automatic recognition capability in traditional tools.

[0009] Firstly, a method for automatically decrypting sensitive information in images is provided, including: Get the image to be processed; The image to be processed is input into a pre-trained image semantic understanding model to identify the information type of the image to be processed; the information type is used to characterize the document category or scene category to which the image to be processed belongs; Determine whether the information type belongs to a preset whitelist type. If so, perform optical character recognition on the image to be processed to extract the text content in the image to be processed and the bounding box coordinate information of each text content in the image to be processed. Determine whether the confidence level of the optical character recognition result is greater than a preset threshold. If so, input the text content, the bounding box coordinate information, and the information type into a pre-trained large language model. Based on the sensitivity thresholds corresponding to different information types, determine whether each text content is sensitive information and output the type and confidence score of the sensitive information. The sensitivity level is determined according to the type of the sensitive information, and the corresponding masking method and masking intensity are determined according to the sensitivity level. The area in the image to be processed that corresponds to the bounding box coordinates of the sensitive information is masked according to the masking method and masking intensity to generate a masked image.

[0010] Secondly, an automatic image sensitive information decryption device is provided, including: The image acquisition module is used to acquire images to be processed. The image semantic recognition module is used to input the image to be processed into a pre-trained image semantic understanding model to identify the information type of the image to be processed; the information type is used to characterize the document category or scene category to which the image to be processed belongs; The type verification module is used to determine whether the information type belongs to a preset whitelist type. If so, the information type is output to the optical character recognition module. An optical character recognition module is used to perform optical character recognition on the image to be processed, and extract the text content in the image to be processed and the bounding box coordinate information of each text content in the image to be processed. The OCR verification module is used to determine whether the confidence level of the optical character recognition result is greater than a preset threshold. If so, the optical character recognition result is output to the sensitive information determination module. The sensitive information determination module is used to input the text content, the bounding box coordinate information, and the information type into a pre-trained large language model, determine whether each text content is sensitive information based on the sensitivity threshold corresponding to different information types, and output the type and confidence score of the sensitive information. An adaptive decryption module is used to determine the corresponding sensitivity level based on the type of sensitive information, determine the corresponding decryption method and decryption intensity based on the sensitivity level, and decrypt the area in the image to be processed corresponding to the bounding box coordinates of the sensitive information according to the decryption method and decryption intensity to generate a decrypted image.

[0011] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-mentioned automatic image sensitive information decryption method.

[0012] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described method for automatically decrypting sensitive information in images.

[0013] The above-mentioned method, apparatus, computer equipment, and storage medium for automatic image sensitive information decryption involves inputting the image to be processed into a pre-trained image semantic understanding model to identify the information type of the image; performing optical character recognition on the image to extract the text content and the bounding box coordinates of each text content within the image; inputting the text content, the bounding box coordinates, and the information type into a pre-trained large language model, which determines whether each text content is sensitive information based on sensitivity thresholds corresponding to different information types, and outputs the type and confidence score of the sensitive information; determining the corresponding sensitivity level based on the type of sensitive information, determining the corresponding decryption method and decryption strength based on the sensitivity level, and decrypting the area in the image to be processed corresponding to the bounding box coordinates of the sensitive information according to the decryption method and decryption strength to generate a decrypted image. This constructs a four-in-one automatic protection chain of "identification → positioning → classification → decryption" that is completely absent in existing technologies, achieving a leap from passive manual operation to proactive intelligent protection. Specifically, through a collaborative mechanism of "image information type recognition + differentiated sensitivity threshold," the targeting and accuracy of sensitive information determination are significantly improved, while reducing false positive and false negative rates. Through spatial-semantic joint analysis of "OCR bounding box coordinate extraction + large language model semantic understanding," integrated and precise image encoding from "semantic detection" to "spatial localization" is achieved for the first time, filling the technical gap of existing technologies that can only detect but not locate. Through the combination of "large language model semantic determination + hierarchical adaptive encoding," a qualitative leap from "sensitive word matching" to "semantic understanding" is achieved, fundamentally solving the technical pain point of traditional solutions that "have recognition but no understanding." Through a hierarchical mapping mechanism of "sensitivity level → encoding method → ​​encoding strength," an optimal balance between privacy protection strength and image information retention is achieved. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating a method for automatically decrypting sensitive information in images according to an embodiment of the present invention.

[0016] Figure 2 yes Figure 1 A flowchart illustrating the sensitive information determination process in step S5.

[0017] Figure 3 This is a schematic diagram of an automatic image sensitive information decoding device according to an embodiment of the present invention.

[0018] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention.

[0019] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The automatic image sensitive information decryption method and device based on image recognition provided by this invention can be widely used in business scenarios that require processing images containing sensitive information, such as insurance claims, auto insurance underwriting, personal injury claims, anti-fraud investigations, identity verification, and financial risk control.

[0022] Please see Figure 1 As shown, Figure 1 A flowchart illustrating an automatic image sensitive information decoding method provided in an embodiment of the present invention includes the following steps: S1 retrieves the image to be processed.

[0023] In this embodiment, the image to be processed can be a photo of a vehicle registration certificate, driver's license, ID card, vehicle, or other images containing sensitive information. Image formats include, but are not limited to, common formats such as JPEG, PNG, and BMP.

[0024] Furthermore, after step S1, the following steps are also included: S101 performs a validity check on the image to be processed.

[0025] The system verifies the validity of the link or data source of the image to be processed. Specifically, if the image to be processed is provided as a URL link, the system checks whether the link is accessible and whether there is a risk of malicious URL injection; if the image to be processed is provided as a data stream, the system checks whether the data stream is complete and readable. If the verification passes, step S102 is executed; otherwise, processing is terminated and an error message is returned. This step effectively prevents malicious URL injection attacks.

[0026] S102 The image to be processed is transferred to the internal storage system.

[0027] Images that pass verification are transferred to an internal storage system (such as a distributed object storage service) and a unified internal storage identifier (such as a fileId) is generated to ensure data traceability. The original images are permanently retained to meet audit compliance requirements.

[0028] S2 inputs the image to be processed into a pre-trained image semantic understanding model to identify the information type of the image to be processed; the information type is used to characterize the document category or scene category to which the image to be processed belongs.

[0029] Document category refers to formal written credentials used to prove an individual's identity, qualifications, or rights. In embodiments of the present invention, the image semantic understanding model can identify the following document categories: (1) Vehicle registration certificate A vehicle registration certificate is a legal document that permits a motor vehicle to be driven on roads within my country. In insurance claims, a photo of the vehicle registration certificate is a crucial document for verifying the information of the insured vehicle. The vehicle registration certificate consists of an original and a duplicate, containing key information such as the vehicle license plate number, vehicle identification number (VIN), engine number, registration date, and inspection validity period. When the image semantic understanding model identifies the image to be processed as a "vehicle registration certificate," the system calls the corresponding sensitivity threshold (e.g., a threshold of 0.85 for the VIN and license plate number) to perform targeted sensitive information determination on the text recognized by OCR.

[0030] (2) Driver's license A driver's license is a legally obtained document proving a driver's qualification to drive. In insurance claims, a copy of the driver's license is a necessary document to verify whether the driver was legally qualified to drive at the time of the accident. A driver's license contains personal information such as the driver's name, license number, permitted vehicle type, and validity period. Once the image semantic understanding model identifies the image to be processed as a "driver's license," the system will focus on detecting and classifying sensitive information fields specific to driver's licenses (such as name and license number).

[0031] (3) ID card An ID card is a legally recognized identity document for citizens. In insurance claims, photocopies of the ID cards of both the insured and the payee are essential claim documents. The front of the ID card contains sensitive information such as name, gender, ethnicity, date of birth, address, and citizen identification number; the back contains the issuing authority and validity period. Once the image semantic understanding model identifies the image to be processed as an "ID card," the system employs a high-sensitivity protection strategy (such as using a privacy shield mask + 15px Gaussian blur on the citizen identification number) to ensure that core privacy information receives the highest level of protection.

[0032] Scene category refers to the type of real-world photograph reflecting the accident scene environment, vehicle condition, or damage. In embodiments of the present invention, the image semantic understanding model can identify the following scene categories: (1) Panoramic photo of the accident scene Panoramic photos of accident scenes are used to reflect the overall environment and overview of the accident scene, including road conditions, surrounding landmarks, the relative positions of the vehicles involved in the accident on the road, and the nature of the accident. Specifically, panoramic photos need to clearly reflect road conditions and the surrounding environment, capture signs at the accident location (road signs, bus stop signs, prominent landmarks), and show the specific location of the scene on the road (such as the top of a slope, the middle of a slope, a curve, etc.). These photos mainly record scene information and usually do not contain sensitive personal information. Therefore, when the model identifies a photo as a "panoramic photo of an accident scene," the system uses a low sensitivity threshold (such as 0.95) to avoid unnecessary masking of non-sensitive areas.

[0033] (2) Photos of the damaged parts of the vehicle Photos of damaged vehicle parts are used to reflect the actual extent and degree of damage to the insured vehicle. Specifically, this includes close-up photos of the damaged areas, clearly showing the depth of the collision, the length of the scratches, and the area of ​​damage. For cases involving significant damage, photos of the overall damage must also be taken from all four angles of the vehicle to reflect the overall damage situation. Furthermore, for photos containing vehicle identification numbers (VINs), the VIN and engine number must be clearly photographed. When the model identifies a photo as a "vehicle damaged part photo," the system will focus on whether the photo contains the VIN—this information is usually located on the dashboard in the lower left corner of the windshield and is considered medium-sensitivity information. The system will use dynamic mosaic combined with Gaussian blur to mask this information, protecting privacy while retaining the damage assessment information.

[0034] (3) A photo of the whole vehicle with license plate Full vehicle photos with license plates are used to reflect the overall appearance of the vehicle involved in the accident. The photos must clearly show the insured vehicle, license plate number, and the damaged parts of the vehicle. The license plate number is core sensitive information in such photos. When the model recognizes a "full vehicle photo with license plate," the system focuses on locating and blurring the license plate area, while retaining non-sensitive information such as vehicle color and model.

[0035] The image to be processed is input into a pre-trained image semantic understanding model, which outputs the information type of the image. The image semantic understanding model adopts the ViT-B / 16 architecture. The input image is first divided into several image patches. Each image patch is linearly projected to form an image patch embedding vector, which, after adding positional encoding, is input into a Transformer encoder. The Transformer encoder extracts global semantic features of the image through a multi-head self-attention mechanism and a feedforward neural network. Finally, a classifier outputs the information type of the image, including vehicle registration certificate, driver's license, ID card, vehicle photo, etc.

[0036] In this embodiment, the ViT-B / 16 model is fine-tuned and trained on a dataset of over 100,000 insurance scene images. The training data covers image samples with different lighting conditions, shooting angles, and degrees of occlusion, ensuring the model's generalization ability in real-world business scenarios.

[0037] S3 determines whether the information type belongs to a preset whitelist type. If so, optical character recognition is performed on the image to be processed to extract the text content in the image to be processed and the bounding box coordinate information of each text content in the image to be processed; otherwise, the processing is terminated.

[0038] determining whether the information type identified in step S2 belongs to a preset whitelist type. The whitelist type is configured according to the application scenario, and in this embodiment includes a driving license, a vehicle driving permit, an identity card, and a vehicle photo. If the identified information type belongs to the whitelist, subsequent processing is continued; otherwise, the processing is terminated. This mechanism ensures that the system only processes images of known types, avoiding unnecessary computation on irrelevant images (such as landscape photos, portrait photos, etc.).

[0039] performing text recognition on the to-be-processed image by using an optical character recognition engine, and extracting the text content in the to-be-processed image and the bounding box coordinate information corresponding to each piece of text content.

[0040] In some cases, the optical character recognition engine may adopt the PaddleOCR high-precision version, which supports skewed text detection. The PaddleOCR engine can effectively process certificate photos with tilted shooting angles, has high recognition accuracy, and the processing speed for a single image is about 200 milliseconds. The results output by the OCR engine include: text content (such as "Name: Zhang San", "License Plate Number: Jing A88888") and corresponding bounding box coordinates (x_min, y_min, x_max, y_max).

[0041] S4: determining whether the confidence level of the optical character recognition result is greater than a preset threshold, and if yes, inputting the text content, the bounding box coordinate information and the information type into a pre-trained large language model, judging whether each piece of text content is sensitive information according to the sensitivity threshold corresponding to different information types, and outputting the type of the sensitive information and a confidence score.

[0042] determining whether the OCR recognition result of step S3 is valid. Specifically, it is determined whether the confidence level of the OCR recognition result is greater than a preset threshold (the threshold is set to 0.75 in this embodiment). If the confidence level is less than the threshold, the processing is terminated. This step effectively filters out low-quality OCR recognition results, and avoids masking incorrectly recognized text as sensitive information.

[0043] Further, after determining whether the confidence level of the optical character recognition result is greater than the preset threshold, the method further includes: performing standardized cleaning on the text content recognized by the optical character recognition, where the standardized cleaning includes removing special characters and / or unifying the encoding format.

[0044] performing standardized cleaning on the text content recognized by the optical character recognition includes removing special characters (such as line breaks and redundant spaces) and unifying the encoding format (such as converting full-width characters into half-width characters). The standardized text content facilitates subsequent semantic analysis by the large language model.

[0045] Input the standardized text content and corresponding bounding box coordinate information into a pre-trained large language model, which determines whether each piece of text content is sensitive information, and outputs the type of the sensitive information and a confidence score.

[0046] In some cases, the large language model is fine-tuned and trained based on the Qwen-7B model. Specifically, as Figure 2 shown, the sensitive information determination process of the present invention includes: First, convert the bounding box coordinates (x_min, y_min, x_max, y_max) output by OCR into spatial embedding vectors. The spatial embedding vectors are obtained by performing normalization encoding on the coordinate values, and reflect the relative position of the text region in the image (such as top left corner, top right corner, center, etc.).

[0047] Next, concatenate the spatial embedding vectors with the semantic vectors of the text content to form spatial-semantic joint encoding vectors. This joint encoding method enables the large language model to not only understand the semantics of the text content (for example, "Jing A88888" is a string of characters), but also use the spatial position feature of the text in the image (for example, "located at the top right corner of the image, a typical license plate position") for auxiliary determination.

[0048] "Semantic vector" refers to the process of converting text content (such as "Jing A88888") into a numerical vector (that is, a set of floating-point numbers) that can be understood and calculated by a computer. Its core goal is to make texts with similar semantics also have a closer distance in the vector space.

[0049] In specific implementation, a pre-trained language model (Embedding Model) is usually used to complete this conversion.

[0050] Code example generated using the Sentence-BERT model: # 1. Install the required libraries # pip install sentence-transformers # 2. Import and load the model from sentence_transformers import SentenceTransformer # Load a pre-trained Chinese semantic model (for example, paraphrase-multilingual-MiniLM-L12-v2) # This model maps text to a 384-dimensional vector space model = SentenceTransformer('paraphrase-multilingual-MiniLM-L12-v2') # 3. Convert the text content recognized by OCR into semantic vectors # Assume this is the text recognized by OCR from the image sentences = "Jing A88888", # License plate number "Zhang San", # Name "A certain street, Chaoyang District, Beijing" # Address # The model.encode() method directly converts the text list into a vector list[reference:6] sentence_embeddings = model.encode(sentences) # View the result: each sentence is converted into a 384-dimensional vector print(sentence_embeddings.shape) # Output: (3, 384) # You can view the vector of the first sentence "Jing A88888" (partially intercepted) print(sentence_embeddings[0][:10]) # Output: [-0.023, 0.145, ...,0.089] Then, the spatial-semantic joint encoding vector is input into the large language model, and the large language model outputs the judgment result of whether each text content is sensitive information and the corresponding confidence score.

[0051] In addition, the input of the large language model also includes the information type identified in step S2. The large language model adopts differentiated sensitivity thresholds for judgment according to different information types: Driving license: the sensitivity threshold for name and certificate number is set to 0.7 (high sensitivity); Vehicle photo: the sensitivity threshold for frame number and license plate number is set to 0.85 (medium sensitivity); Ordinary landscape photo: the sensitivity threshold is set to 0.95 (low sensitivity).

[0052] This differentiated threshold setting enables the system to adaptively adjust the strictness of sensitive information judgment according to the image type, ensuring that no sensitive information is missed while avoiding excessive mosaicking.

[0053] ​The present invention initiates a three-layer recognition architecture of "image type prior judgment + OCR text recognition + large language model semantic verification", and proposes a spatial-semantic joint coding method for sensitive information localization. All components adopt a modular decoupling design, supporting independent upgrade and replacement; misjudged samples are continuously collected for incremental training through a closed-loop feedback mechanism; containerized deployment and elastic scaling are supported. The technical solution of the present invention can be quickly migrated to more than ten business links such as auto insurance underwriting, personal injury claim adjustment, and anti-fraud investigation.

[0054] Further, step S4 further includes a dual-channel verification mechanism: In S401, when the judgment result conflicts with the optical character recognition result, a secondary verification process is triggered, and a third-party optical character recognition interface is called for cross verification; A weighted voting mechanism is used to perform fusion decision on the judgment result of the large language model, the optical character recognition result and the verification result of the third-party optical character recognition interface, and output the final sensitive information determination result.

[0055] Specifically, channel A and channel B are arranged for parallel processing, wherein channel A is the structured output of the optical character recognition engine, and channel B is the semantic judgment output of the large language model. When the OCR recognition result of channel A conflicts with the large language model judgment result of channel B (for example, OCR recognizes "License Plate No.: Jing A88888" but the large language model judges it as non-sensitive), a secondary verification process is triggered, and a third-party OCR API is called for cross verification. Finally, based on the weighted voting mechanism (the weight of channel A is 0.6, and the weight of channel B is 0.4), fusion decision is performed on channel A, channel B and the third-party verification result, and the final sensitive information determination result is output.

[0056] The present invention creatively proposes a dual-channel verification mechanism for sensitive information with the collaboration of OCR and a large language model. The traditional OCR solution can only recognize text content and cannot judge whether the text is sensitive information; while the present invention can distinguish the field name of "license plate number" from the field value of "Jing A88888" through the semantic understanding capability of the large language model, so as to accurately identify the real sensitive information. At the same time, the present invention converts the bounding box coordinates output by OCR into spatial embedding vectors, which are spliced with text semantic vectors and then input into the large language model, so that the model can use the spatial position feature of "Jing A88888 is located at the upper right corner of the image" to assist judgment, further improving the accuracy of sensitive information recognition. When the judgment results of OCR and the large language model conflict, the secondary verification process of cross verification by the third-party OCR API is triggered, which effectively suppresses false positives.

[0057] Further, post-processing is performed on the judged sensitive information, including: S402 maps the bounding box coordinates of sensitive information from the OCR recognition coordinate system to the original image coordinate system, removes duplicate bounding boxes, and merges adjacent bounding boxes.

[0058] For example, when “Name: Zhang San” is split into two independent OCR recognition results, “Name:” and “Zhang San”, and both are identified as sensitive information, the system merges these two adjacent bounding boxes into a complete sensitive information region.

[0059] Furthermore, after step S4, the following steps are also included: S403 determines whether the area of ​​each sensitive information region is greater than the preset minimum area threshold. If so, the sensitive information is retained and step S5 is executed; otherwise, the sensitive information is filtered.

[0060] Specifically, it is determined whether the area of ​​each sensitive information region after processing in step S4 is greater than a preset minimum area threshold (set to 50 square pixels in this embodiment). If it is greater than the threshold, the sensitive information is retained and step S5 is executed; otherwise, the sensitive information is filtered. This step effectively filters out noise regions with excessively small areas, avoiding unnecessary masking of tiny text (such as page numbers, watermarks, etc.) in the image.

[0061] S5 determines the corresponding sensitivity level based on the type of the sensitive information, determines the corresponding redacting method and redacting intensity based on the sensitivity level, and redacts the area in the image to be processed that corresponds to the bounding box coordinates of the sensitive information according to the redacting method and redacting intensity to generate a redacted image.

[0062] The corresponding sensitivity level is determined based on the type of sensitive information output in step S4, and the corresponding censoring method and censoring strength are determined based on the sensitivity level.

[0063] Specifically: (1) When the sensitivity level is high sensitivity level (such as ID card number, mobile phone number), privacy shield masking with 15px Gaussian blur radius is used for masking, and the edge of the masked area is feathered to avoid hard boundary.

[0064] "Privacy Shield Mask" is an image protection algorithm for areas with extremely high privacy risks. Its features include: extracting and calculating the optimal background fill color for the area, completely covering the original pixels with a solid color mask to block reverse restoration; on this basis, superimposing a strong Gaussian blur to eliminate texture residuals, and using edge feathering technology to achieve a smooth and seamless fusion between the overlay layer and the background image, thereby achieving "absolutely irreversible desensitization" while ensuring the overall visual professionalism and naturalness of the image.

[0065] (2) When the sensitivity level is medium sensitivity level (such as license plate number, vehicle frame number), dynamic mosaic with a Gaussian blur radius of 10px is used for masking. The cell size of the dynamic mosaic is determined according to 1.2 times the height of the text in the sensitive information area.

[0066] "Dynamic Mosaic" is an adaptive pixelation occlusion technique that differs from traditional fixed-size mosaics. Its core feature is that the cell size of the mosaic (i.e., the blur granularity) is not a pre-set fixed value (such as a fixed 8×8 pixels), but is calculated in real time based on the actual height of the text within the current sensitive information area.

[0067] (3) When the sensitivity level is low (such as name, address), use lightweight mosaic (8×8 pixel block) with 5px Gaussian blur radius for masking.

[0068] "Lightweight Mosaic" is the pixelation occlusion technique with the lightest computational load and highest information retention in the three-level coding system of this invention. Its core feature is that it uses fixed small-sized pixel blocks (preferably 8×8 pixels in this solution) to perform light blurring processing on low-sensitivity information areas, aiming to achieve "minimum necessary occlusion" rather than complete pixel destruction.

[0069] This invention proposes a graded adaptive masking strategy based on sensitivity levels. Traditional mosaic masking suffers from a "one-size-fits-all" problem, either excessively obscuring information and affecting the recognition of effective information, or insufficiently masking information and leading to privacy leaks. This invention determines the sensitivity level (high / medium / low) based on the type of sensitive information and adopts differentiated masking methods and intensities for different levels: highly sensitive information (ID numbers, mobile phone numbers) uses a privacy shield mask combined with 15px Gaussian blur and edge feathering; medium sensitive information (license plate numbers, vehicle identification numbers) uses dynamic mosaic combined with 10px Gaussian blur; and low sensitive information (names, addresses) uses lightweight mosaic combined with 5px Gaussian blur. This graded processing method effectively protects privacy while preserving the effective damage assessment information in the image to the greatest extent.

[0070] In this embodiment, the masking process is implemented based on an adaptive blurring algorithm using a generative adversarial network (GAN). This algorithm can adaptively adjust the blurring parameters according to the content characteristics of the sensitive information area (such as text density, color distribution, etc.), effectively masking the sensitive information while allowing the masked area to transition naturally with the surrounding background.

[0071] The generated, decrypted image is uploaded to the storage system, a new file identifier (such as a new fileId) is generated, and the decrypted image and the file identifier are output. The original image and the newly generated decrypted image are distinguished and managed using different file identifiers, facilitating auditing and traceability.

[0072] This invention constructs a complete processing chain from image acquisition to decryption output. Each processing node is equipped with validity checks and exception handling to avoid unnecessary computational resource consumption; OCR validity checks and sensitive information validity checks form a double filtering process before and after; after decryption, the image generates an independent file identifier, while the original image is permanently retained; all threshold parameters (confidence, area, blur radius) support dynamic adjustment at runtime. This closed-loop system ensures mandatory coverage and audit traceability of the processing flow, fundamentally eliminating the process loopholes in traditional solutions that "depend on human integrity."

[0073] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0074] This invention employs a distributed pipeline architecture to achieve asynchronous parallel processing. Steps S1 to S5 are executed in parallel using the distributed pipeline architecture, which includes multiple worker nodes. Each worker node is responsible for at least one of the following tasks: image download and format conversion, image information type recognition, optical character recognition, large language model semantic analysis, coordinate calculation and region merging, image decryption processing, image uploading, quality sampling, and log recording.

[0075] The main thread is responsible for receiving user requests, writing tasks to the task queue, tracking task status, and returning processing results to the user.

[0076] The worker pool contains 8 concurrent processes: Worker1: Responsible for image downloading and format conversion; Worker2: Responsible for ViT image information type recognition; Worker3: Responsible for OCR text extraction; Worker4: Responsible for semantic analysis of large language models; Worker5: Responsible for coordinate calculation and region merging; Worker6: Responsible for image decryption processing; Worker7: Responsible for uploading images after blurring; Worker8: Responsible for quality spot checks and log recording.

[0077] Each worker node executes in parallel without blocking the others. Testing showed that the distributed pipeline architecture of this invention reduces the average processing time for a single image to 3.2 seconds, achieving a system throughput of over 500 QPS.

[0078] In one embodiment, an automatic image sensitive information decryption device is provided, which corresponds one-to-one with the image sensitive information automatic decryption method described in the above embodiments. For example... Figure 3 As shown, the automatic image sensitive information decryption device includes an image acquisition module 101, an image semantic recognition module 102, a type verification module 103, an optical character recognition module 104, an OCR verification module 105, a sensitive information determination module 106, and an adaptive decryption module 107. Detailed descriptions of each functional module are as follows: The image acquisition module is used to acquire images to be processed. The image semantic recognition module is used to input the image to be processed into a pre-trained image semantic understanding model to identify the information type of the image to be processed; the information type is used to characterize the document category or scene category to which the image to be processed belongs; The type verification module is used to determine whether the information type belongs to a preset whitelist type. If so, the information type is output to the optical character recognition module. An optical character recognition module is used to perform optical character recognition on the image to be processed, and extract the text content in the image to be processed and the bounding box coordinate information of each text content in the image to be processed. The OCR verification module is used to determine whether the confidence level of the optical character recognition result is greater than a preset threshold. If so, the optical character recognition result is output to the sensitive information determination module. The sensitive information determination module is used to input the text content, the bounding box coordinate information, and the information type into a pre-trained large language model. The large language model determines whether each text content is sensitive information based on the sensitivity threshold corresponding to different information types, and outputs the type of sensitive information and confidence score. An adaptive decryption module is used to determine the corresponding sensitivity level based on the type of sensitive information, determine the corresponding decryption method and decryption intensity based on the sensitivity level, and decrypt the area in the image to be processed corresponding to the bounding box coordinates of the sensitive information according to the decryption method and decryption intensity to generate a decrypted image.

[0079] Furthermore, the automatic image sensitive information decryption device also includes: The standardization module is used to perform standardization cleaning on the text content recognized by the optical character recognition module. The standardization cleaning includes removing special characters and / or unifying the encoding format.

[0080] Furthermore, the automatic image sensitive information decryption device also includes a sensitive information post-processing module, which is used for: Map the coordinates of the bounding box identified as sensitive information to the original coordinate system of the image to be processed; Remove duplicate bounding boxes that overlap after mapping; Merge adjacent bounding boxes after mapping.

[0081] Furthermore, the automatic image sensitive information decryption device also includes an area filtering module, which is used for: Determine whether the area of ​​each sensitive information region is greater than a preset minimum area threshold. If so, retain the sensitive information and trigger the adaptive masking module; otherwise, filter the sensitive information.

[0082] Furthermore, the sensitive information determination module includes: A spatial coding unit is used to convert the bounding box coordinate information into a spatial embedding vector; The vector concatenation unit is used to concatenate the spatial embedding vector with the semantic vector of the text content to form a spatial-semantic joint encoding vector; The model inference unit is used to input the spatial-semantic joint encoding vector into the large language model.

[0083] Furthermore, the sensitive information determination module also includes a dual-channel verification unit, which is used for: When the judgment result of the large language model conflicts with the recognition result of the optical character recognition, a secondary verification process is triggered, and a third-party optical character recognition interface is called for cross-verification. The judgment results of the large language model, the recognition results of optical character recognition, and the verification results of the third-party optical character recognition interface are fused and decided based on a weighted voting mechanism to output the final sensitive information judgment result.

[0084] Furthermore, the adaptive coding module includes: Sensitivity level determination unit, used to determine the corresponding sensitivity level based on the type of sensitive information; The code-breaking strategy determination unit is used to match the corresponding code-breaking method and code-breaking intensity from a preset code-breaking strategy library according to the sensitivity level. The image decryption execution unit is used to perform image decryption processing on the corresponding area in the image according to the matched decryption method and decryption intensity; Edge blending unit is used to feather the edges of the censored area.

[0085] For specific limitations of the automatic image sensitive information mosaicing apparatus, reference may be made to the above limitations for the automatic image sensitive information mosaicing method, which will not be repeated herein. Each module in the above automatic image sensitive information mosaicing apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0086] The specific application process of the present invention is described below taking the insurance claim settlement scenario as an example.

[0087] When an insurance claim adjuster submits a claim case containing 30 photos in the system (including vehicle license photos, driver's license photos, vehicle panorama photos, vehicle damage part photos, etc.), the system automatically calls the automatic image sensitive information mosaicing method of the present invention: First, the system acquires 30 to-be-processed images.

[0088] Second, the system performs information type identification on each image. The vehicle license photo is identified as the "vehicle license" type, the driver's license photo is identified as the "driver's license" type, and the vehicle photo is identified as the "vehicle photo" type.

[0089] Then, the system performs OCR text recognition on each image, and extracts all text content and coordinate information in the image. For example, texts such as "Name: Zhang San" and "Certificate No.: 123456789012345678" are recognized in the vehicle license photo.

[0090] Next, the large language model performs sensitive information determination on the OCR recognition result. For the vehicle license photo, the large language model determines that "Zhang San" is a name (low sensitivity, confidence 0.92), and "123456789012345678" is a certificate number (high sensitivity, confidence 0.98). For the vehicle photo, the large language model determines that "Jing A88888" is a license plate number (medium sensitivity, confidence 0.95). The system performs coordinate mapping, deduplication and merging on the determined sensitive information, and filters out tiny regions with an area smaller than 50 square pixels.

[0091] Finally, the system performs adaptive mosaicing according to the sensitivity level. The certificate number is of high sensitivity, adopting a privacy shield mask combined with 15px Gaussian blur and edge feathering; the license plate number is of medium sensitivity, adopting dynamic mosaic combined with 10px Gaussian blur; the name is of low sensitivity, adopting lightweight mosaic combined with 5px Gaussian blur. All 30 photos are processed in about 96 seconds (3.2 seconds per photo × 30 photos, processed in parallel), and the mosaiced images are generated and output.

[0092] In comparison, traditional manual processing of 30 photos takes 90 minutes (3 minutes per photo x 30 photos). This invention increases processing efficiency by 56 times while achieving 100% mandatory process coverage. Based on processing an average of 500 major cases per day, it can save claims adjusters approximately 12,000 man-hours annually, equivalent to freeing up 8 full-time staff for higher-value work. The system's rate of missed sensitive information reports is expected to decrease by 80% compared to manual operations, fundamentally eliminating customer privacy leaks caused by human error.

[0093] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described automatic image sensitive information decryption method.

[0094] This computer device can be a server, and its internal structure diagram can be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a server-side method for automatically decrypting sensitive information in images.

[0095] This computer device can be a client, and its internal structure diagram can be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the client-side functions or steps of an automatic image sensitive information decryption method.

[0096] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method for automatically decrypting sensitive information in images.

[0097] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0098] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0099] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0100] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for automatically decrypting sensitive information in images, characterized in that, include: Get the image to be processed; The image to be processed is input into a pre-trained image semantic understanding model to identify the information type of the image to be processed. The information type is used to characterize the document category or scene category to which the image to be processed belongs; Determine whether the information type belongs to a preset whitelist type. If so, perform optical character recognition on the image to be processed to extract the text content in the image to be processed and the bounding box coordinate information of each text content in the image to be processed. Determine whether the confidence level of the optical character recognition result is greater than a preset threshold. If so, input the text content, the bounding box coordinate information, and the information type into a pre-trained large language model. Based on the sensitivity thresholds corresponding to different information types, determine whether each text content is sensitive information and output the type and confidence score of the sensitive information. The sensitivity level is determined according to the type of the sensitive information, and the corresponding masking method and masking intensity are determined according to the sensitivity level. The area in the image to be processed that corresponds to the bounding box coordinates of the sensitive information is masked according to the masking method and masking intensity to generate a masked image.

2. The automatic image sensitive information decoding method as described in claim 1, characterized in that, After determining whether the confidence level of the optical character recognition result is greater than a preset threshold, and before inputting the text content, the bounding box coordinate information, and the information type into the pre-trained large language model, the method further includes: standardizing and cleaning the text content recognized by the optical character, wherein the standardization and cleaning includes removing special characters and / or unifying the encoding format.

3. The method for automatically decrypting sensitive information in images as described in claim 1, characterized in that, Inputting the text content, the bounding box coordinates, and the information type into a pre-trained large language model includes: Convert the bounding box coordinate information into a spatial embedding vector; The spatial embedding vector is concatenated with the semantic vector of the text content to form a spatial-semantic joint encoding vector; The spatial-semantic joint encoding vector is input into the large language model.

4. The method for automatically decrypting sensitive information in images as described in claim 1, characterized in that, The method determines whether each piece of text is sensitive information based on the sensitivity threshold corresponding to different information types, and outputs the type and confidence score of the sensitive information; it also includes a dual-channel verification mechanism: When the judgment result conflicts with the optical character recognition result, a secondary verification process is triggered, and a third-party optical character recognition interface is called for cross-verification. The judgment result, the optical character recognition result, and the verification result of the third-party optical character recognition interface are fused and decided based on a weighted voting mechanism to output the final sensitive information judgment result.

5. The method for automatically decrypting sensitive information in images as described in claim 1, characterized in that, After outputting the type and confidence score of the sensitive information, and before determining the corresponding sensitivity level based on the type of sensitive information, the process also includes: Map the coordinates of the bounding box identified as sensitive information to the original coordinate system of the image to be processed; Remove duplicate bounding boxes that overlap after mapping; Merge adjacent bounding boxes after mapping.

6. The method for automatically decrypting sensitive information in images as described in claim 1, characterized in that, After outputting the type and confidence score of sensitive information, and before determining the corresponding sensitivity level based on the type of sensitive information, the method further includes determining whether the area of ​​each sensitive information region is greater than a preset minimum area threshold.

7. The method for automatically decrypting sensitive information in images as described in claim 1, characterized in that, The step of determining the corresponding sensitivity level based on the type of sensitive information, determining the corresponding masking method and masking intensity based on the sensitivity level, and masking the area in the image to be processed corresponding to the bounding box coordinates of the sensitive information according to the masking method and masking intensity further includes: When the sensitivity level is high sensitivity, a privacy shield mask is used in conjunction with the first Gaussian blur radius for masking, and the edges of the masked area are feathered. When the sensitivity level is medium sensitivity level, dynamic mosaic combined with a second Gaussian blur radius is used for masking. The cell size of the dynamic mosaic is determined according to a multiple of the text height within the sensitive information area. When the sensitivity level is low, lightweight mosaic combined with a third Gaussian blur radius is used for coding. The first Gaussian blur radius is greater than the second Gaussian blur radius, and the second Gaussian blur radius is greater than the third Gaussian blur radius.

8. An automatic image sensitive information decryption device, characterized in that, include: The image acquisition module is used to acquire images to be processed. The image semantic recognition module is used to input the image to be processed into a pre-trained image semantic understanding model to identify the information type of the image to be processed; the information type is used to characterize the document category or scene category to which the image to be processed belongs; The type verification module is used to determine whether the information type belongs to a preset whitelist type. If so, the information type is output to the optical character recognition module. An optical character recognition module is used to perform optical character recognition on the image to be processed, and extract the text content in the image to be processed and the bounding box coordinate information of each text content in the image to be processed. The OCR verification module is used to determine whether the confidence level of the optical character recognition result is greater than a preset threshold. If so, the optical character recognition result is output to the sensitive information determination module. The sensitive information determination module is used to input the text content, the bounding box coordinate information, and the information type into a pre-trained large language model, determine whether each text content is sensitive information based on the sensitivity threshold corresponding to different information types, and output the type and confidence score of the sensitive information. An adaptive decryption module is used to determine the corresponding sensitivity level based on the type of sensitive information, determine the corresponding decryption method and decryption intensity based on the sensitivity level, and decrypt the area in the image to be processed corresponding to the bounding box coordinates of the sensitive information according to the decryption method and decryption intensity to generate a decrypted image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the automatic image sensitive information decoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the automatic image sensitive information decoding method as described in any one of claims 1 to 7.

Images