Facial anonymization processing system and method for clinical ophthalmic diagnosis

Abstract

The invention discloses a face anonymization processing system for clinical ophthalmic diagnosis, and the system comprises a face identity conversion module which is used for carrying out the encryption and decryption of face features containing identity information in an input original image; the eye feature decoupling module is used for extracting eye disease features required by diagnosis; and the image restoration module is used for fusing the processed face and eye features to generate an encrypted image. Eye signs of the image generated through encryption are complete, various diagnosis based on the image is supported, the encrypted image is more real, and the encryption technology guarantees image privacy safety.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a facial anonymization processing system and method for clinical ophthalmology diagnosis. Background Art

[0002] In ophthalmic clinical diagnosis, facial images are widely used as an important diagnostic basis for remote screening and disease identification. However, these images inevitably contain sensitive patient identity information, which poses a serious privacy and security risk in the storage, sharing, and distribution of medical data. Acquiring large-scale, high-quality patient image collections is crucial for disease diagnosis, medical research, medical education, and the development of artificial intelligence-assisted diagnostic systems. However, these images, which contain in vivo biological materials and physical signs on the body surface, also increase the risk of personal privacy leakage. Facial images are particularly sensitive because they contain rich biometric features. On the other hand, many ophthalmic diseases manifest themselves through abnormal signs on the face or, in particular, in the eye area. Therefore, it is necessary to design facial privacy protection methods specifically for ophthalmology.

[0003] The existing identity anonymization methods for ophthalmological privacy protection have the following two main flaws:

[0004] First, ocular signs are not intentionally preserved, resulting in low diagnostic accuracy. Previous facial encryption technologies (such as G2Face) can protect patient privacy, but they treat all facial regions equally, altering the clinical attributes of facial images while removing privacy. This distortion reduces the usefulness of processed images for medical diagnosis. Although some methods, such as the Attribute-Aware Anonymization Network (A3GAN), claim to preserve facial attributes during face de-identification, they primarily target common attributes such as hairstyle and facial expression, leaving out disease characteristics, resulting in low diagnostic accuracy.

[0005] Second, insufficient preservation of ocular signs leads to poor generalization: Traditional privacy anonymization methods, such as facial cropping or blurring, may result in the loss of key medical information. For example, in ophthalmic images, cropping out the eye area or blurring sensitive areas may hide disease signs such as eye fractures. The recent digital mask technology covers the patient's face with an anonymized three-dimensional facial mesh controlled by facial landmarks, eyelid landmarks, and iris positions. Although this technology effectively removes identity features, it comes at the cost of losing clinical signs of diseases such as eye tumors. The above methods all fail to adequately preserve ocular signs, resulting in reduced diagnostic utility for some diseases. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to develop a facial encryption technology for ophthalmic patients, aiming to protect the privacy of ophthalmic patients while retaining the key information required for medical diagnosis.

[0007] To achieve the above objectives, the present invention proposes a facial anonymization processing system for clinical ophthalmological diagnosis, comprising:

[0008] The facial identity conversion module is used to encrypt and decrypt facial features containing identity information in the input original image;

[0009] Eye feature decoupling module, used to extract eye disease features required for diagnosis;

[0010] The image restoration module is used to fuse the processed facial and eye features to generate an encrypted image.

[0011] Optionally, the facial identity conversion module includes: a facial encryptor, which is used to confuse the identity of facial areas other than the eyes and input a password for subsequent decryption; a facial decryptor, which is used to restore the original facial information when a password consistent with the encryption process is input, and generate facial information of a virtual face if an incorrect password is entered.

[0012] Optionally, the facial encryptor includes:

[0013] A feature extractor, configured to convert an input raw image into a feature map, wherein the feature map can be flattened into a one-dimensional feature;

[0014] a plurality of conversion blocks for storing the one-dimensional features and the password, and converting the biometric identity information in the feature map using a self-attention mechanism;

[0015] The feature anti-flattening module is used to flatten one-dimensional features into feature maps.

[0016] Optionally, the eye feature decoupling module includes:

[0017] an eye sign detector for extracting features of the eye disease, including eyelid error, eyeball error, eye gaze error, eye structure fidelity, and eye color fidelity, while discarding features of unimportant parts;

[0018] The eye decoder is used to complete the discarded part and generate a natural eye image.

[0019] Optionally, the image restoration module includes:

[0020] Feature adapter, used to repair the uneven boundary transition caused by the splicing of facial and eye features;

[0021] Face decoder, used to decode features into face images.

[0022] In addition, the present invention also proposes a facial anonymization processing method for clinical ophthalmological diagnosis, comprising the following steps:

[0023] S1, performing identity obfuscation processing on the facial area other than the eyes in the input original image;

[0024] S2, extracting ocular disease features required for ocular diagnosis;

[0025] S3. Fusing the processed facial and eye features to generate an encrypted image.

[0026] Optionally, the S1 step includes: using a facial encryptor based on a self-attention module to process facial areas other than the eyes, and inputting a password for subsequent decryption.

[0027] Optionally, the S2 step includes: extracting eye feature maps using the first three stages of the deep residual network, each feature map corresponding to a block of the original image; classifying each feature block as normal or abnormal using a shared classifier, and retaining the maximum abnormality score as the overall result.

[0028] Optionally, the step of fusing the processed facial and eye features includes: using a feature adapter to repair an uneven boundary transition caused by splicing the facial and eye features; and using a face decoder to decode the features into a face image.

[0029] Optionally, it also includes:

[0030] Convert the input image into a feature map through a feature extractor;

[0031] Flatten the feature map into one-dimensional features;

[0032] The flattened features are passed to the transformation block along with the password, and the self-attention mechanism is used to transform the biometric identity information in the features.

[0033] Unflatten the output features back into a feature map.

[0034] The present invention has the following beneficial effects:

[0035] First, the ocular signs of the encrypted images are complete, supporting both doctors' clinical diagnoses and AI-based medical diagnoses without interfering with clinical results.

[0036] Second, the encrypted images have photorealistic quality, which enables ophthalmologists to seamlessly transfer their expertise and serve as educational materials for imparting clinical ophthalmology knowledge.

[0037] Third, security during data storage and distribution is improved.

[0038] Finally, the encryption technology is reversible, meaning that when a predefined password and decryptor application are authorized, the original facial image can be restored. This ensures that the imaging material itself is traceable, providing an additional chain of record for medical auditors. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the anonymization processing method of the present invention;

[0040] Figure 2 Schematic diagram of feature graph extraction of the anonymization processing method of the present invention;

[0041] Figure 3 Schematic diagram of an image processed by the anonymization method of the present invention;

[0042] Figure 4 and Figure 5 Comparison of the effects of different anonymization methods on images;

[0043] Figure 6 The figure is a simulation diagram of the anonymization processing method of the present invention in an application scenario. DETAILED DESCRIPTION

[0044] The facial anonymization processing system and method for clinical ophthalmological diagnosis proposed by the present invention are further described in detail below in conjunction with the accompanying drawings and specific embodiments. As described below, the advantages and features of the present invention will become clearer. It should be noted that the drawings are in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In order to make the purposes, features and advantages of the present invention more obvious and easy to understand, please refer to the drawings. It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the conditions for the implementation of the present invention, so they have no technical significance. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed by the present invention without affecting the efficacy and purpose that can be achieved by the present invention.

[0045] The present invention provides a facial anonymization system for clinical ophthalmological diagnosis, comprising a facial identity conversion module, an eye feature decoupling module, and an image restoration module. The facial identity conversion module encrypts and decrypts facial features containing identity information in the input raw image; the eye feature decoupling module extracts ocular disease features required for diagnosis; and the image restoration module fuses the processed facial and eye features to generate an encrypted image.

[0046] The facial identity conversion module includes: a facial encryptor, which is used to obfuscate the identity of the facial area other than the eyes and input a password for subsequent decryption; a facial decryptor, which is used to restore the original facial information when a password consistent with the encryption process is input, and generate facial information of a virtual face if an incorrect password is entered.

[0047] During the encryption process, the system employs a divide-and-conquer strategy to process the input image. First, for facial regions outside the eyes, a self-attention module-based face encryptor is used to obfuscate their identities. This requires a password string for subsequent decryption. Second, for the eye region, an eye sign detector is used to extract key features, including eyelid error, eyeball error, eye gaze error, eye structure fidelity, and eye color fidelity. The system discards features of less important areas as appropriate. Subsequently, an eye decoder is used to complete the discarded features, generating a natural-looking eye image. The encrypted facial region is then combined with the processed eye region features. Finally, a high-quality encrypted image is generated after passing through a feature adapter and a face decoder.

[0048] During the decryption process, the system reversibly restores the encrypted image based on the encryption password and authorized decryption software. First, the encrypted facial region is decrypted using a facial decryptor based on a self-attention module. Decryption is successful only if the password used during encryption is entered.

[0049] Optionally, the facial encryptor includes: a feature extractor for converting an input original image into a feature map, wherein the feature map can be flattened into a one-dimensional feature; a plurality of conversion blocks for storing the one-dimensional features and the password, and converting the biometric identity information in the feature map using a self-attention mechanism; and the facial decryptor includes a feature unflattening module for unflattening the one-dimensional features back into a feature map.

[0050] Specifically, the face decryptor aims to change the identity information in facial images. The process first converts the input image into a deep feature map through a feature extractor, which consists of a series of convolutional layers. Subsequently, the feature map is flattened into one-dimensional features. These flattened features are passed along with the private key (i.e., password) to six transformation blocks, which use a self-attention mechanism to transform the biometric identity information in the features, thereby effectively protecting privacy. Finally, the output features are flattened back into feature maps. The architecture of the face decryptor is basically the same as that of the face encryption, except that its input contains a learnable flag that instructs the network to operate in "recovery mode." When the same private key as in the privacy protection process is used, the original facial information can be recovered. Conversely, if an incorrect private key is used, such as a random key entered by an attacker, the network will generate facial information of a virtual face, thereby protecting the patient's privacy.

[0051] Optionally, the eye feature decoupling module includes: an eye sign detector, used to extract the eye disease features, including eyelid error, eyeball error, eye gaze error, eye structure fidelity and eye color fidelity, while discarding the features of non-important parts; an eye decoder, used to complete the discarded parts and generate a natural eye image.

[0052] Specifically, the primary goal of the proposed eye sign detector is to identify key features containing essential information necessary to support ophthalmic diagnosis. This approach ensures that information irrelevant to diagnosis is discarded, thereby improving resilience to biometric information leakage. Specifically, when an image of the eye region is input, the first three stages (Conv1, Res1, and Res2) of ResNet50 (a deep residual network based on residual networks) are used to extract feature maps. In this process, each feature corresponds to a patch of the original image. A shared classifier (consisting of a linear layer) is used to classify each patch as normal or abnormal. The maximum anomaly score is then used as the overall result for the entire image. The eye sign detector is trained using coarse-grained labels (i.e., indicators of health or disease). This approach aims to reduce the manual labor of annotating eye images. The detector's training dataset consists of 40,000 real outpatient images labeled "disease" and 40,000 images labeled "healthy" from a public dataset. Despite being trained using only image-level annotations, the detector is able to effectively identify regions relevant to classification decisions in a weakly supervised manner. When using an eye sign detector to detect key eye regions, we rank image patches by classification scores and retain those with a similarity exceeding a certain threshold. Experiments on a cohort of nearly 10,000 people have shown that retaining the top 50% of scoring patches achieves the best balance between sign preservation and anonymization.

[0053] Optionally, the image restoration module includes: a feature adapter for repairing the uneven boundary transition caused by splicing facial and eye features; and a face decoder for decoding the features into a face image.

[0054] The feature adapter, consisting of feature flattening, two self-attention modules, and feature spatialization, repairs the uneven boundary transitions caused by the splicing of facial and eye features. The face decoder, used to decode features into facial images, consists of three alternating residual and upsampling layers: four residual layers, an upsampling layer, two residual layers, an upsampling layer, and one residual layer, followed by an upsampling layer.

[0055] In addition, based on the system, the present invention also proposes a facial anonymization processing method for clinical ophthalmology diagnosis. The process can be found in Figure 1 and Figure 2 , including the following steps:

[0056] S1, performing identity obfuscation processing on the facial area other than the eyes in the input original image;

[0057] S2, extracting ocular disease features required for ocular diagnosis;

[0058] S3. Fusing the processed facial and eye features to generate an encrypted image.

[0059] Optionally, the S1 step includes: using a facial encryptor based on a self-attention module to process facial areas other than the eyes, and inputting a password for subsequent decryption.

[0060] Optionally, the S2 step includes: extracting eye feature maps using the first three stages of the deep residual network, each feature map corresponding to a block of the original image; classifying each feature block as normal or abnormal using a shared classifier, and retaining the maximum abnormality score as the overall result.

[0061] Optionally, the step of fusing the processed facial and eye features includes: using a feature adapter to repair an uneven boundary transition caused by splicing the facial and eye features; and using a face decoder to decode the features into a face image.

[0062] Optionally, it also includes:

[0063] Convert the input image into a feature map through a feature extractor;

[0064] Flatten the feature map into one-dimensional features;

[0065] The flattened features are passed to the transformation block along with the password, and the self-attention mechanism is used to transform the biometric identity information in the features.

[0066] Unflatten the output features back into a feature map.

[0067] To validate the performance of this system, we conducted quantitative evaluation and comparative experiments. The results demonstrate that the system, while protecting privacy, reconstructs more accurate eye features and outperforms other privacy-preserving technologies, including the ophthalmology-specific Digital Mask and the general-purpose G2Face, across all evaluation metrics. We also included the open-source face-swapping software Face Swap and the generative AI-based AIGC technology for comparison.

[0068] Regarding the preservation of ocular disease features, we included four common ophthalmic signs: conjunctival abnormalities, eyelid abnormalities, lacrimal abnormalities, and iris abnormalities. Images processed by our system showed significant consistency with the original images on all three validation sets (Cohen's k ≥ 0.81). Taking the conjunctival abnormality features of the internal SNPH dataset as an example, the k value of the method used in the present invention is 0.9223 (95% CI: 0.8691-0.9755), which is much higher than the values of the mosaic technology (k = 0.3299, 95% CI: 0.2079-0.4518), the AIGC technology (k = 0.2399, 95% CI: 0.1260-0.3538), the Face Swap technology (k = 0.5208, 95% CI: 0.4176-0.6239), the Digital Mask technology (k = 0.0955, 95% CI: -0.0250-0.2162) and the G2Face technology (k = 0.4489, 95% CI: 0.3406-0.5571). Our method significantly outperforms the state-of-the-art deep face privacy protection method G2Face for all ophthalmic features in all validation sets.

[0069] Regarding the diagnosis of eye diseases, we used Cohen's Kappa (k) to evaluate the effectiveness of different privacy-preserving methods for image processing in medical diagnosis. We diagnosed 11 eye diseases, including strabismus, ptosis, BCC (basal cell carcinoma), SCC (squamous cell carcinoma), CM (conjunctival melanoma), CL (corneal leukoma), TED (thyroid eye disease), EoE (entropy or ectropion), MN (melanocytic nevus), and DEN (schizontopharyngeal nevus). The results showed that our method achieved k ≥ 0.81 for all eye diseases on the three validation sets, indicating that the diagnostic results of the protected images were highly consistent with the original images. For example, on the SNPH dataset, our method achieved k values of 0.9594 (95% CI: 0.9033-1.0154) for BCC, 0.9046 (95% CI: 0.7735-1.0357) for CM, 0.8732 (95% CI: 0.7318-1.0147) for CL, and 1.0 (95% CI: 1.0-1.0) for SCC. Similar high scores were achieved for strabismus, ptosis, and thyroid eye disease. Given that k values exceeding 0.81 indicate significant consistency, our method is suitable for remote clinical diagnosis applications without significantly impacting clinical diagnostic outcomes. In contrast, Mosaic and AIGC performed poorly on all diseases due to their removal of key disease features during processing. Digital Mask performed well for diseases such as thyroid eye disease and ptosis, which can be assessed using handcrafted features (e.g., eyelid shape), but performed poorly for other diseases, such as BCC and CM. On the ECXHCSU dataset, their k values were 0.0712 (95% CI: -0.0986-0.2409) and 0.0993 (95% CI: -0.1400-0.3386), respectively, which are much lower than our method, whose k values were 0.9692 (95% CI: 0.9091-1.0294) and 1.0 (95% CI: 1.0-1.0) in the above two diseases, respectively. Face Swap and G2Face outperformed Mosaic, AIGC, and Digital Mask, but did not achieve k>0.81 in any ophthalmic disease, indicating that they are not suitable for clinical applications.

[0070] Furthermore, we compared the ROC curves of our method with those of other methods and the AUROC for each disease. Our method demonstrated performance that was more similar to the ROC curves generated from the original image data. Our method performed on par with or better than other methods in terms of AUROC for each disease. For example, for the ViT diagnostic model on the SNPH dataset, our method achieved an AUROC of 0.950 for BCC disease, compared to 0.908 for Face Swap and 0.904 for G2Face. These results highlight the high compatibility of our method with medical AI models.

[0071] Finally, we also evaluated the effect of this method in face recognition applications. Experimental results show that compared with other privacy protection methods (Mosaic, Digital Mask, G2Face), this method maintains high ID recovery performance and structural fidelity while protecting privacy ( Figure 3 、 Figure 4 and Figure 5 Finally, we tested the retrieval success rate of protected facial images in the database. This technology protected over 96% of patient identities, surpassing previous methods in security.

[0072] Based on deep learning technology, this invention proposes an advanced eye privacy protection system by learning from a large number of real clinic photos. The technology uses facial identity conversion and feature decoupling technology to remove the identity information of the patient's facial image while retaining the features related to eye diseases. First, the ocular landmarks of the images of this system are complete, which can well support human doctors and artificial intelligence-based medical diagnosis without interfering with clinical results. Second, the images of this system have photorealism, allowing ophthalmologists to seamlessly impart their expertise and be used as educational materials for imparting clinical ophthalmology knowledge. Third, the security of the data storage and distribution process is improved, where only encrypted data is saved or transmitted remotely. Even if an attacker intercepts or steals the data during transmission, the patient's privacy is still strictly protected. Finally, the encrypted image is reversible, that is, when a predefined password and decryptor application are authorized, the original facial image can be restored. This ensures that the diagnostic material itself (here, the patient's electronic photo) is traceable, providing medical auditors with more record chains (see Figure 6 ).

[0073] Although the present invention has been described in detail through the above preferred embodiments, it should be understood that the above description is not intended to limit the present invention. After reading the above description, various modifications and substitutions of the present invention will become apparent to those skilled in the art. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A facial anonymization processing system for clinical ophthalmological diagnosis, characterized in that: include: The facial identity conversion module is used to encrypt and decrypt facial features containing identity information in the input original image; Eye feature decoupling module, used to extract eye disease features required for diagnosis; The image restoration module is used to fuse the processed facial and eye features to generate an encrypted image.

2. The facial anonymization processing system according to claim 1, wherein: The facial identity conversion module includes: A facial encryptor, used to obfuscate facial areas other than the eyes, and a password is entered for subsequent decryption; The facial decryptor is used to restore the original facial information when a password consistent with the encryption process is entered. If an incorrect password is entered, the facial information of a virtual face is generated.

3. The facial anonymization processing system according to claim 2, wherein: The facial encryptor includes: A feature extractor, configured to convert an input raw image into a feature map, wherein the feature map can be flattened into a one-dimensional feature; a plurality of conversion blocks for storing the one-dimensional features and the password, and converting the biometric identity information in the feature map using a self-attention mechanism; The anti-flattening module is used to flatten the one-dimensional features into feature maps.

4. The facial anonymization processing system according to claim 1, wherein: The eye feature decoupling module includes: an eye sign detector for extracting features of the eye disease, including eyelid error, eyeball error, eye gaze error, eye structure fidelity, and eye color fidelity, while discarding features of unimportant parts; The eye decoder is used to complete the discarded part and generate a natural eye image.

5. The facial anonymization processing system according to claim 1, wherein: The image restoration module includes: Feature adapter, used to repair the uneven boundary transition caused by the splicing of facial and eye features; Face decoder, used to decode features into face images.

6. A facial anonymization processing method for clinical ophthalmological diagnosis, characterized in that: The following steps are involved: S1, performing identity obfuscation processing on the facial area other than the eyes in the input original image; S2, extracting ocular disease features required for ocular diagnosis; S3. Fusing the processed facial and eye features to generate an encrypted image.

7. The facial anonymization processing method according to claim 6, characterized in that: The S1 step includes: using a facial encryptor based on a self-attention module to process facial areas other than the eyes, and inputting a password for subsequent decryption.

8. The facial anonymization processing method according to claim 6, wherein: The S2 step includes: extracting eye feature maps using the first three stages of the deep residual network, each feature map corresponding to a block of the original image; using a shared classifier to classify each feature block as normal or abnormal, and retaining the maximum abnormality score as the overall result.

9. The facial anonymization processing method according to claim 6, wherein: The step of fusing the processed facial and eye features includes: using a feature adapter to repair the uneven boundary transition caused by splicing the facial and eye features; and using a face decoder to decode the features into a face image.

10. The facial anonymization processing method according to claim 6, characterized in that: Also includes: Convert the input image into a feature map through a feature extractor; Flatten the feature map into one-dimensional features; The flattened features are passed to the transformation block along with the password, and the self-attention mechanism is used to transform the biometric identity information in the features. Unflatten the output features back into a feature map.

Images