Methods and devices for protecting medical image privacy data based on reversible adversarial patches

Abstract

This invention provides a method and apparatus for protecting medical image privacy data based on reversible adversarial patches, relating to the field of medical and health technology. The method accurately locates lesion regions based on Canny edges and diffusion masks, applies adversarial perturbations only to key regions to avoid damage to healthy tissues; it utilizes gradient-sensitive weighted SW and cross-model gradient alignment CMGA to jointly optimize perturbations in the frequency-spatial domain, ensuring that multiple deep learning architectures are simultaneously misled; the perturbation-quantized binary bitstream is embedded into the low-variance region of the image through reversible data hiding RDH-GI, and the licensing end can extract and reverse the perturbation without loss, achieving high-fidelity restoration of the original image.

Description

Technical Field

[0001] This invention relates to the field of medical and health technology, and specifically to a method and apparatus for protecting medical image privacy data based on reversible adversarial patches. Background Technology

[0002] Medical imaging data plays an irreplaceable role in assisting disease diagnosis and advancing medical research. However, with the widespread application of digital technology, this imaging data, which carries a large amount of patient privacy information, is facing increasingly prominent security risks. Particularly during data transmission, storage, and analysis, there is a risk of unauthorized access and use by artificial intelligence or humans, threatening patient privacy. Current common data protection methods, such as encryption and anonymization, have significant limitations. Some methods damage image quality, reducing the visibility of critical diagnostic information and affecting the accuracy of clinical judgment; others have limited defense capabilities, struggling to cope with evolving malicious attack methods and failing to provide reliable protection for sensitive medical data. This dilemma of balancing protection and usability severely restricts the sharing and application of medical data in a secure environment. Therefore, a new solution is needed that can effectively prevent unauthorized use while maximizing the diagnostic value of images. This solution must prevent unauthorized analysis and use by artificial intelligence or humans while ensuring that its clinical usability remains unaffected, allowing doctors to make accurate diagnoses based on protected images. For authorized users, the original image details should be restored without loss, meeting the stringent requirements of medical diagnosis for image authenticity and accuracy, and achieving a balance between security and clinical applicability.

[0003] Traditional adversarial attack methods include the Fast Gradient Sign Method (FGSM), which marks the beginning of adversarial example generation. Building upon this, researchers proposed the Basic Iterative Method (BIM), which generates stronger adversarial examples by applying small perturbations multiple times. Additionally, there are schemes incorporating Projected Gradient Descent (PGD) attacks. PGD iteratively updates the adversarial perturbation and projects it back into a predefined norm sphere, ensuring the perturbation remains within acceptable limits. These methods have not only successfully misled deep learning models but also paved the way for more advanced adversarial attacks, including attacks that improve transferability across various model architectures, from CNNs to ViTs.

[0004] Based on this, researchers have developed various methods to exploit shared vulnerabilities across different models, thereby increasing attack success rates and enhancing cross-model consistency. One effective strategy is GSW (gradient sensitivity weighting), which dynamically adjusts the perturbation strength based on the vulnerability of each model to ensure consistent attack direction. Techniques such as CMGA (cross-model gradient alignment) mitigate the differences in responses to adversarial examples by synchronizing perturbations across multiple models. These ensemble-based methods demonstrate the potential to achieve cross-model adversarial effects and are crucial for ensuring the transferability of adversarial examples across diverse architectures.

[0005] However, existing reversible adversarial attack methods still have some shortcomings. First, most reversible adversarial attack methods generate global perturbations across the entire image, significantly increasing the storage requirements for reversible data hiding. Since reversible data hiding requires embedding all perturbation information into the image, global perturbations impose a heavy storage burden, thus limiting the overall strength and effectiveness of the attack. This problem is particularly prominent in privacy-sensitive scenarios such as healthcare, where stronger perturbations are often required to provide adequate protection. Second, many reversible adversarial attack methods still follow traditional approaches, pursuing minimal perceptibility and attempting to generate perturbations that are imperceptible to the human eye, similar to traditional adversarial attacks. However, in the context of privacy protection, the core objective should be to prevent unauthorized access to critical information, not merely to deceive the model in a way that is imperceptible to the human eye. Overly pursuing imperceptibility may weaken the defensive capabilities of adversarial attacks in data protection scenarios. For medical images or other sensitive data, concentrating stronger, more visually significant perturbations on key areas such as lesions can effectively block unauthorized interpretation while ensuring privacy and security. This misalignment with imperceptible targets limits the potential of reversible adversarial attacks in achieving robust privacy protection.

[0006] In view of the above, this application is hereby submitted. Summary of the Invention

[0007] This invention provides a method and apparatus for protecting medical image privacy data based on reversible adversarial patches, which can at least partially improve the above-mentioned problems.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A method for protecting medical image privacy data based on reversible adversarial patches, comprising:

[0010] The input image is acquired, and the Canny edge detection algorithm is used to locate the lesion area.

[0011] Based on the lesion region, the weight values ​​of the deep learning model are dynamically calculated using the GSW algorithm. The CMGA algorithm is introduced to jointly optimize the perturbation in the frequency and spatial domains, minimizing the perceptual difference between the original image and the adversarial sample, and obtaining the final aggregated similarity.

[0012] Based on the final aggregated similarity, the reversible adversarial perturbation is calculated, embedded into the adversarial image, and the embedded binary bit stream is extracted from the adversarial image using RDH-GI technology. The perturbation is then reversed to obtain an image restored to its original state.

[0013] The present invention also provides a medical image privacy data protection device based on reversible adversarial patching, comprising:

[0014] The lesion area localization unit is used to acquire the input image, and uses the Canny edge detection algorithm to localize the input image and identify the lesion area;

[0015] The aggregation similarity calculation unit is used to dynamically calculate the weight values ​​of the deep learning model based on the lesion region using the GSW algorithm. The CMGA algorithm is introduced to jointly optimize the perturbation in the frequency domain and spatial domain, minimizing the perceptual difference between the original image and the adversarial example, and obtaining the final aggregation similarity.

[0016] The perturbation embedding and restoration unit is used to calculate reversible adversarial perturbations based on the final aggregated similarity, embed the reversible adversarial perturbations into the adversarial image, and use RDH-GI technology to extract the embedded binary bit stream from the adversarial image, perform reverse perturbation removal, and obtain an image restored to the original state.

[0017] In summary, this invention proposes a reversible adversarial framework, SEMPRA, which is a method to improve data protection for authorized users by protecting the privacy of sensitive parts of medical images through reversible adversarial patching techniques. SEMPRA combines RDH for reversible perturbation, uses Canny edge detection to accurately locate lesion regions, and employs GSW and CMGA to improve perturbation efficiency, thereby enhancing attack performance and transferability. Experiments on various medical image datasets demonstrate that SEMPRA effectively prevents unauthorized model classification while maintaining the reversibility of authorized access, providing a reliable solution for medical image privacy protection.

[0018] Specifically, this method addresses the pain point of patient privacy leaks caused by unauthorized AI or human analysis of medical images outside hospital boundaries. It proposes for the first time a privacy protection paradigm of "precise lesion perturbation + reversible, traceless restoration." SEMPRA introduces adversarial perturbations targeting key areas of medical images to ensure data confidentiality while also guaranteeing its usability in legal scenarios. The framework consists of three main modules: lesion area identification, adversarial perturbation generation (using advanced gradient-based technology), and embedding and restoration of reversible perturbations. This method locates lesions by coupling Canny edge detection with diffusion masking, concentrating adversarial noise only in critical diagnostic areas with zero interference to healthy tissue, thus solving the problems of visual distortion and hidden capacity expansion caused by traditional global perturbations. It innovatively embeds gradient-sensitive weighted SW and cross-model gradient alignment CMGA into a unified optimization process, collaboratively generating efficient perturbations consistent across architectures in the frequency and spatial domains, achieving an attack success rate of over 99% against heterogeneous models such as CNN and ViT, overcoming the bottlenecks of insufficient reversible adversarial strength and poor transferability in existing methods. Furthermore, the perturbation quantized bitstream is implanted into the low-variance channel of the image using RDH-GI reversible data hiding technology. Authorized users can extract and reverse the perturbation with a single click, restoring the image to pixel-level and diagnostic-level consistency with the original data, meeting the requirements for remote consultation, research sharing, and regulatory auditing. Experiments covering multiple modalities of medical datasets demonstrate that the protected image is completely ineffective against unauthorized models, and authorized doctors find no statistical difference between the protected image and the original image in their visual interpretation. Therefore, it simultaneously meets clinical implementation standards in four dimensions: "privacy invisible, diagnostic usable, storage not increasing, and compliance auditable," providing an engineeringable ultimate solution for the secure flow of medical data. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the medical image privacy data protection method based on reversible adversarial patching provided in the first embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram outlining the SEMPRA framework process provided by the present invention;

[0021] Figure 3 This is a schematic diagram illustrating the effect of different sensitivity settings on mask area selection provided by the present invention;

[0022] Figure 4 This is a schematic diagram illustrating an example of the GSW algorithm provided by this invention;

[0023] Figure 5 This is a schematic diagram illustrating an example of the CMGA algorithm provided by this invention;

[0024] Figure 6 This is a comparative diagram of different ASR methods provided by the present invention;

[0025] Figure 7This is a schematic diagram comparing the ASR of different methods provided by this invention on different datasets and test models;

[0026] Figure 8 This is a schematic diagram of the original image, adversarial attack image, and restored image obtained using the SEMPRA framework provided by the present invention;

[0027] Figure 9 This is a schematic diagram of a medical image privacy data protection device based on reversible adversarial patches provided in the second embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0029] refer to Figures 1 to 3 , Figures 6 to 7 As shown, the first embodiment of the present invention discloses a method for protecting medical image privacy data based on reversible adversarial patches, which can be executed by a medical image privacy data protection device based on reversible adversarial patches (hereinafter referred to as the protection device), specifically, by one or more processors within the protection device, to implement the following method:

[0030] S1. Acquire the input image, and use the Canny edge detection algorithm to locate the input image and identify the lesion area;

[0031] Specifically, step S1 further includes: acquiring the input image. Using Gaussian kernel For the input image Gaussian smoothing is applied to reduce noise; the formula is as follows: ,in, For convolution operations, The scale is that of the Gaussian kernel. The x-coordinate data of the input image, The input image contains the ordinate data.

[0032] Gradient calculation is performed on the input image after Gaussian smoothing, in the horizontal direction. gradient and vertical direction The gradient is calculated using the Sobel operator, whose formula is: , , and All are Sobel kernels, where the gradient magnitude G is related to the gradient direction. The calculation formulas are respectively , ;

[0033] Based on nonmaximum suppression, only local maxima are retained along the gradient direction, and the formula is as follows: ;

[0034] Based on high threshold and low threshold For local maxima The process involves identifying strong and weak edges, using the following formula: Then, the strong edges and weak edges are connected to obtain the complete edges;

[0035] Morphological operations are used to connect intact edges into closed regions, resulting in the final lesion area. This region consists of all pixels (x, y). The edge of the lesion area can be divided into i smaller edges. For the first small edge, For the i-th small edge, , Let be the i-th pixel.

[0036] In this embodiment, in the practical application of medical image privacy protection, the core issue that needs to be addressed first is how to accurately locate the "protectable" areas within the entire image. Applying a uniform perturbation to the entire image not only wastes valuable hidden space on a large amount of irrelevant healthy tissue, but also blurs diagnostic information such as blood vessels and textures due to noise diffusion, directly reducing the clinical readability of the image. Therefore, in step S1, a "precise lesion localization" mechanism based on Canny edge detection is introduced: after the system acquires an input image of DICOM or arbitrary depth, it empirically selects... A Gaussian kernel of 1.5 is used to smooth the entire image. This convolution kernel suppresses quantum noise commonly found in CT and MRI while preserving the sharpness of lesion edges. Subsequently, the Sobel operator is used to calculate the gradients in the horizontal and vertical directions, with the gradient magnitude G related to the direction. Together, they characterize each potential edge. The non-maximum suppression step retains only local peak pixels along the gradient direction, transforming coarse edges into a single-pixel-width skeleton, reducing subsequent false edge interference and making lesion boundaries clearer. The dual-threshold strategy further divides edges into "strong" and "weak" levels: pixels above the high threshold are marked as reliable edges, while pixels between are considered candidates, and can be included in the final contour as long as they are connected to strong edges. This mechanism effectively compensates for edge breaks caused by noise or local gray-level unevenness, ensuring that low-contrast lesions such as ground-glass nodules and microbleeds can also be completely wrapped by the closed contour. Finally, dilation operations expand the single-pixel edges into a planar mask, obtaining a "lesion region" that closely matches the actual lesion.

[0037] In short, lesion region identification is based on the Canny edge detection algorithm, which precisely applies adversarial perturbations to the most critical lesion regions. This strategy maximizes the effectiveness of adversarial attacks while minimizing the risk of detection. By focusing on the most vulnerable regions, unauthorized models find it more difficult to make accurate classifications or diagnoses. In addition, considering the potential diffusion of adjacent lesions in medical images, we designed a diffusion-based mask detection scheme that can cover lesion regions more broadly, thereby protecting privacy data more comprehensively. The key steps of lesion region identification are as follows: (1) Gaussian smoothing: The input image is smoothed using a Gaussian kernel to suppress noise. (2) Gradient calculation: The gradient of the image in the horizontal and vertical directions is calculated using the Sobel operator. (3) Non-maximum suppression: Only local maxima are retained along the gradient direction, and the remaining non-maximum responses are suppressed. (4) Dual thresholding and edge connection: Strong and weak edges are identified using high and low thresholds respectively, and complete edges are formed through connection operations. (5) Lesion region extraction: The edges are connected into closed regions through morphological operations such as dilation, thereby obtaining the final lesion region. As the image size increases, the sensitivity of the edge detection-based masking strategy significantly improves, enabling it to gradually delineate more complete lesion boundaries. When multiple lesions exist in an image, this method can effectively detect and segment each individual lesion, thereby improving the overall accuracy and completeness of the segmentation. This fully demonstrates that the method can adapt to different lesion morphologies and achieve robust recognition and accurate segmentation at various sensitivity thresholds.

[0038] Please see Figure 4 , Figure 5 S2, based on the lesion area, the weight values ​​of the deep learning model are dynamically calculated using the GSW algorithm, and the CMGA algorithm is introduced to jointly optimize the perturbation in the frequency domain and spatial domain, so as to minimize the perceptual difference between the original image and the adversarial sample and obtain the final aggregated similarity.

[0039] Specifically, step S2 further includes: for each deep learning model in the ensemble Calculate its loss function Regarding adversarial examples gradient ,in, Let x' be the current adversarial sample at the t-th iteration; initially (t=0) it is set to a clean image x', and it is continuously updated iteratively throughout the attack.

[0040] Based on these gradients, for each deep learning model Generate corresponding temporary adversarial examples , The disturbance amplitude control parameter is usually set to a minimum value (e.g., 1×10). -4 ), used to generate temporary adversarial samples with only slight perturbations;

[0041] Calculate the loss for each adversarial example. This is used to evaluate the model's performance under the perturbation; the loss value reflects the deep learning model's performance. The higher the loss, the more sensitive the model is to the current disturbance;

[0042] After generating and evaluating adversarial examples for each model, the next step is to accumulate weights based on the performance of each model on adversarial examples generated by other models, in order to measure the loss proportion of each model under different perturbations, thereby dynamically adjusting its contribution to the overall perturbation.

[0043] For each deep learning model in the ensemble traverse all other models , and calculate the model In the model Loss on generated adversarial examples ;

[0044] The model is calculated by summing the weighted loss ratios of all other models. weight N is the number of deep learning models. The modulation coefficient is used to control the impact of the loss ratio on the weights. This formula allows models that perform poorly (i.e., have higher losses) when facing adversarial examples to receive greater weights, thereby directing more perturbation intensity to these models in subsequent iterations.

[0045] The CMGA algorithm is introduced to align gradients across different models, ensuring that perturbations remain consistently effective across all models, thus avoiding weakened adversarial effects due to gradient conflicts. Cosine similarity of model gradients is calculated, quantifying the directional consistency between any two model gradients based on a similarity metric. Higher cosine similarity indicates closer proximity between the two gradient directions, meaning the two models respond similarly to perturbations along that direction. This systematic approach aims to capture pairwise similarities between all models in the ensemble.

[0046] Construct a similarity graph, the similarity matrix of which is defined as a A matrix, where each element of the matrix represents the model. With model Cosine similarity between gradients;

[0047] Among them, for all satisfying and The model is subjected to cosine similarity calculation to obtain the corresponding matrix elements. The calculation formula is as follows: i and j are model metrics, c is the number of channels in the gradient tensor, h is the height of the gradient tensor, and w is the width dimension of the gradient tensor. To avoid redundant calculations, only those satisfying the given conditions are calculated. The similarity between the model pairs is calculated by directly assigning values ​​to the remaining matrix elements, since cosine similarity is symmetric. The formula is as follows: ;

[0048] Calculate each model The average similarity score, representing its overall alignment with the remaining models in the ensemble, is calculated using the following formula: , with quantification model The degree of consistency with all other models in the ensemble in the gradient direction;

[0049] To obtain a comprehensive measure of the overall alignment of the entire ensemble along the gradient direction, the average similarity scores of all models are aggregated to obtain the final aggregated similarity. The final result It represents the overall similarity of gradients among integrated models and can be used for regularization and optimization of the distribution of adversarial perturbations.

[0050] Traditional ensemble attacks often fall into a "one wins, the other loses" dilemma: the gradient direction effective for model A may be negatively correlated with the gradient of model B, resulting in the perturbations canceling each other out during iteration, ultimately outputting a compromise sample that "cannot fool anyone". In this embodiment, the adversarial perturbation generation introduces the GSW and CMGA algorithms to optimize the model ensemble components of SEMPRA. With the help of GSW, the weights of each model are dynamically adjusted according to the sensitivity of the gradient of each model, thereby generating perturbations that are robust to multiple network architectures. The core step of GSW is dynamic weight calculation, which generates temporary adversarial samples with small perturbations based on the gradient information of each model, and calculates the loss of these temporary samples on each model. The next step is to accumulate weights based on the performance of each model on the temporary samples generated by other models, and then calculate the weight of the model by summing the weighted loss ratio of all other models. With the help of CMGA, these perturbations are jointly optimized in the frequency domain and spatial domain to minimize the perceptual difference between the original image and the adversarial sample. The implementation steps of CMGA are as follows: (1) Calculate the cosine similarity of the model gradient. (2) Construct a similarity map to systematically capture the pairwise similarity between all models in the set. (3) Calculate the average similarity of each model and calculate the average similarity score for each model to represent its overall consistency with the rest of the set. (4) Aggregate the similarity scores.

[0051] Specifically, GSW and CMGA are integrated to optimize the model ensemble component of SEMPRA. The GSW algorithm is introduced to optimize the generation of adversarial perturbations in a multi-model ensemble environment. GSW dynamically adjusts the gradient weights of each model in the ensemble to ensure that the generated perturbations are not only effective and difficult to detect for multiple models, but also significantly improve the overall attack performance. Dynamic weight calculation is the core step of the GSW algorithm. Temporary adversarial examples containing only slight perturbations are generated using the gradient information of each model, and the loss of these temporary examples on each model is calculated. This process is used to evaluate the sensitivity of each model to the current perturbation. First, with the current lesion mask as the scope, the gradient of the loss with respect to the adversarial example is calculated in parallel for the N deep learning models in the ensemble; then, the GSW dynamic weighting stage is entered: each model generates temporary adversarial examples containing only preset amplitudes based on its own gradient, and immediately feeds them back to all other models for forward inference to obtain the cross-model loss matrix. This dynamic mechanism enables the perturbation distribution to be automatically adjusted in a short time regardless of any new architecture in the ensemble, ensuring an improvement in the overall attack success rate without the need for manual retuning.

[0052] After GSW completes its operations, the CMGA algorithm further performs consistency correction on the gradient direction. The system calculates the cosine similarity between all pairs of models, constructing a symmetric... Similarity maps are used to quickly identify outlier gradients in semantically rich regions like lesion areas, where different models often focus on similar key textures. This implementation calculates cosine values ​​point-by-point across the channel, height, and width dimensions, preserving spatial structure information and resulting in a more nuanced similarity evaluation. Subsequently, an average similarity score is calculated for each model, and all scores are aggregated into a final aggregated similarity. In actual iterations, the aggregated similarity is directly added to the total loss as a regularization term, applying a soft constraint to the gradient direction. This significantly reduces the perceptual difference between the original image and the adversarial example without sacrificing attack strength.

[0053] Please see Figure 8 S3. Based on the final aggregated similarity, calculate the reversible adversarial perturbation, embed the reversible adversarial perturbation into the adversarial image, and use RDH-GI technology to extract the embedded binary bit stream from the adversarial image, perform reverse perturbation removal, and obtain the image restored to the original state.

[0054] Specifically, step S3 further includes: embedding information using the R and B channels of the color image, while adaptively adjusting the pixel values ​​of the G channel to maintain grayscale consistency.

[0055] Based on the final aggregated similarity, reversible adversarial perturbations are generated using deep learning models. and will reversibly counteract disturbances. Quantization is performed to convert it into a binary bit stream;

[0056] Based on prediction error analysis, low-variance regions in the adversarial image are located, and the bitstream is embedded into them to achieve seamless fusion.

[0057] The embedded binary bitstream is extracted from the adversarial image using RDH-GI technology. This extraction process decodes the bitstream by checking pixel changes that conform to grayscale invariance constraints to ensure accurate recovery of perturbation parameters.

[0058] The binary bitstream is decoded to reconstruct the perturbation matrix, which records the adversarial modifications initially applied to the image. By applying these perturbations in reverse to counteract the effects, the image is restored to its original, unaltered state.

[0059] In this embodiment, the embedding and recovery of reversible perturbations integrates RDH-GI technology. Information is embedded using the R (red) and B (blue) channels of the color image, while grayscale consistency is maintained through adaptive adjustment of the G (green) channel pixel values. Adversarial perturbations are generated through deep model integration. These perturbations are carefully optimized to both mislead unauthorized classifiers and minimize visual distortion. Subsequently, the perturbations are quantized and converted into a compact and efficient binary bitstream. Low-variance regions in the medical image are located through prediction error analysis, and the bitstream is embedded within them for seamless fusion. The recovery phase utilizes RDH-GI technology to extract the embedded binary bitstream from the adversarial image. The extraction process decodes the bitstream by examining pixel changes that conform to grayscale invariance constraints, thereby accurately recovering the perturbation parameters. After extraction, the binary bitstream is decoded to reconstruct the perturbation matrix, which records the adversarial modifications initially applied to the image. By inversely eliminating these perturbations, the framework can completely counteract their effects, restoring the image to its original, unaltered state.

[0060] Specifically, adversarial perturbations are generated using deep learning model ensembles. These perturbations are finely optimized to mislead unauthorized classifiers with minimal visual alteration. The resulting two-dimensional perturbation matrix is ​​linearly quantized and converted into a binary bitstream for a compact and efficient representation. Subsequently, low-variance regions in the image are located using prediction error analysis, and the bitstream is embedded into them for seamless integration with the image. In the restoration phase, the embedded perturbation data is extracted, and the original medical image is reconstructed with high fidelity. The embedded binary bitstream is extracted from the adversarial image using RDH-GI technology, and then decoded to reconstruct the perturbation matrix. By inversely applying these perturbations, the framework can counteract their effects, restoring the image to its original, unaltered state.

[0061] In summary, the SEMPRA approach presented in this study represents a significant advancement in medical image privacy protection, with its adversarial design focusing on lesion-centric diseases. The combination of Canny edge detection and a diffusion-based masking mechanism enables precise perturbation localization, strictly limiting adversarial noise to critical diagnostic regions while minimizing interference with healthy anatomical structures. The introduction of GSW and CMGA algorithms overcomes key limitations of traditional ensemble attacks. By dynamically aligning gradients across heterogeneous architectures, SEMPRA achieves an attack success rate exceeding 99% on all models, even under low-noise conditions. This performance highlights its ability to resolve inherent gradient conflicts in multi-model optimization, a problem not well addressed in previous studies.

[0062] Furthermore, SEMPRA's reversible mechanism ensures seamless restoration of diagnostic fidelity. Even with adversarial mosaic perturbations, the restored image maintains edge sharpness and anatomical consistency indistinguishable from the original, fully meeting clinical diagnostic criteria. This reversibility bridges the gap between data security and clinical application, allowing hospitals to share SEMPRA-protected images for telemedicine or research without compromising patient privacy or diagnostic accuracy.

[0063] Please see Figure 9 A second embodiment of the present invention provides a medical image privacy data protection device based on reversible adversarial patches, comprising:

[0064] The lesion area localization unit 101 is used to acquire the input image, use the Canny edge detection algorithm to localize the input image, and identify the lesion area;

[0065] The aggregation similarity calculation unit 102 is used to dynamically calculate the weight values ​​of the deep learning model based on the lesion region using the GSW algorithm, and introduces the CMGA algorithm to jointly optimize the perturbation in the frequency domain and spatial domain to minimize the perceptual difference between the original image and the adversarial sample, so as to obtain the final aggregation similarity.

[0066] The perturbation embedding and restoration unit 103 is used to calculate the reversible adversarial perturbation based on the final aggregated similarity, embed the reversible adversarial perturbation into the adversarial image, and use RDH-GI technology to extract the embedded binary bit stream from the adversarial image to perform reverse perturbation removal and obtain an image restored to the original state.

[0067] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for medical image privacy data protection based on reversible adversarial patch, characterized in that, The method comprises the following steps: An input image is acquired, and a Canny edge detection algorithm is used to perform positioning processing on the input image to identify a lesion area; Based on the lesion area, the weight values of the deep learning model are dynamically calculated using the GSW algorithm, and the CMGA algorithm is introduced to jointly optimize the disturbance in the frequency domain and the spatial domain to minimize the perceptual difference between the original image and the adversarial sample to obtain the final aggregated similarity; Based on the final aggregated similarity, a reversible adversarial disturbance is calculated, the reversible adversarial disturbance is embedded in an adversarial image, and the embedded binary bit stream is extracted from the adversarial image using the RDH-GI technology to perform reverse disturbance elimination to obtain an image restored to the original state; Based on the lesion area, the weight values of the deep learning model are dynamically calculated using the GSW algorithm, specifically: For each deep learning model in the ensemble , compute its loss function with respect to the gradient of the adversarial sample where is the current adversarial sample at the t-th iteration, is the longitudinal coordinate data of the input image; Based on these gradients, a corresponding temporary adversarial sample is generated for each deep learning model Based on these gradients, a corresponding temporary adversarial sample is generated for each deep learning model , a perturbation amplitude control parameter; computing a loss for each temporary adversarial sample , the loss value reflects the sensitivity of the deep learning model to the current perturbation, the higher the loss, the more sensitive the model is to the perturbation; The performance of each model on the temporary samples generated by other models is used to accumulate the weights to measure the loss proportion of each model under different disturbances; For each deep learning model in the ensemble , traverse all other models , , and compute the loss of the model on the temporary adversarial sample generated by the model ; The weighted loss ratios of all other models are accumulated to calculate the weight of the model N is the number of deep learning models, is a modulation coefficient for controlling the influence of the loss ratio on the weight.​ The implementation steps of the CMGA are as follows: calculating the cosine similarity of the model gradient; constructing a similarity graph to systematically capture the similarity between all models in the set; calculating the average similarity of each model, calculating the average similarity score for each model to represent the overall consistency with the rest of the whole; aggregating the similarity score. 2.The reversible adversarial patch-based medical image privacy data protection method of claim 1, wherein, An input image is acquired, and a Canny edge detection algorithm is used to perform positioning processing on the input image to identify a lesion area, specifically: Obtaining an input image , using a Gaussian kernel to the input image for noise reduction, the formula is: wherein, is a convolution operation, is the scale of the Gaussian kernel, is the horizontal coordinate data of the input image; Gradient calculation is performed on the input image after Gaussian smoothing, and the gradient in the horizontal direction and the gradient in the vertical direction are calculated using a Sobel operator, and the formula is as follows: , , and are all Sobel kernels, wherein the calculation formula of the gradient amplitude G and the gradient direction are as follows: , ; Based on non-maximum suppression, only local maximum value is reserved along the gradient direction, and its formula is: ; According to the high threshold value and the low threshold value The local maximum value is identified, and the formula is: The strong edge and the weak edge are obtained, and the strong edge and the weak edge are connected to obtain a complete edge. The complete edges are connected into a closed region by morphological operation to obtain the final lesion region The region is composed of all pixel points (x, y), The edge of the lesion region can be divided into i small edges, The first small edge, The i-th small edge, , The i-th pixel point. 3.The reversible adversarial patch-based medical image privacy data protection method of claim 2, wherein, The CMGA algorithm is introduced to jointly optimize the disturbance in the frequency domain and the spatial domain to minimize the perceptual difference between the original image and the adversarial sample to obtain the final aggregated similarity, specifically: The CMGA algorithm is introduced to align the gradients of different models, the cosine similarity of the model gradient is calculated, and the directional consistency between any two model gradients is quantified based on the similarity index, wherein the higher the cosine similarity, the closer the directions of the two gradients. A similarity graph is constructed, defined as a matrix of cosine similarities between models and model gradients.​ wherein, for all pairs of models satisfying and a cosine similarity calculation is performed to obtain corresponding matrix elements, and the calculation formula is: , i and j are model indexes, c is the channel of the gradient tensor, h is the height of the gradient tensor, and w is the width dimension of the gradient tensor. The remaining matrix elements are directly assigned values, and the formula is: ; Calculate each model The average similarity score is calculated using the following formula: , with quantification model The degree of consistency with all other models in the ensemble in the gradient direction; aggregating all model average similarity scores to obtain a final aggregated similarity . 4.The reversible adversarial patch-based medical image privacy data protection method of claim 1, wherein, Based on the final aggregated similarity, a reversible adversarial disturbance is calculated, the reversible adversarial disturbance is embedded in an adversarial image, specifically: The R channel and the B channel of the color image are used to embed information, and the G channel pixel value is adaptively adjusted to maintain gray consistency; Generating reversible adversarial perturbations using deep learning model ensembles based on final aggregated similarity and quantizing the reversible adversarial perturbations into a binary bitstream; The low-variance region in the adversarial image is located according to the prediction error analysis, and the bit stream is embedded therein to realize seamless fusion. 5.The reversible adversarial patch-based medical image privacy data protection method of claim 1, wherein, The embedded binary bit stream is extracted from the adversarial image using the RDH-GI technology to perform reverse disturbance elimination to obtain an image restored to the original state, specifically: The RDH-GI technology is used to extract the embedded binary bit stream from the adversarial image, and the extraction process decodes the bit stream by checking the pixel changes that meet the gray invariant constraint to ensure accurate recovery of the disturbance parameters; The binary bit stream is decoded to reconstruct the disturbance matrix, which records the adversarial modifications initially applied to the image, and by reversely applying these disturbances, the influence is offset, and the image is restored to the original, unaltered state.

6. A reversible adversarial patch-based medical image privacy data protection apparatus for implementing the reversible adversarial patch-based medical image privacy data protection method according to any one of claims 1 to 5, characterized in that, The method comprises the following steps: A lesion area positioning unit is configured to acquire an input image, perform positioning processing on the input image using a Canny edge detection algorithm, and identify a lesion area. The aggregation similarity calculation unit is configured to calculate weight values of the deep learning model dynamically based on the lesion area by using a GSW algorithm, introduce a CMGA algorithm to jointly optimize disturbance in a frequency domain and a spatial domain, minimize a perceptual difference between an original image and the adversarial sample, and obtain a final aggregation similarity. The disturbance embedding and recovery unit is configured to calculate reversible adversarial disturbance based on the final aggregation similarity, embed the reversible adversarial disturbance in the adversarial image, extract a binary bit stream embedded in the adversarial image by using RDH-GI technology, reverse the disturbance, and obtain an image restored to an original state. The GSW algorithm is used to calculate weight values of the deep learning model dynamically based on the lesion area, and the calculation comprises the following steps: for each deep learning model in the ensemble , compute its loss function with respect to the adversarial sample where is the current adversarial sample at the t-th iteration, is the longitudinal coordinate data of the input image;​ Based on these gradients, a corresponding temporary adversarial sample is generated for each deep learning model , a perturbation amplitude control parameter;​ computing a loss for each temporary adversarial sample the loss value reflects the sensitivity of the deep learning model to the current perturbation, the higher the loss, the more sensitive the model is to the perturbation; The weight of each model is accumulated according to the performance of each model on the temporary sample generated by other models, so as to measure the loss proportion of each model under different disturbances. For each deep learning model in the ensemble , traverse all other models , , and compute the loss of the model on the temporary adversarial sample generated by the model ; The weighted loss ratios of all other models are accumulated to calculate the weight of the model N is the number of deep learning models, is a modulation coefficient for controlling the impact of the loss ratio on the weight.​ The implementation steps of the CMGA are as follows: calculating a cosine similarity of model gradients; constructing a similarity graph to systematically capture the similarity between all models in the set; calculating the average similarity of each model, calculating the average similarity score for each model to represent the overall consistency with the overall other part; and obtaining an aggregation similarity score.

Images