A semi-open source medical image dataset hierarchical protection and verification method based on double watermarking

Abstract

The application belongs to the technical field of digital watermarking, and discloses a semi-open medical image dataset hierarchical protection and verification method based on double watermarking, which comprises a blind watermark embedding algorithm and a visible watermark embedding algorithm, and the overall process comprises two main steps, one is blind watermark embedding of a medical image dataset, which realizes passive copyright verification protection of the dataset; the other is visible watermark embedding of the medical image dataset embedded with the blind watermark, which realizes active authorization protection of the dataset. The application can realize hierarchical protection according to hierarchical authorization of different user groups by a dataset developer, and the double watermarking mechanism realizes active authorization control function of the dataset and copyright verification function of the dataset after being stolen. Through experiments, it is found that the application has good imperceptibility, harmlessness, effectiveness and universality, and can well protect the semi-open medical image dataset.

Description

Technical Field

[0001] The present invention relates to the field of digital watermark technology, and in particular to a hierarchical protection and verification method for a semi-open source medical image data set based on double watermarks. Background Art

[0002] In recent years, deep neural networks (DNNs), with their remarkable performance, have been widely and successfully applied in many fields, such as image classification and detection, speech recognition, natural language processing, medical diagnostic systems, and self-driving cars. Datasets, especially high-quality open-source datasets, are a key factor in the prosperity of DNNs. With the continuous application of deep learning in various fields, the tasks required of deep learning are becoming increasingly intelligent. Consequently, the demand for datasets suitable for training deep learning models is also increasing. These datasets enable researchers to design powerful deep learning models, accelerating the development of deep learning and promoting its application in various fields. The open-source nature of datasets has accelerated the rapid development of the deep learning field. Developers of open-source datasets generally require that these datasets be used only for academic or educational purposes, not commercially.

[0003] There are also semi-open source datasets, where developers create datasets that contain private or non-public information and are intended for circulation and use only within a small circle. For example, in a regional medical collaboration service platform, different hospitals collaborate on the platform, and medical images are only used by hospitals within the collaboration. When hackers gain access, it can lead to the leakage of a large amount of sensitive information about doctors or patients; at the same time, illegal access and use by authorized users can also cause information leakage. From the perspective of protecting the copyright of medical images, valuable medical data, such as medical images obtained by expensive and high-end medical imaging equipment, is transmitted over the public network. Once leaked, it is very likely to lead to accidents and disputes. Therefore, copyright protection of semi-open source medical imaging datasets is crucial.

[0004] Digital watermarking is a common technique for copyright protection in multimedia information. From the perspective of actively protecting datasets, Wang et al. [Wang Nan, "Research and Implementation of a Digital Watermarking System Based on Copyright Protection," Guizhou University, 2019] proposed a video protection scheme using a dual watermarking mechanism based on spatial and transform domain watermarking. This scheme essentially embeds an image watermark into a single frame of a video. Furthermore, they proposed a visually reversible watermarking algorithm for medical images based on a regression support vector machine. This algorithm uses the texture of medical images as a basis for embedding quality and the regression support vector machine as a measure of pixel correlation. This active dataset protection scheme has the potential for leakage. Once the protection mechanism is compromised, the dataset's protection becomes ineffective.From the perspective of passive verification datasets: Banitalebi et al. [Banitalebi, A.; Nader-Esfahani, S.; and Avanaki, AN2018. Robust lsb watermarking optimized for local structural similarity. arXiv preprint arXiv:1803.04617.] proposed an LSB algorithm that embeds blind watermarks by replacing the lowest bit plane of the carrier image with the highest bit plane of the watermark image; Parah et al. [Parah, SA; Sheikh, JA; Loan, NA; and Bhat, GM2016. Robust and blind watermarking technique in dct domain using inter block coefficient differencing. Digital Signal Processing 53:11–24.] use the correlation between the DCT coefficients of adjacent blocks of the image to embed blind watermarks; Lu et al. [Lu, J.; Wang, M.; Dai, J.; Huang, Q.; Li, L.; and Chang, C. 2015. Multiple watermark scheme based on [dwt-dct quantization for medical images. Journal of Information Hiding and Multimedia Signal Processing 6(3):458–472.] proposed using wavelet transform to embed watermarks. By embedding the watermark in the low-frequency subband, the watermarked image has better imperceptibility and higher robustness. Li et al. [Open-sourced Dataset Protection via Backdoor Watermarking] verify whether the dataset has been used to train a third-party model, thereby completing copyright verification and protecting open-source datasets. The above passive copyright verification methods mainly optimize the watermarking scheme by focusing on the imperceptibility of the embedded watermark and its robustness against attacks. However, all of these methods occur after the dataset has been used, and the legitimate rights and interests of the dataset owner have been violated, making it impossible to protect the dataset from the root.

[0005] However, the data protection methods mentioned above are not suitable for protecting semi-open-source medical image datasets. These methods only address one aspect: active protection or passive verification. If a malicious user in a regional medical collaborative service platform leaks medical images for personal gain, copyright protection cannot be implemented to ensure accountability, effectively protecting the medical image dataset. Specifically, active protection cannot guarantee copyright verification in the event of a medical image dataset leak; furthermore, passive verification cannot prevent unauthorized users from using the dataset. Therefore, the key challenge in protecting semi-open-source medical image datasets is verifying whether different protection measures can be implemented to address different needs. Summary of the Invention

[0006] Aiming at the problem that existing technical solutions cannot well protect semi-open source medical image datasets, the present invention proposes a hierarchical protection and verification method for semi-open source medical image datasets based on double watermarks.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A hierarchical protection and verification method for a semi-open-source medical image dataset based on double watermarking, including:

[0009] Step 1: Clean medical image dataset C data and the blind watermark image W M After adjusting the corresponding size and channel through the preprocessing network, the blind watermark embedding module Gan-E network is input to obtain the medical image dataset C embedded with blind watermark M ;

[0010] Step 2: C data , C M and W M As the input of the blind watermark extraction module Gan-R network; the Gan-E network and the Gan-R network have the same network structure, including an image generator G and a discriminator D; the generator G is a U-Net network with a residual network added; the discriminator D is a convolutional neural network composed of multiple convolutional layers;

[0011] Step 3: Gan-R network distinguishes the input medical image dataset. If the medical image comes from C M , then extract the blind watermark image W from it RM And constantly and W M To make a comparison, RM The quality is getting higher and higher; if the medical image comes from C data , then no watermark information can be extracted from it, and a noise image is randomly generated;

[0012] Step 4: Use the feature image of the first convolutional layer of the convolutional neural network toM analyzing the images in C V , picking up the salient parts, embedding the visible watermark image W MV .

[0013] Further, the step 1 further comprises:

[0014] by C M and the medical images in C data , the Gan-E network is constantly optimized by the loss function.

[0015] Further, the loss function comprises an adversarial loss function L gan , an L1 loss function L 1c and a structural similarity loss function L ssim :

[0016]

[0017]

[0018]

[0019] where x n is the output of the generator, y n is its corresponding label, n represents the number of medical images in the dataset, μ x and σ x respectively represent the mean and standard deviation of W M , μ y and σ y respectively represent the mean and standard deviation of the medical images in C data , σ xy represents the covariance of the medical images in W M and C data , and m and l are constants set to avoid the denominator being 0.

[0020] Further, the step 3 further comprises: constantly optimizing the Gan-R network by the loss function.

[0021] Further, the loss function of the Gan-R network comprises an adversarial loss function L gan , an L1 loss function L 1w and a structural similarity loss function L ssim , wherein,

[0022]

[0023] Except for the L1 loss function, the remaining loss functions are consistent with the loss function of the Gan-E network.

[0024] Furthermore, the step 4 includes:

[0025] Step 41: Generate C using convolutional neural network M The characteristic image of

[0026] Step 42: Divide the feature image into 16 blocks of equal size by length and width, compare the number of pixels occupied by the feature images in the four adjacent blocks, select the four adjacent blocks with the largest number of pixels occupied by the feature images, and record the pixel coordinates (a, b) of the upper left vertex of the upper left block of the four adjacent blocks;

[0027] Step 43: Visual watermark image W V The size and transparency are optimized, where the visible watermark image W V The size of is less than or equal to the size of the above four adjacent blocks, and the transparency α∈(0,1);

[0028] Step 44: According to formula (8), the processed watermark image is placed in C M Embedding is performed at the medical image (a, b) in:

[0029] C MV =C M +αW V (s,a,b) (8)

[0030] Where α is the visible watermark image W V The transparency parameter, s is the scaled visible watermark image W V Parameters;

[0031] Step 45: Place C MV Put the target model into the test to test its accuracy and record it;

[0032] Step 46: Change the visible watermark image W V Repeat steps 44 and 45 to compare the accuracy until the best visible watermark image W that affects the accuracy is found. V The transparency of the key is obtained (s, a, b) and the embedded double watermark medical image dataset C MV .

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) A hierarchical copyright protection and verification scheme for a semi-open-source medical image dataset based on dual watermarking is proposed. This scheme can provide hierarchical protection based on the hierarchical authorization granted by the dataset developer to different user groups. The dual watermarking mechanism implements the active authorization control function of the dataset and the copyright verification function after the dataset is stolen.

[0035] (2) A visual watermark optimization algorithm is proposed. It aims to maximize the effectiveness of the dataset by screening the key parts of the dataset, selecting the embedding location of the visible watermark, and optimizing the watermark size and transparency. It generates a secret key for the coordinate parameters and transparency parameters of the embedded double-watermark dataset and the embedded visual watermark image. This greatly reduces the effectiveness of the target model trained with the double-watermark dataset, making it impossible to use the dataset for model training. The secret key is used to remove the visible watermark image, restore the dataset training function, and realize the dataset's active authorization function.

[0036] (3) A deep learning-based blind watermark embedding and extraction algorithm is proposed to improve the security of the dataset. By generating images through a deep learning network, the image watermark is embedded into the target dataset, meeting the characteristics of the dataset watermark's concealment and harmlessness, and realizing the function of passive verification of the dataset's copyright information. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic flow chart of a method for hierarchical protection and verification of a semi-open source medical image dataset based on double watermarking according to an embodiment of the present invention;

[0038] Figure 2 The effect of the blind watermark embedding and extraction algorithm of the embodiment of the present invention;

[0039] Figure 3 This is a flow chart of a visual watermark embedding algorithm according to an embodiment of the present invention;

[0040] Figure 4 This is the effect of the visual watermark embedding algorithm of the embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0042] The present invention is a semi-open source medical image dataset hierarchical protection and verification method based on dual watermark, including blind watermark embedding algorithm and visual watermark embedding algorithm. The overall process is as follows: Figure 1 The overall process consists of two main steps: one is to embed blind watermarks into the medical image dataset to achieve passive copyright verification and protection of the dataset; the other is to embed visible watermarks into the blind watermarked medical image dataset to achieve active authorization and protection of the dataset.

[0043] Specifically, the clean medical image dataset (i.e., the medical image dataset without any watermark added) C data and the blind watermark image W M After adjusting the corresponding size and channel through the preprocessing network, the blind watermark embedding module Gan-E network is input to obtain the medical image dataset C embedded with blind watermark M. The clean medical image dataset C data , Medical image dataset C embedded with blind watermark M and the original blind watermark image W M As the input of the blind watermark extraction module Gan-R network. The Gan-R network distinguishes the input medical image dataset. If the medical image comes from the medical image dataset C embedded with the blind watermark M , then extract the blind watermark image W from it RM And continuously and the original blind watermark image W M Make a comparison to make the extracted blind watermark image W RM The quality is getting higher and higher; if the medical image comes from a clean medical image dataset C data , then no watermark information can be extracted from it, and the training network randomly generates noise images. For the obtained medical image dataset C embedded with blind watermark M , using the feature image of the first convolutional layer of the convolutional neural network, the medical image dataset C embedded with blind watermark M Analyze the image in the image, select the significant part, and embed the visual watermark image W V , and finally obtain the medical image dataset C embedded with double watermarks MV .

[0044] Specifically, the Generative Adversarial Network (GAN), proposed by Goodfello et al. [Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y. 2014. Generative adversarial nets. In Advances in neural information processing systems, 2672–2680.], is a DNN model consisting of a generator network and a discriminator network. The generator network's task is to generate new samples that are similar or identical to real samples, as required by the designer, and continuously feed these samples to the discriminator until the discriminator cannot identify the generated samples as fake, thus successfully training. The discriminator's task is to continuously distinguish between real and generated samples and improve the discriminator network's discrimination ability.

[0045] In our solution, we use a generative adversarial network to embed blind watermark images into a clean medical image dataset, making the watermarked images visually indistinguishable from the clean images. The network for extracting the blind watermark also uses a generative adversarial network to extract the blind watermarked images from the embedded medical image dataset.

[0046] (1) Blind watermark embedding and extraction algorithm

[0047] In this embodiment, we describe the proposed blind watermark embedding and extraction algorithm in detail. It consists of a blind watermark embedding module, the Gan-E network, and a blind watermark extraction module, the Gan-R network. The Gan-E and Gan-R networks share the same network structure, consisting of an image generator G and a discriminator D. The generator G is an optimized U-Net network with a residual neural network (RBN) added to the original network structure; the discriminator D is a convolutional neural network with five convolutional layers. The specific embedding and extraction steps are as follows:

[0048] Step S11: Set up a clean medical image dataset C according to actual needs data and the blind watermark image W M ;

[0049] Step S12: Use Gan-E network to blindly watermark the image W M Embedding Clean Medical Image Dataset C data , get the medical image dataset C embedded with blind watermark M , which can be expressed as formula (1):

[0050] C M =Gan-E(C data ,W M ) (1)

[0051] Step S13: Obtain the medical image dataset C embedded with the blind watermark M Medical image dataset C data The Gan-E network is continuously optimized through loss functions by comparing the medical images in the dataset. The loss functions involved are: adversarial loss function, L1 loss function, and structural similarity loss function, which are specifically expressed as follows:

[0052] Adversarial loss function L gan : Use binary cross entropy loss to express the adversarial loss of embedding and extraction networks:

[0053]

[0054] Where x n is the output of the generator, y n is its corresponding label.

[0055] L1 loss function L 1c : It is composed of the reconstruction error of the embedding network and the extraction network. The purpose is to make the medical image embedded with the blind watermark as close as possible to the clean medical image:

[0056]

[0057] Where C M represents a medical image dataset embedded with blind watermarks, C data It represents a clean medical image dataset, and n represents the number of medical images in the dataset.

[0058] Structural similarity loss function L ssim : This loss is more sensitive to local structural changes and can improve the quality of generated images:

[0059]

[0060] Where μ x and σ x They represent the image W embedded with blind watermark respectively. M The mean and standard deviation, μ y and σ y They represent the clean medical image dataset C data The mean and standard deviation of traditional Chinese medicine images, σ xy It represents the covariance of the two images, and the m and l constants are used to avoid the denominator being 0 and maintain stability. ) is a number between 0 and 1. A larger value indicates a smaller difference between the output image and the undistorted image, i.e., better image quality. When two images are identical, SSIM = 1, so when used as a loss function, it is negated and added one.

[0061] Step S14: clean medical image dataset C data , Medical image dataset C embedded with blind watermark M and the original blind watermark image W M As the input of the blind watermark extraction module Gan-R network. The Gan-R network distinguishes the input medical image dataset. If the medical image comes from the medical image dataset C embedded with blind watermark M , then extract the blind watermark image W from it RM And continuously and the original blind watermark image W M Make a comparison to make the extracted blind watermark image W RM The quality is getting higher and higher; if the medical image comes from a clean medical image dataset C data , then no watermark information can be extracted from it, and the network is trained to randomly generate noise images.

[0062] Step S15: Use the Gan-R network to extract the blind watermark image from the medical image dataset C embedded with the blind watermark M Extract it and get the extracted blind watermark image W RM , which can be expressed as follows:

[0063] WRM =Gan-R(C data ,C M ,W M ) (5)

[0064] Step S16: Extract the blind watermark image W RM and the original blind watermark image W M For comparison, the Gan-R network is continuously optimized through the loss function. The loss function involved is the same as the loss function of the Gan-E network, with only the L1 loss function being different, as shown below:

[0065]

[0066] W RM Represents the extracted blind watermark image W RM , W M Represents the blind watermark image W M .

[0067] In summary, the total loss function designed in this embodiment can be written as follows:

[0068] L all =αL gan (C data ,C M ,W M ,W RM )+β[L 1c (C data ,C M )+L 1w (W M ,W RM )+L ssim (C data ,C M ,W M ,W RM )](7)

[0069] The effects of the blind watermark embedding and extraction algorithms are shown as follows: Figure 2 As shown: Figure 2 (a) is a medical image in the clean medical image dataset, Figure 2 (b) is the embedded blind watermark image, Figure 2 (c) is a medical image embedded in the blind watermark medical image dataset, Figure 2 (d) is the blind watermark image extracted from the blind watermark embedded medical image dataset.

[0070] (2) Visual watermark embedding algorithm

[0071] In this embodiment, the proposed visual watermark embedding algorithm is described in detail, which consists of four stages. Figure 3As shown in the figure, the first stage is to use convolutional neural network to generate a medical image dataset C embedded with blind watermarks. M The second stage is to select the significant feature parts in the feature map; the third stage is to select the visual watermark image W V The size and transparency parameters when embedding; the fourth stage is to visualize the watermark image W V Embedding, generate a medical image dataset C embedded with double watermarks MV .

[0072] The specific embedding steps are as follows:

[0073] Step S21: Generate a medical image dataset C embedded with blind watermark using convolutional neural network CNN M feature image.

[0074] Step S22: Divide the feature image into four equal parts in length and width, and evenly divide it into 16 blocks of the same size. Compare the pixels occupied by the feature images in the four adjacent blocks, select the four adjacent blocks with the most pixels occupied by the feature images, and record the pixel coordinates (a, b) of the upper left vertex of the upper left block of the four adjacent blocks.

[0075] Step S23: Visual watermark image W V Optimize the size and transparency, and require the visible watermark image W V The size of must be smaller than or equal to the size of the above four adjacent blocks, and the transparency α requires α∈(0, 1).

[0076] Step S24: embed the processed watermark image in the blind watermark medical image dataset C M The medical image (a, b) in is embedded. The specific embedding method is shown in formula (8):

[0077] C MV =C M +αW V (s,a,b) (8)

[0078] Where α is the visible watermark image W V The transparency parameter, s is the scaled visible watermark image W V Parameters, C MV To embed a double watermark medical image dataset, we embed a blind watermark medical image dataset C M Medical image and visual watermark image W in V Perform pixel-by-pixel operations.

[0079] Step S25: embed the double-watermarked medical image dataset C under transparency MV Put the target model into the test. Take the classification model as an example, test its classification accuracy and record it.

[0080] Step S26: Change the visible watermark image W V Repeat steps S24 and S25 to compare the accuracy until the best visual watermark image W that affects the classification accuracy is found. V The transparency of the key is obtained (s, a, b) and the embedded double watermark medical image dataset C MV .

[0081] It is worth noting that after step S26, the obtained C MV Restore the functions of the trained model for the medical image dataset:

[0082] Step S27: Get the secret key (s, a, b) and embed the double watermark medical image dataset C MV , perform the reverse operation of step S24 to obtain the image W with the visible watermark removed V Embedded blind watermark medical image dataset C M .

[0083] C M =C MV -αW V (s,a,b) (9)

[0084] The effect of the visual watermark embedding algorithm is shown as follows Figure 4 As shown: Figure 4 (a) is the feature image of the medical image embedded in the blind watermark medical image dataset, Figure 4 (b) is the embedded visible watermark image, Figure 4 (c) is a medical image embedded in the blind watermark medical image dataset, Figure 4 (d) is a medical image embedded in the double-watermark medical image dataset, Figure 4 (e) is a medical image embedded in a dual-watermark medical image dataset, after the visible watermark image is removed according to the secret key.

[0085] In order to verify the effect of the present invention, the following experiments were performed:

[0086] The experimental simulation platform is as follows: CPU: Intel(R) Core(TM) i7-6700, 3.40GHz; Memory: 8.00GB; Operating system: Windows 10; Coding tool: Python 3.7. In the experimental simulation, NIHCHESTXRAY was selected as the clean medical image dataset C for testing. data , Henan University logo as blind watermark image W M , the logo of the School of Artificial Intelligence of Henan University is used as the visible watermark image W V , respectively as Figure 2 (a) Figure 2 (b) and Figure 4 (b) as shown.

[0087] (a) imperceptibility analysis

[0088] Imperceptibility is the degree that the watermark can be perceived after being embedded into the host image. The peak signal-to-noise ratio (PSNR) and SSIM criteria are chosen to evaluate the imperceptibility of the proposed scheme, the unit of PSNR is decibel (dB). The PSNR and SSIM of the embedding algorithm of the scheme are shown in Table 1, columns 2 and 3, which are obtained by calculating the corresponding medical images in the embedding blind watermark medical image dataset C M and the clean medical image dataset C data .

[0089] For the design scheme of the present embodiment, in addition to meeting the imperceptibility of embedding the watermark, the watermark needs to be extracted from the embedded carrier image without destroying the carrier image. The PSNR and SSIM of the extraction algorithm of the scheme are shown in Table 1, columns 4 and 5, which are obtained by calculating the extracted blind watermark image W RM and the original blind watermark image W M .

[0090] Table 1 imperceptibility comparison experiment

[0091]

[0092] In this experiment, the third row of Table 1 is the design algorithm of the present embodiment, the PSNR on the embedding blind watermark medical image dataset C M reaches 38.69 dB, and the SSIM reaches 0.986; the PSNR on the embedding blind watermark medical image dataset C MThe PSNR on the image reaches 40.25dB, and the SSIM reaches 0.997, and the average value of the test data set is calculated. When the PSNR is greater than 30dB and the SSIM is greater than 0.97, the two compared images are already very close. When the PSNR is larger, the better, and the SSIM is close to 1, the watermark embedded in the watermark image is not easy to detect, so the scheme has good invisibility. It can be seen from the PSNR and SSIM data in the third and fourth rows of Table 1 that the performance of the algorithm designed in this embodiment is better than that of the algorithm designed by Yu et al. [C.Yu,"Attention baseddata hiding with generative adversarial networks,"in Proceedings of the AAAIConference on Artificial Intelligence,2020,vol.34,no.01,pp.1120-1128.] in embedding blind watermark medical image dataset C M The PSNR on the extracted blind watermark image W is 6.78dB higher and the SSIM is 0.048 higher. RM The PSNR on the dataset C is 9.59dB higher and the SSIM is 0.044 higher. From the PSNR and SSIM data in the third and fifth rows of Table 1, it can be seen that the performance of the algorithm designed in this embodiment is better than the comparison algorithm of Zhang et al. [C.Zhang, P.Benz, A.Karjauv, G.Sun, and ISKweon, "Udh: Universal deep hiding for steganography, watermarking, and light field messaging," Advances in Neural Information Processing Systems, vol.33, pp.10223-10234, 2020.] in embedding blind watermarks on the medical image dataset C. M The PSNR on the extracted blind watermark image W is 2.91dB higher and the SSIM is 0.011 higher. RMThe PSNR on the dataset C is 5.53dB higher and the SSIM is 0.016 higher. From the PSNR and SSIM data in the third and sixth rows of Table 1, it can be seen that the performance of the algorithm designed in this embodiment is only slightly better than that of the algorithm designed by Zhang et al. [C. Zhang, P. Benz, A. Karjauv, G. Sun, and ISKweon, "Udh: Universal deephiding for steganography, watermarking, and light field messaging," Advances in Neural Information Processing Systems, vol. 33, pp. 10223-10234, 2020.] in embedding blind watermarks on the medical image dataset C. M The PSNR on the extracted blind watermark image W is 0.44dB lower and the SSIM is 0.001 higher. RM The PSNR is 5.25dB higher and the SSIM is 0.021 higher. Combined with the experimental data, the algorithm performance of this embodiment is relatively excellent, indicating that the solution meets the invisibility requirement.

[0093] (b) Harmlessness and effectiveness analysis

[0094] The harmlessness is manifested as a blind watermark image W M Embedding Clean Medical Image Dataset C data After that, the clean medical image dataset C is not destroyed data The function of embedding blind watermark medical image dataset C M Can be used by the model for normal training. Taking the classification model as an example, the blind watermark image W is embedded M The classification model was trained using the same medical image datasets before and after the experiment. The difference in test accuracy between the two datasets was minimal, or even negligible. The experimental data, shown in the first and second rows of Table 2, shows that the difference in accuracy did not exceed 0.002, demonstrating the good safety of the invention.

[0095] The effectiveness is shown in the embedding blind watermark medical image dataset C M Embed a visible watermark W V After that, the clean medical image dataset C is destroyed data The function of the double-watermarked medical image dataset cannot be trained normally by the model, and the original function of the medical image dataset is lost. The target models used in the experiment are VGG16, VGG19, RESNET50, RESNET101 and RESNET152, which are all classification models of deep learning networks. The experimental data are shown in the first and third rows of Table 2. The double-watermarked medical image dataset C is embedded in the MVThe accuracy of the tested classification model has dropped significantly compared to the clean medical image dataset C data The classification accuracy of the test dropped by 50%-65%, which shows that the invention has good effectiveness.

[0096] Table 2 Data analysis of the harmlessness of blind watermarks and the effectiveness of visible watermarks

[0097]

[0098] (c) Universality Analysis

[0099] To verify the universality of the proposed design, repeated experiments were conducted on IMAGENET, PASCAL VOC, GENKI, and NIHCHEST XRAY. The experimental results are shown in Table 3. As shown in Table 3, although the PSNR and SSIM values ​​of the proposed design on PASCAL VOC and GENKI are slightly lower, the PSNR is still greater than 35 dB and the SSIM is still greater than 0.97, indicating excellent invisibility. Furthermore, the PSNR value on IMAGENET is higher than that on NIHCHEST XRAY, used in this case, while the SSIM values ​​are the same. This demonstrates the universality of the proposed method.

[0100] Table 3 Comparison of imperceptibility of blind watermarks on different datasets

[0101] Metrics / Datasets IMAGENET PASCAL VOC GENKI NIHCHEST XRAY PSNR(dB) 38.91 35.38 37.75 38.69 SSIM 0.986 0.969 0.973 0.986

[0102] In summary, the double-watermark-based hierarchical protection and verification method for semi-open source medical image datasets proposed in this embodiment has good imperceptibility, harmlessness, effectiveness and universality, and can effectively protect semi-open source medical image datasets.

[0103] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A hierarchical protection and verification method for a semi-open source medical image dataset based on double watermarks, characterized by: include: Step 1: Clean medical image dataset C data and the blind watermark image W M After adjusting the corresponding size and channel through the preprocessing network, the blind watermark embedding module Gan-E network is input to obtain the medical image dataset C embedded with blind watermark M ; Step 2: C data , C M and W M As the input of the blind watermark extraction module Gan-R network; the Gan-E network and the Gan-R network have the same network structure, including an image generator G and a discriminator D; the generator G is a U-Net network with a residual network added; the discriminator D is a convolutional neural network composed of multiple convolutional layers; Step 3: Gan-R network distinguishes the input medical image dataset. If the medical image comes from C M , then extract the blind watermark image W from it RM And constantly and W M To make a comparison, RM The quality is getting higher and higher; If the medical image comes from C data , then no watermark information can be extracted from it, and a noise image is randomly generated; Step 4: Use the feature image of the first convolutional layer of the convolutional neural network to M Analyze the image in the image, select the significant part, and embed the visual watermark image W V , and finally obtain the medical image dataset C embedded with double watermarks MV .

2. The method for hierarchical protection and verification of a semi-open source medical image dataset based on double watermarking according to claim 1 is characterized in that: The step 1 further comprises: By C M Medical images and C data The Gan-E network is continuously optimized through the loss function.

3. The method for hierarchical protection and verification of a semi-open source medical image dataset based on double watermarking according to claim 2 is characterized in that: The loss function includes the adversarial loss function L gan , L1 loss function L 1c And the structural similarity loss function L ssim : Where x n is the output of the generator, y n is its corresponding label, n represents the number of medical images in the dataset, μ x and σ x Respectively represent W M The mean and standard deviation, μ y and σ y Represents C data The mean and standard deviation of traditional Chinese medicine images, σ xy W M and C data The covariance of traditional Chinese medicine images, m and l are constants set to avoid the denominator being zero.

4. The method for hierarchical protection and verification of a semi-open source medical image dataset based on double watermarking according to claim 3 is characterized in that: The step 3 also includes: continuously optimizing the Gan-R network through the loss function.

5. The method for hierarchical protection and verification of a semi-open source medical image dataset based on double watermarking according to claim 4 is characterized in that: The loss function of the Gan-R network includes the adversarial loss function L gan , L1 loss function L 1w And the structural similarity loss function L ssim ,in, Except for the L1 loss function, the other loss functions are consistent with the loss function of the Gan-E network.

6. The method for hierarchical protection and verification of a semi-open source medical image dataset based on double watermarking according to claim 1 is characterized in that: The step 4 comprises: Step 41: Generate C using convolutional neural network M The characteristic image of Step 42: Divide the feature image into 16 blocks of equal size by length and width, compare the number of pixels occupied by the feature images in the four adjacent blocks, select the four adjacent blocks with the largest number of pixels occupied by the feature images, and record the pixel coordinates (a, b) of the upper left vertex of the upper left block of the four adjacent blocks; Step 43: Visual watermark image W V The size and transparency are optimized, where the visible watermark image W V The size of is less than or equal to the size of the above four adjacent blocks, and the transparency α∈(0,1); Step 44: According to formula (8), the processed watermark image is placed in C M Embedding is performed at the medical image (a, b) in: C MV =C M +αW V (s,a,b) (8) Where α is the visible watermark image W V The transparency parameter, s is the scaled visible watermark image W V Parameters; Step 45: Place C MV Put the target model into the test to test its accuracy and record it; Step 46: Change the visible watermark image W V Repeat steps 44 and 45 to compare the accuracy until the best visible watermark image W that affects the accuracy is found. V The transparency of the key is obtained (s, a, b) and the embedded double watermark medical image dataset C MV .

Images