Medical image sensitive information processing method, device, computer equipment and medium

Abstract

The present application relates to a method, device, computer equipment, storage medium and computer program product for processing sensitive information of medical images. The method obtains a medical image to be processed; performs text detection on the medical image to obtain a text area image; based on a conditional generative adversarial network, fills the text area image according to the pixels of the adjacent area image of the text area image to obtain a desensitized image; encrypts the text in the text area image and embeds it into the desensitized image to obtain a target image. The whole scheme first obtains the text area of ​​the medical image, and based on a conditional generative adversarial network, fills the text area of ​​the medical image. The image after removing the text is more realistic and reasonable, and can retain more original image features. The text is then encrypted and embedded into the desensitized image. Legal users can view the target image containing text according to their needs. While ensuring the security of the text information in the medical image, they can also legally obtain the information contained in the medical image.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for processing image-sensitive information. Background Art

[0002] With the advancement of medical technology, clinical medicine increasingly relies on medical imaging to diagnose or measure patients' physical data. At the same time, thanks to the development of deep learning, medical image processing technology has continued to advance, driving the development of auxiliary diagnosis and generating a large amount of medical image data in the process.

[0003] However, these medical images often carry a large amount of sensitive information, such as the patient's personal information and disease diagnosis information. This sensitive information can be easily obtained, posing risks such as privacy leaks.

[0004] Current medical image processing methods can directly erase text from images, but the erased image cannot be restored. Therefore, current medical image processing methods cannot balance information security and legal access to text content in images. Summary of the Invention

[0005] Based on this, it is necessary to provide an image-sensitive information processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve information security and legally obtain text information to address the above technical problems.

[0006] In a first aspect, the present application provides a method for processing sensitive information in medical images. The method comprises:

[0007] Acquiring a medical image to be processed;

[0008] Perform text detection on medical images to obtain text area images;

[0009] Based on the conditional generative adversarial network, the text area image is filled with pixels of the adjacent area image of the text area image to obtain a desensitized image;

[0010] The text in the text area image is encrypted and embedded into the desensitized image to obtain the target image.

[0011] In one embodiment, performing text detection on a medical image to obtain a text area image includes: performing feature extraction on the medical image to obtain a feature map; performing feature fusion on the feature map to obtain a fused feature map; and performing text detection based on the fused feature map to obtain a text area image.

[0012] In one embodiment, performing feature fusion on the feature map to obtain a fused feature map includes: performing feature enhancement on the feature map based on a feature pyramid to obtain an enhanced feature map; and performing feature fusion and dimensionality reduction on the enhanced feature map to obtain a fused feature map.

[0013] In one embodiment, performing text detection based on the fused feature map to obtain a text area image includes: post-processing the fused feature map to obtain a post-processed fused feature map; performing text detection on the post-processed fused feature map to obtain a text area image.

[0014] In one embodiment, based on a conditional generative adversarial network, filling a text area image according to pixels of an area image adjacent to the text area image to obtain a desensitized image includes: based on a conditional generative adversarial network, extracting pixels of an area image adjacent to the text area image; filling the text area image according to pixels of an area image adjacent to the text area image to obtain a desensitized image.

[0015] In one embodiment, encrypting the text in the text area image and embedding it into the desensitized image to obtain the target image includes: performing discrete wavelet transform and discrete cosine transform on the desensitized image to obtain the transformed desensitized image; encrypting the text in the text area image and embedding it into the transformed desensitized image to obtain the desensitized image with text; performing inverse discrete wavelet transform and inverse discrete cosine transform on the desensitized image with text to obtain the target image.

[0016] In one embodiment, the method further includes: extracting text from the target image to obtain text in the encrypted text area image; and decrypting the text in the encrypted text area image to obtain a desensitized image with text.

[0017] In a second aspect, the present application also provides a medical image sensitive information processing device. The device includes:

[0018] An acquisition module, used for acquiring medical images to be processed;

[0019] A text detection module is used to detect text in medical images and obtain text area images;

[0020] A desensitization module is used to fill the text area image with pixels of adjacent area images based on a conditional generative adversarial network to obtain a desensitized image;

[0021] The target image generation module is used to encrypt the text in the text area image and embed it into the desensitized image to obtain the target image.

[0022] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0023] Acquiring a medical image to be processed;

[0024] Perform text detection on medical images to obtain text area images;

[0025] Based on the conditional generative adversarial network, the text area image is filled with pixels of the adjacent area image of the text area image to obtain a desensitized image;

[0026] The text in the text area image is encrypted and embedded into the desensitized image to obtain the target image.

[0027] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0028] Acquiring a medical image to be processed;

[0029] Perform text detection on medical images to obtain text area images;

[0030] Based on the conditional generative adversarial network, the text area image is filled with pixels of the adjacent area image of the text area image to obtain a desensitized image;

[0031] The text in the text area image is encrypted and embedded into the desensitized image to obtain the target image.

[0032] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0033] Acquiring a medical image to be processed;

[0034] Perform text detection on medical images to obtain text area images;

[0035] Based on the conditional generative adversarial network, the text area image is filled with pixels of the adjacent area image of the text area image to obtain a desensitized image;

[0036] The text in the text area image is encrypted and embedded into the desensitized image to obtain the target image.

[0037] The above-mentioned medical image sensitive information processing method, device, computer equipment, storage medium and computer program product obtain the medical image to be processed; perform text detection on the medical image to obtain a text area image; based on a conditional generative adversarial network, fill the text area image according to the pixels of the adjacent area image of the text area image to obtain a desensitized image; encrypt the text in the text area image and embed it into the desensitized image to obtain a target image. The whole scheme first obtains the text area of ​​the medical image, and then fills the text area of ​​the medical image based on the conditional generative adversarial network. Compared with the traditional text removal method, the image after removing the text in this application is more realistic and reasonable, and can retain more original image features. Afterwards, the text is encrypted and embedded in the desensitized image. Legal users can view the target image containing text according to their needs. While ensuring the security of the text information in the medical image, they can also legally obtain the information contained in the medical image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A diagram showing an application environment of a method for processing sensitive information of medical images in one embodiment;

[0039] Figure 2 1 is a flow chart of a method for processing sensitive information of medical images in one embodiment;

[0040] Figure 3 A schematic diagram of the structure of a conditional generative adversarial network in one embodiment;

[0041] Figure 4 Schematic diagram of the structure of a feature pyramid enhancement module in one embodiment;

[0042] Figure 5 is a structural block diagram of a medical image sensitive information processing device in one embodiment;

[0043] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0045] Currently, the four main types of medical imaging used in clinical practice are X-rays, MRIs, PET, and ultrasound. Doctors use these imaging methods to observe areas of interest, enabling accurate diagnosis or measurement. Thanks to the advancement of deep learning, medical image processing has garnered significant attention and research from numerous institutions and researchers, driving the development of assisted diagnosis and generating a vast amount of medical image data. Ultrasound prenatal examinations are radiation-free, relatively affordable, and safe. They can measure fetal growth parameters during pregnancy and detect fetal anomalies, making them a very common practice in clinical practice. Every pregnant woman undergoes multiple ultrasound examinations throughout her pregnancy.

[0046] However, these ultrasound images often contain a wealth of personal, private, and sensitive information, including the patient's or examiner's name and related parameters. This information, including name, fetal illnesses, and the age of the mother and fetus, is often exposed within the images. This private information is easily accessible, and if it is exploited or leaked by those engaging in fraud, marketing, or other illegal activities, it could have significant consequences. Currently, data protection primarily involves signing confidentiality agreements with partners, strengthening personnel management, and enhancing network security.

[0047] For medical staff, this sensitive information is necessary in some cases. Therefore, it is necessary to embed the sensitive information into the image without affecting the display, and then restore the desensitized image with embedded sensitive information to facilitate medical staff to obtain case information.

[0048] The medical image sensitive information processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network, and the conditional generative adversarial network is trained on the server. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 obtains the medical image to be processed; performs text detection on the medical image to obtain a text area image; based on the conditional generative adversarial network trained on the server, the terminal fills the text area image according to the pixels of the adjacent area images of the text area image to obtain a desensitized image; the text in the text area image is encrypted and embedded in the desensitized image to obtain a target image. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0049] In one embodiment, Figure 2 As shown, a method for processing sensitive information of medical images is provided, and the method is applied to Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0050] Step 202: Acquire a medical image to be processed.

[0051] The medical image to be processed refers to a medical image with text. Medical images include, but are not limited to, images obtained by scanning an object using a medical scanning device. After scanning, the medical scanning device can send the scanned image to a computer device in real time or periodically, depending on actual needs.

[0052] Specifically, the terminal is connected to the medical scanning device via a network, the medical scanning device scans the object to be scanned to obtain a medical image to be processed, and the medical scanning device sends the medical image to be processed to the terminal. The terminal receives the medical image to be processed sent by the medical scanning device.

[0053] The medical image to be processed can also be uploaded by the user on the terminal display interface. The terminal listens to and responds to the user's image upload operation to obtain the medical image to be processed.

[0054] Step 204: Perform text detection on the medical image to obtain a text area image.

[0055] Specifically, the terminal detects the text position of the medical image based on the text detection network of the text segmentation, obtains the text area in the medical image, and uses the image of the text area as the text area image.

[0056] Step 206 , based on the conditional generative adversarial network, fill the text area image according to the pixels of the adjacent area images of the text area image to obtain a desensitized image.

[0057] Specifically, the terminal obtains pixels of an image area adjacent to the text area image based on the generator of the conditional generative adversarial network, fills the text area according to the pixels of the image area adjacent to the text area image, and generates a medical image with the text area removed. The terminal divides the medical image and the medical image with the text area removed into a preset number of image blocks based on the discriminator of the conditional generative adversarial network, averages the pixels of the segmented image blocks, and obtains the output of the discriminator. When the discriminator identifies that the medical image with text removed generated by the generator is true, a desensitized image is obtained.

[0058] Step 208: encrypt the text in the text area image and embed it into the desensitized image to obtain a target image.

[0059] Specifically, the terminal extracts text from the text area image, encrypts it using a key, and embeds the encrypted text into the desensitized image, resulting in a target image that requires decryption to extract the text information. Authorized medical personnel can then use the key to restore the text-containing medical image to meet medical needs.

[0060] In the above-mentioned method for processing sensitive information of medical images, a medical image to be processed is obtained; text detection is performed on the medical image to obtain a text area image; based on a conditional generative adversarial network, the text area image is filled according to the pixels of the adjacent area image of the text area image to obtain a desensitized image; the text in the text area image is encrypted and embedded into the desensitized image to obtain a target image. The entire scheme first obtains the text area of ​​the medical image, and then fills the text area of ​​the medical image based on a conditional generative adversarial network. Compared with the traditional text removal method, the image after removing the text in this application is more realistic and reasonable, and can retain more original image features. Afterwards, the text is encrypted and embedded into the desensitized image. Legal users can view the target image containing text according to their needs. While ensuring the security of the text information in the medical image, they can also legally obtain the information contained in the medical image.

[0061] In an optional embodiment, performing text detection on a medical image to obtain a text area image includes: performing feature extraction on the medical image to obtain a feature map; performing feature fusion on the feature map to obtain a fused feature map; and performing text detection based on the fused feature map to obtain a text area image.

[0062] Specifically, if Figure 3 As shown in the figure, the terminal inputs the medical image into the text detection network for text segmentation. Based on the multiple layers of the ResNet50 network, the medical image features are extracted, resulting in multiple corresponding feature maps with different resolutions. Feature fusion is then performed based on each feature map to obtain a fused feature map corresponding to each feature map.

[0063] The multiple layers are feature extraction layers of the ResNet50 network. The feature extraction layers may include convolutional layers and pooling layers, or they may only include convolutional layers. If the feature extraction layers include convolutional layers and pooling layers, that is, after the convolutional layers perform convolution operations on the medical image to extract the corresponding feature maps, the pooling layers pool the feature maps before outputting them. If the feature extraction layers include four feature extraction layers, four feature maps can be obtained.

[0064] In an optional embodiment, performing feature fusion on the feature map to obtain a fused feature map includes: performing feature enhancement on the feature map based on a feature pyramid to obtain an enhanced feature map; and performing feature fusion and dimensionality reduction on the enhanced feature map to obtain a fused feature map.

[0065] Specifically, if Figure 3 As shown in Figure 1, the terminal is based on the three-layer FPEM (Feature Pyramid Enhance Module) to fuse the high-scale and low-scale features of the feature map. The output of the previous layer is used as the input of the next layer. After three feature enhancements, an enhanced feature map is obtained. The network structure of FPEM is shown in Figure 1. Figure 4 As shown in the figure, each layer of FPEM has the same number of input and output layers. Based on the resolution of each feature map, features are added in order from low to high to obtain the first enhanced feature map corresponding to each feature map. Then, based on the resolution of each feature map, features are added in order from high to low to obtain the second enhanced feature map corresponding to each feature map. The first enhanced feature map and the second enhanced feature map are merged to obtain the enhanced map. Merging can be understood as merging image channels.

[0066] Afterwards, the terminal adds the outputs of the three layers of FPEM based on FFM (Feature Fusion Module) to obtain the first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map, upsamples the second fusion feature map, the third fusion feature map and the fourth fusion feature map to obtain three sampling feature maps of the same size as the first fusion feature map, and fuses the four sampling feature maps to obtain a fusion feature map.

[0067] In this embodiment, feature enhancement is performed on the feature map through the feature pyramid enhancement module and the feature fusion module, which can fuse the low-resolution feature map with strong semantic information and the high-resolution feature map with weak semantic information but rich spatial information.

[0068] In an optional embodiment, performing text detection based on the fused feature map to obtain a text area image includes: post-processing the fused feature map to obtain a post-processed fused feature map; performing text detection on the post-processed fused feature map to obtain a text area image.

[0069] Among them, post-processing refers to convolving the fused feature map to obtain a feature matrix, extracting the text area from the feature matrix, and obtaining a text area image.

[0070] Specifically, the terminal uses three convolutions on the fused feature map to obtain a text region matrix, a kernel region matrix, and a similarity matrix. The text region matrix is ​​T(1*W*H), the kernel region matrix is ​​K(1*W*H), and the similarity vector is S(4*W*H), where S represents a pixel among the W*H pixels. T represents whether a pixel is a text region among the W*H pixels, and K represents whether a pixel is a kernel among the W*H pixels. Text regions often overlap, and kernels are used to distinguish text instances. A text instance is a complete text region in an image, and a kernel is the region obtained after scaling the text region, which is used to distinguish different text blocks.

[0071] When training the network, the loss is calculated based on the distance between the kernel and pixels within the same text instance, minimizing the distance. The distance between different kernels is then calculated to maximize the distance between kernels of different instances. Dice loss is calculated between the segmented text and the ground truth. The similarity vector represents four channels, representing W*H pixels, and is used to calculate the similarity between a pixel and its upper, lower, left, and right neighbors.

[0072] During prediction, connected domains are first selected. Each connected domain is treated as a kernel, and the average similarity eigenvalue of the kernel is calculated. Then, pixels surrounding each kernel that are within the text area and whose distance from the average similarity direction is less than d are merged with the kernel. This process is repeated until the kernel can no longer merge with surrounding pixels. The maximum value of the top, bottom, left, and right values ​​of each region is then calculated to obtain an image of the text area containing sensitive information.

[0073] In an optional embodiment, based on a conditional generative adversarial network, filling the text area image according to pixels of an area image adjacent to the text area image to obtain a desensitized image includes: based on a conditional generative adversarial network, extracting pixels of an area image adjacent to the text area image; filling the text area image according to pixels of an area image adjacent to the text area image to obtain a desensitized image.

[0074] Specifically, the terminal uses a Pix2pix conditional generative adversarial network to obtain pixels from adjacent regions of the text region image. Based on the Pix2pix conditional generative adversarial network generator, a Unet structure is used to fill the text region with pixels from adjacent regions of the text region image, generating a medical image with the text region removed. Based on the patchGAN discriminator, the terminal segments the medical image and the medical image with the text region removed into a preset number of image blocks, which can be 70*70 pixels. The pixels of the segmented image blocks are averaged to obtain the discriminator output. When the discriminator identifies the text-removed medical image generated by the generator as genuine, a desensitized image is obtained.

[0075] During the training process of the Pix2pix conditional generative adversarial network (CGN), an L1 loss function is introduced to minimize error, ensuring that the input image with text and the output image without text are identical except for the text. For text in selected images, the goal is to erase the text area and then fill it with a visually plausible background to maintain consistency between the erased text area and the surrounding texture. The labeled ultrasound images with text removed were divided into training and test sets in a 4:1 ratio. The CGN was then trained. Testing and loss analysis revealed that the best results were achieved at the 50th epoch.

[0076] In this embodiment, the Unet structure is used to facilitate the generation of image details, and realistic results can be obtained at different scales. Filling the text area with adjacent pixels can preserve more original image features.

[0077] In an optional embodiment, encrypting the text in the text area image and embedding it into the desensitized image to obtain the target image includes: performing discrete wavelet transform and discrete cosine transform on the desensitized image to obtain the transformed desensitized image; encrypting the text in the text area image and embedding it into the transformed desensitized image to obtain the desensitized image with text; performing inverse discrete wavelet transform and inverse discrete cosine transform on the desensitized image with text to obtain the target image.

[0078] Specifically, the terminal first converts the desensitized image from the RGB (Red, Green, Blue) color space to the YUV (Luminance Chrominance) color space, then performs a discrete wavelet transform on the luminance component Y, uses a discrete cosine transform to divide the low-frequency components into blocks, and performs SVD (Singular Value Decomposition) on each block. After that, the text in the text area image is encrypted and embedded in the desensitized image after decomposition to obtain a desensitized image with text; the desensitized image with text is subjected to an inverse discrete wavelet transform and an inverse discrete cosine transform to obtain the target image in RGB format.

[0079] Furthermore, the terminal first binarizes the text region image (i.e., the original medical image to be processed) to obtain a binarized text region image. Next, the binarized text region image is encrypted using the key, and the encrypted text region image matrix is ​​flattened into a sequence. The sequence is sorted according to the key, and the sorted sequence is restored to the original matrix size to obtain the encrypted image. For example, if the text region image is 80*100, the block is expanded to obtain an array of 8000. Each block has an index. The key is used as a random seed, and the blocks are shuffled in a fixed order according to the index.

[0080] Then, the terminal converts the desensitized image from RGB format to YUV format. Generate a random sequence, select the brightness component Y and apply discrete wavelet transform (DWT) to subdivide Y into four dimensions, and the DWT level is set to 2. The desensitized image is divided into pixel blocks of a preset size (i.e., N*N), and then each pixel block is subjected to discrete cosine transform (DCT) transform one by one. After that, singular value decomposition is performed on each transformed block to obtain U, S, and V vectors. According to the embedding expression U1S1V1=USV+k*W, the encrypted image is embedded into the S vector to obtain an embedding map, where k is the embedding strength. Finally, the embedded map U1S1V1 is subjected to inverse discrete cosine transform (IDCT) and inverse discrete wavelet transform to obtain Y', and then the converted embedded map is converted to RGB format to obtain the target image.

[0081] In this embodiment, by binarizing the text area image, the space can be compressed to improve the embedding calculation efficiency and the acquisition efficiency and accuracy of the target image.

[0082] In an optional embodiment, the above method further includes: extracting text from the target image to obtain text in the encrypted text area image; and decrypting the text in the encrypted text area image to obtain a desensitized image with text.

[0083] Specifically, the target image is input with a key as a random seed to obtain the target image containing the encrypted text. The encrypted target image is then converted to YUV format. The converted luminance component Y is then subjected to a discrete wavelet transform and divided into four sub-bands: LL, LH, HL, and HH. L stands for low and H stands for high, meaning L represents low frequency and H represents high frequency. Generally speaking, after the wavelet transform, the image is decomposed into low-low frequency, low-high frequency, and high-high frequency. Low frequency reflects the image's general appearance, while high frequency reflects image details.

[0084] Each pixel of the converted target image carrying the encrypted text is decoded and the LL is subjected to a DCT transform. Then, SVD is performed to obtain U1S1V1. The restoration expression W = (U1S1V1 - S1) / k is solved to obtain the encrypted image. Since the orthogonal matrices U and V are not used in the embedding process, this restoration expression can be used to obtain the medical image carrying the text. The W pixel indices are sorted according to a random seed to obtain an image of the text region containing the text. The text region image is then restored onto the desensitized image to obtain the desensitized image with the text and the original medical image.

[0085] In this embodiment, the image containing text and its location are embedded in the desensitized image without affecting the image's display. Without the key, the image cannot be extracted from it. When needed, the fixed key is used to extract the image containing text and its location from the image to restore the sensitive information.

[0086] To facilitate understanding of the technical solutions provided in the embodiments of the present application, the method for processing sensitive medical image information provided in the embodiments of the present application is briefly described using the complete process of processing sensitive medical image information.

[0087] (1) Obtain the medical image to be processed.

[0088] (2) Extract features from medical images and obtain feature maps.

[0089] (3) Based on the feature pyramid, the feature map is enhanced to obtain an enhanced feature map.

[0090] (4) Perform feature fusion and dimensionality reduction on the enhanced feature map to obtain a fused feature map.

[0091] (5) Post-process the fused feature map to obtain the post-processed fused feature map.

[0092] (6) Perform text detection on the post-processed fusion feature map to obtain a text area image.

[0093] (7) Based on the conditional generative adversarial network, the pixels of the adjacent area image of the text area image are extracted.

[0094] (8) Filling the text region image with pixels of the region image adjacent to the text region image to obtain a desensitized image.

[0095] (9) Perform discrete wavelet transform and discrete cosine transform on the desensitized image to obtain the transformed desensitized image.

[0096] (10) Encrypt the text in the text area image and embed it into the transformed desensitized image to obtain a desensitized image with text.

[0097] (11) Perform inverse discrete wavelet transform and inverse discrete cosine transform on the desensitized image with text to obtain the target image.

[0098] (12) Extracting text from the target image to obtain text in the encrypted text area image; decrypting the text in the encrypted text area image to obtain a desensitized image with text.

[0099] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0100] Based on the same inventive concept, embodiments of the present application also provide a medical image sensitive information processing device for implementing the aforementioned medical image sensitive information processing method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more embodiments of the medical image sensitive information processing device provided below can be found in the aforementioned limitations of the medical image sensitive information processing method and will not be further elaborated here.

[0101] In one embodiment, Figure 5 As shown, a medical image sensitive information processing device is provided, comprising: an acquisition module 502, a text detection module 504, a desensitization module 506 and a target image generation module 508, wherein:

[0102] An acquisition module 502 is configured to acquire a medical image to be processed;

[0103] A text detection module 504 is used to perform text detection on the medical image to obtain a text area image;

[0104] Desensitization module 506, configured to fill the text region image with pixels of adjacent region images based on a conditional generative adversarial network to obtain a desensitized image;

[0105] The target image generation module 508 is used to encrypt the text in the text area image and embed it into the desensitized image to obtain the target image.

[0106] In one embodiment, the text detection module 504 is further configured to perform feature extraction on the medical image to obtain a feature map; perform feature fusion on the feature map to obtain a fused feature map; and perform text detection based on the fused feature map to obtain a text area image.

[0107] In one embodiment, the text detection module 504 is further configured to perform feature enhancement on the feature map based on the feature pyramid to obtain an enhanced feature map; and perform feature fusion and dimensionality reduction on the enhanced feature map to obtain a fused feature map.

[0108] In one embodiment, the text detection module 504 is further configured to perform post-processing on the fused feature map to obtain a post-processed fused feature map; and perform text detection on the post-processed fused feature map to obtain a text area image.

[0109] In one embodiment, the desensitization module 506 is used to extract pixels of an image in an area adjacent to the text area image based on a conditional generative adversarial network; and fill the text area image with pixels of an image in an area adjacent to the text area image to obtain a desensitized image.

[0110] In one embodiment, the target image generation module 508 is also used to perform discrete wavelet transform and discrete cosine transform on the desensitized image to obtain a transformed desensitized image; encrypt the text in the text area image and embed it into the transformed desensitized image to obtain a desensitized image with text; perform inverse discrete wavelet transform and inverse discrete cosine transform on the desensitized image with text to obtain a target image.

[0111] In one embodiment, the medical image sensitive information processing device also includes a restoration module for extracting text from the target image to obtain text in the encrypted text area image; and decrypting the text in the encrypted text area image to obtain a desensitized image with text.

[0112] Each module in the aforementioned medical image sensitive information processing device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0113] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for processing sensitive information of medical images is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0114] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0115] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0116] Acquiring a medical image to be processed;

[0117] Perform text detection on medical images to obtain text area images;

[0118] Based on the conditional generative adversarial network, the text area image is filled with pixels of the adjacent area image of the text area image to obtain a desensitized image;

[0119] The text in the text area image is encrypted and embedded into the desensitized image to obtain the target image.

[0120] In one embodiment, when the processor executes the computer program, it also implements the following steps: performing text detection on the medical image to obtain a text area image, including: performing feature extraction on the medical image to obtain a feature map; performing feature fusion on the feature map to obtain a fused feature map; performing text detection based on the fused feature map to obtain a text area image.

[0121] In one embodiment, when the processor executes the computer program, it also implements the following steps: performing feature fusion on the feature map to obtain a fused feature map, including: performing feature enhancement on the feature map based on a feature pyramid to obtain an enhanced feature map; performing feature fusion and dimensionality reduction on the enhanced feature map to obtain a fused feature map.

[0122] In one embodiment, when the processor executes the computer program, it also implements the following steps: performing text detection based on the fused feature map to obtain a text area image, including: post-processing the fused feature map to obtain a post-processed fused feature map; performing text detection on the post-processed fused feature map to obtain a text area image.

[0123] In one embodiment, when the processor executes the computer program, it also implements the following steps: based on a conditional generative adversarial network, filling the text area image according to the pixels of the area image adjacent to the text area image to obtain a desensitized image, including: based on a conditional generative adversarial network, extracting the pixels of the area image adjacent to the text area image; filling the text area image according to the pixels of the area image adjacent to the text area image to obtain a desensitized image.

[0124] In one embodiment, when the processor executes the computer program, it also implements the following steps: encrypting the text in the text area image and embedding it into the desensitized image to obtain the target image, including: performing discrete wavelet transform and discrete cosine transform on the desensitized image to obtain the transformed desensitized image; encrypting the text in the text area image and embedding it into the transformed desensitized image to obtain the desensitized image with text; performing inverse discrete wavelet transform and inverse discrete cosine transform on the desensitized image with text to obtain the target image.

[0125] In one embodiment, when the processor executes the computer program, it further implements the following steps: extracting text from the target image to obtain text in the encrypted text area image; decrypting the text in the encrypted text area image to obtain a desensitized image with text.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0127] Acquiring a medical image to be processed;

[0128] Perform text detection on medical images to obtain text area images;

[0129] Based on the conditional generative adversarial network, the text area image is filled with pixels of the adjacent area image of the text area image to obtain a desensitized image;

[0130] The text in the text area image is encrypted and embedded into the desensitized image to obtain the target image.

[0131] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing text detection on the medical image to obtain a text area image, including: performing feature extraction on the medical image to obtain a feature map; performing feature fusion on the feature map to obtain a fused feature map; performing text detection based on the fused feature map to obtain a text area image.

[0132] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing feature fusion on the feature map to obtain a fused feature map, including: performing feature enhancement on the feature map based on the feature pyramid to obtain an enhanced feature map; performing feature fusion and dimensionality reduction on the enhanced feature map to obtain a fused feature map.

[0133] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing text detection based on the fused feature map to obtain a text area image, including: post-processing the fused feature map to obtain a post-processed fused feature map; performing text detection on the post-processed fused feature map to obtain a text area image.

[0134] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on a conditional generative adversarial network, the text area image is filled according to the pixels of the area image adjacent to the text area image to obtain a desensitized image, including: based on a conditional generative adversarial network, the pixels of the area image adjacent to the text area image are extracted; based on the conditional generative adversarial network, the text area image is filled according to the pixels of the area image adjacent to the text area image to obtain a desensitized image.

[0135] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: encrypting the text in the text area image and embedding it into the desensitized image to obtain the target image, including: performing discrete wavelet transform and discrete cosine transform on the desensitized image to obtain the transformed desensitized image; encrypting the text in the text area image and embedding it into the transformed desensitized image to obtain the desensitized image with text; performing inverse discrete wavelet transform and inverse discrete cosine transform on the desensitized image with text to obtain the target image.

[0136] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: extracting text from the target image to obtain text in the encrypted text area image; decrypting the text in the encrypted text area image to obtain a desensitized image with text.

[0137] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0138] Acquiring a medical image to be processed;

[0139] Perform text detection on medical images to obtain text area images;

[0140] Based on the conditional generative adversarial network, the text area image is filled with pixels of the adjacent area image of the text area image to obtain a desensitized image;

[0141] The text in the text area image is encrypted and embedded into the desensitized image to obtain the target image.

[0142] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing text detection on the medical image to obtain a text area image, including: performing feature extraction on the medical image to obtain a feature map; performing feature fusion on the feature map to obtain a fused feature map; performing text detection based on the fused feature map to obtain a text area image.

[0143] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing feature fusion on the feature map to obtain a fused feature map, including: performing feature enhancement on the feature map based on the feature pyramid to obtain an enhanced feature map; performing feature fusion and dimensionality reduction on the enhanced feature map to obtain a fused feature map.

[0144] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing text detection based on the fused feature map to obtain a text area image, including: post-processing the fused feature map to obtain a post-processed fused feature map; performing text detection on the post-processed fused feature map to obtain a text area image.

[0145] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on a conditional generative adversarial network, the text area image is filled according to the pixels of the area image adjacent to the text area image to obtain a desensitized image, including: based on a conditional generative adversarial network, the pixels of the area image adjacent to the text area image are extracted; based on the conditional generative adversarial network, the text area image is filled according to the pixels of the area image adjacent to the text area image to obtain a desensitized image.

[0146] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: encrypting the text in the text area image and embedding it into the desensitized image to obtain the target image, including: performing discrete wavelet transform and discrete cosine transform on the desensitized image to obtain the transformed desensitized image; encrypting the text in the text area image and embedding it into the transformed desensitized image to obtain the desensitized image with text; performing inverse discrete wavelet transform and inverse discrete cosine transform on the desensitized image with text to obtain the target image.

[0147] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: extracting text from the target image to obtain text in the encrypted text area image; decrypting the text in the encrypted text area image to obtain a desensitized image with text.

[0148] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0149] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0150] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for processing sensitive information of medical images, characterized in that: The method comprises: Acquiring a medical image to be processed; Performing text detection on the medical image to obtain a text area image; Based on a conditional generative adversarial network, the text area image is filled according to pixels of an area image adjacent to the text area image to obtain a desensitized image; Encrypting the text in the text area image and embedding the text into the desensitized image to obtain a target image; Wherein, based on the conditional generative adversarial network, filling the text area image according to the pixels of the adjacent area image of the text area image to obtain the desensitized image includes: Based on the Pix2pix conditional generative adversarial network, the pixels of the image adjacent to the text area image are obtained; The generator based on Pix2pix conditional generative adversarial network uses the Unet structure to fill the text area according to the pixels of the adjacent area of ​​the text area image, and generates a medical image without the text area; Based on the patchGAN discriminator, the medical image and the medical image with text removed are segmented into a preset number of image blocks. The pixels of the segmented image blocks are averaged to obtain the discriminator output. When the discriminator identifies that the text-removed medical image generated by the generator is real, a desensitized image is obtained.

2. The method according to claim 1, characterized in that The performing text detection on the medical image to obtain a text area image includes: performing feature extraction on the medical image to obtain a feature map; Performing feature fusion on the feature map to obtain a fused feature map; Text detection is performed based on the fused feature map to obtain a text area image.

3. The method according to claim 2, characterized in that The performing feature fusion on the feature map to obtain a fused feature map includes: Based on the feature pyramid, feature enhancement is performed on the feature map to obtain an enhanced feature map; Feature fusion and dimensionality reduction are performed on the enhanced feature map to obtain a fused feature map.

4. The method according to claim 2, characterized in that The performing text detection according to the fused feature map to obtain a text area image includes: Post-processing the fused feature map to obtain a post-processed fused feature map; Perform text detection on the post-processed fusion feature map to obtain a text area image.

5. The method according to claim 1, wherein The step of encrypting the text in the text area image and embedding the text in the desensitized image to obtain the target image includes: Performing discrete wavelet transform and discrete cosine transform on the desensitized image to obtain a transformed desensitized image; Encrypting the text in the text area image and embedding the text into the transformed desensitized image to obtain a desensitized image with text; Performing inverse discrete wavelet transform and inverse discrete cosine transform on the desensitized image with text to obtain a target image.

6. The method according to claim 5, characterized in that Also includes: Extracting text from the target image to obtain text in the encrypted text area image; The text in the encrypted text area image is decrypted to obtain a desensitized image with text.

7. A medical image sensitive information processing device, characterized in that: The device comprises: An acquisition module, used for acquiring medical images to be processed; A text detection module, configured to perform text detection on the medical image to obtain a text area image; A desensitization module is configured to fill the text area image with pixels of an area image adjacent to the text area image based on a conditional generative adversarial network to obtain a desensitized image; a target image generation module, configured to encrypt the text in the text area image and embed the text into the desensitized image to obtain a target image; Among them, the desensitization module is also used to obtain pixels of the image area adjacent to the text area image based on the Pix2pix conditional generative adversarial network; the generator based on the Pix2pix conditional generative adversarial network adopts the Unet structure to fill the text area according to the pixels of the image area adjacent to the text area image, and generate a medical image with the text area removed; based on the patchGAN discriminator, the medical image and the medical image with the text area removed are divided into a preset number of image blocks, the pixels of the segmented image blocks are averaged to obtain the output of the discriminator; when the discriminator identifies that the medical image with the text removed generated by the generator is true, a desensitized image is obtained.

8. The device according to claim 7, characterized in that The text detection module is further used to extract features from the medical image to obtain a feature map; perform feature fusion on the feature map to obtain a fused feature map; and perform text detection based on the fused feature map to obtain a text area image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images