Image desensitization method, model training method, device, equipment and storage medium

Abstract

The present disclosure provides an image desensitization method, a model training method, an apparatus, a device and a storage medium, which relate to computer vision, image processing, deep learning, medical imaging, smart medical care, big data and other technical fields in the field of artificial intelligence technology. The image desensitization method includes: obtaining an image to be processed, on which sensitive information is superimposed; encoding the image to be processed by an encoder included in an image reconstruction model to obtain an encoded image corresponding to the image to be processed; decoding the encoded image by a decoder included in the image reconstruction model to restore the non-sensitive information contained in the image to be processed, and obtain a restored image corresponding to the image to be processed, and the image reconstruction model is trained based on a training image without superimposed information. Thus, by reconstructing the non-sensitive information in the image to be processed, the desensitization of the image to be processed is achieved, and the desensitization effect of the image desensitization is improved.

Description

Technical Field

[0001] The present disclosure relates to computer vision, image processing, deep learning, medical imaging, smart healthcare, big data and other technical fields in the field of artificial intelligence technology, and in particular to an image desensitization method, model training method, device, equipment and storage medium. Background Art

[0002] Images often carry sensitive information. For example, medical institutions use explicit and / or implicit watermarking to overlay private patient data, such as name and age, on medical images. While sensitive information in images is not essential for image analysis, transmitting images carrying private user data is also highly sensitive and poses a risk of leaking user privacy. Therefore, image desensitization is necessary.

[0003] In the related technologies of image desensitization, the sensitive information superimposed on the image can be blacked out or coded according to the pre-agreed overprint position of the sensitive information, or the overprint position of the user's privacy obtained through optical character recognition (OCR) or text positioning, thereby achieving image desensitization.

[0004] However, the above desensitization methods have poor desensitization effects. Summary of the Invention

[0005] The present disclosure provides an image desensitization method, model training method, apparatus, device, and storage medium for improving image desensitization effects.

[0006] According to a first aspect of the present disclosure, there is provided an image desensitization method, comprising:

[0007] Acquiring an image to be processed, where sensitive information is superimposed on the image to be processed;

[0008] Encoding the image to be processed by an encoder included in the image reconstruction model to obtain an encoded image corresponding to the image to be processed;

[0009] The encoded image is decoded by a decoder included in the image reconstruction model to restore the non-sensitive information contained in the image to be processed, thereby obtaining a restored image corresponding to the image to be processed. The image reconstruction model is trained based on a training image without superimposed information.

[0010] According to a second aspect of the present disclosure, a model training method is provided, comprising:

[0011] Acquire a training data set, wherein the training data set includes training images without superimposed information;

[0012] The image reconstruction model is trained according to the training data set to obtain a trained image reconstruction model, wherein the image reconstruction model includes an encoder and a decoder, wherein the encoder is used for encoding processing of the image, and the decoder is used for recovering non-sensitive information in the image.

[0013] According to a third aspect of the present disclosure, there is provided an image desensitization device, comprising:

[0014] an acquisition unit, configured to acquire an image to be processed, wherein the image to be processed is superimposed with sensitive information;

[0015] an encoding unit, configured to encode the image to be processed by an encoder included in an image reconstruction model to obtain an encoded image corresponding to the image to be processed;

[0016] A decoding unit is used to decode the encoded image through a decoder included in the image reconstruction model to restore the non-sensitive information contained in the image to be processed and obtain a restored image corresponding to the image to be processed. The image reconstruction model is trained based on a training image without superimposed information.

[0017] According to a fourth aspect of the present disclosure, a model training device is provided, comprising:

[0018] an acquiring unit, configured to acquire a training data set, wherein the training data set includes training images without superimposed information;

[0019] A training unit is used to train an image reconstruction model based on the training data set to obtain a trained image reconstruction model, wherein the image reconstruction model includes an encoder and a decoder, the encoder is used for encoding processing of the image, and the decoder is used for recovering non-sensitive information in the image.

[0020] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the image desensitization method described in the first aspect, or to enable the at least one processor to execute the model training method described in the second aspect.

[0021] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the image desensitization method described in the first aspect, or the computer instructions are used to enable the computer to execute the model training method described in the second aspect.

[0022] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device performs the steps of the image desensitization method described in the first aspect, or the at least one processor executes the computer program so that the electronic device performs the steps of the model training method described in the second aspect.

[0023] According to the technical solution provided by the present disclosure, an image reconstruction model is trained based on a training image without superimposed information. During the training process, the image reconstruction model can learn to reconstruct the image content but will not learn to reconstruct sensitive information. Therefore, the image reconstruction model can restore non-sensitive information in the image but will not restore sensitive information in the image. In the process of image desensitization, the image to be processed is encoded by the encoder included in the image reconstruction model to obtain an encoded image of the image to be processed; the encoded image is decoded by the decoder included in the image reconstruction model to restore the non-sensitive information contained in the image to be processed, and a restored image corresponding to the image to be processed is obtained. In this way, the desensitization of the image to be processed is achieved, and the desensitization effect of the image is improved.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0026] Figure 1 A schematic diagram of an application scenario applicable to the embodiments of the present disclosure;

[0027] Figure 2 A schematic diagram of another application scenario applicable to the embodiments of the present disclosure;

[0028] Figure 3 Schematic diagram of the process of the image desensitization method provided according to the embodiment of the present disclosure Figure 1 ;

[0029] Figure 4 Schematic diagram of the process of the image desensitization method provided according to the embodiment of the present disclosure Figure 2 ;

[0030] Figure 5 Schematic diagram of the process of the image desensitization method provided according to the embodiment of the present disclosure Figure 3 ;

[0031] Figure 6 An example diagram of the noise addition process provided in an embodiment of the present disclosure;

[0032] Figure 7 Schematic diagram of the process of the image desensitization method provided according to the embodiment of the present disclosure Figure 4 ;

[0033] Figure 8 Examples of transmitted images when the desensitization levels provided in the embodiment of the present disclosure are the first and second levels respectively Figure 1 ;

[0034] Figure 9 An example diagram of the reconstruction effect when the desensitization levels provided in the embodiment of the present disclosure are the first level and the second level respectively;

[0035] Figure 10 An example diagram of a restored image when the desensitization levels provided in the embodiment of the present disclosure are respectively the first level and the second level;

[0036] Figure 11 Schematic diagram of the model structure provided in the embodiment of the present disclosure Figure 1 ;

[0037] Figure 12 Schematic diagram of the model structure provided in the embodiment of the present disclosure Figure 2 ;

[0038] Figure 13 The process diagram of the model training method provided according to the embodiment of the present disclosure is as follows Figure 1 ;

[0039] Figure 14 The process diagram of the model training method provided according to the embodiment of the present disclosure is as follows Figure 2 ;

[0040] Figure 15 A schematic diagram of the structure of an image desensitization device provided in an embodiment of the present disclosure;

[0041] Figure 16 A schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure;

[0042] Figure 17 FIG1 is a schematic block diagram of an example electronic device 1700 that may be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0043] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0044] With the continuous development and advancement of medical imaging and computer technology, deep learning (DL) and convolutional neural networks (CNNs) have rapidly become research hotspots for automated medical image analysis. These methods rely on large quantities of medical images to train their models. However, the collection and transmission of medical images are highly sensitive, as they involve patient privacy. This limits the willingness of institutions with large amounts of medical image data to distribute them, and also restricts the efficiency of collaboration between medical institutions and medical imaging algorithm development teams. This ultimately negatively impacts the performance of medical imaging models and wastes data resources.

[0045] Among them, image desensitization processing includes the following aspects:

[0046] 1. Elimination and desensitization of meta information:

[0047] Taking medical images as an example, meta-information in medical images refers to information embedded in the medical image data format but not superimposed on the image content. Typically, meta-information may include archival information such as the patient's name, age, and image acquisition time. This information is highly sensitive but also easily removed using software tools. Therefore, this disclosure does not address the desensitization of this meta-information.

[0048] 2. Desensitizing sensitive information embedded in images

[0049] Sensitive information embedded in an image can be either explicit or implicit. Explicit information refers to information superimposed on an image in the form of text or patterns. Implicit information refers to sensitive information encoded in the image's frequency domain. This information is invisible to the naked eye and can be superimposed or extracted from the image using specific algorithms. A typical example of implicit information is a "hidden watermark" superimposed on an image.

[0050] In addition to artificially added information, the implicit sensitive information mentioned in this disclosure also includes sensitive information naturally carried by the image content and can be distinguished from other medical images. For example, fundus images captured by cameras of different brands and models have different color styles. By counting these styles, people can trace back to the cameras used to capture these fundus images.

[0051] The schemes for desensitizing explicit sensitive information are as follows: 1) Mechanical desensitization scheme, in which the overprint position of sensitive information on the image is pre-agreed, and based on the pre-agreed overprint position, the superimposed text content at that position is erased or blacked out; 2) Desensitization scheme that uses computer vision algorithms, such as Optical Character Recognition (OCR) or text positioning, to identify sensitive information in the image, obtain the location of the sensitive information, and then black out or encrypt the superimposed text content at that location.

[0052] The scheme for desensitizing implicit sensitive information is as follows: 1) Establish an encoder-decoder structure, where the sender of the image data inputs the data to be desensitized into the encoder, obtains the encoding result, transmits the encoding result to the receiver, and uses the decoder at the receiver to decode the encoding result; 2) Based on the adversarial generative model, "conditional image generation" replaces image desensitization. Specifically, the sender transmits "image generation conditions" to the receiver, and the receiver reconstructs the image based on these generation conditions to achieve image desensitization.

[0053] 3. Disadvantages of the above desensitization scheme

[0054] Among the explicit desensitization solutions for sensitive information, mechanical desensitization solutions are suitable for image desensitization within institutions with data access rights, and have limited application scenarios. Solutions based on computer vision algorithms to identify the location of sensitive information will fail to perform desensitization if any characters are missed. Moreover, the above solutions erase information superimposed on the image by blacking out or coding, which will cause loss of image content.

[0055] Among the implicit desensitization schemes for sensitive information, the desensitization scheme based on the encoder-decoder structure is mainly used to desensitize sensor data, or to desensitize model information on images, and has limited desensitization capabilities for sensitive information on images. The desensitization scheme based on the adversarial generative model has a poor desensitization effect and also requires a front-end semantic extractor to extract semantics from the image to assist in image synthesis. The computational complexity is large and the computational efficiency is low, resulting in low desensitization efficiency.

[0056] In order to solve the above-mentioned defects, the present disclosure provides an image desensitization method, which is applied to the technical fields of computer vision, image processing, deep learning, medical imaging, smart medical care, big data, etc. in the field of artificial intelligence technology. In the image desensitization method, the image reconstruction model used for image desensitization is obtained based on training images without superimposed information, so during the training process, the image reconstruction model can learn to reconstruct the image content, but will not learn to reconstruct the information superimposed on the image. In other words, the information superimposed on the image belongs to "out-of-distribution information" for the image reconstruction model and will not be reconstructed, and the image content in the superimposed area will be reconstructed. Using the image reconstruction model for desensitization can restore the image content without restoring the sensitive information superimposed on the image, will not affect the image quality, and there will be no character omission, thereby improving the image desensitization effect. In addition, the image reconstruction model does not require a pre-placed semantic extractor to assist in image reconstruction, thereby improving the efficiency of image desensitization.

[0057] Figure 1 This is a schematic diagram of an application scenario applicable to the embodiment of the present disclosure. In the application scenario, the device involved includes an electronic device for image desensitization, which can be a server or a terminal. Figure 1 Taking the electronic device used for image desensitization as server 101 as an example, server 101 can adopt the image desensitization method provided by the embodiment of the present disclosure on server 101, encode the image to be processed by the encoder in the image reconstruction model to obtain an encoded image, and then decode the encoded image by the decoder in the image reconstruction model to restore the non-sensitive information contained in the image to be processed, and obtain a restored image corresponding to the image to be processed, thereby achieving desensitization of the image to be processed.

[0058] Figure 2 This is a schematic diagram of another application scenario applicable to the embodiment of the present disclosure. In the application scenario, the devices involved include a sending end 201 for sending image data and a receiving end 202 for receiving image data. The sending end 201 and the receiving end 202 can be servers or terminals. Figure 2 Take the example where both the sending end 201 and the receiving end 202 are servers.

[0059] In one embodiment, using the image desensitization method provided by the embodiment of the present disclosure, the sending end 201 can encode the image to be processed through the encoder in the image reconstruction model to obtain an encoded image, and send the encoded image to the receiving end 202. The receiving end 202 decodes the encoded image through the decoder in the image reconstruction model to restore the non-sensitive information contained in the image to be processed, and obtains a restored image corresponding to the image to be processed, thereby realizing desensitization and transmission of the image to be processed.

[0060] In another embodiment, the sending end 201 may encode the image to be processed through the encoder in the image reconstruction model to obtain an encoded image, and then decode the encoded image through the decoder in the image reconstruction model to restore the non-sensitive information contained in the image to be processed, and obtain a restored image corresponding to the image to be processed; thereafter, the sending end 201 may send the restored image to the receiving end 202 to achieve desensitization and transmission of the image to be processed.

[0061] The sending end 201 may be the end where the institution that holds the image data is located, such as a medical institution, physical examination institution, scientific research institute, etc. The receiving end 202 may be the end where the personnel who need the image data are located, such as a research and development team that needs to use the data.

[0062] In the above application scenarios, the server can be a centralized server, a distributed server, or a cloud server. The terminal can be a personal digital assistant (PDA), a handheld device with wireless communication capabilities (such as a smartphone or tablet), a computing device (such as a personal computer (PC)), an in-vehicle device, a wearable device (such as a smart watch or smart bracelet), and a smart home device (such as a smart speaker or smart display device).

[0063] The following specific embodiments describe in detail the technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above-mentioned technical problems. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The following embodiments of the present disclosure are described in conjunction with the accompanying drawings.

[0064] Figure 3 Schematic diagram of the process of the image desensitization method provided according to the embodiment of the present disclosure Figure 1 .like Figure 3 As shown, image desensitization methods include:

[0065] S301: Acquire an image to be processed, where sensitive information is superimposed on the image to be processed.

[0066] The image to be processed is the image to be desensitized. The desensitized information superimposed on the image to be processed can be textual content superimposed on the image to be processed, or it can be sensitive information encoded in the image frequency domain. In other words, the desensitized information superimposed on the image to be processed can be explicit information superimposed on the image to be processed, or it can be implicit information superimposed on the image to be processed.

[0067] In this embodiment, the image to be processed can be obtained from a database; alternatively, the image to be processed can be input by a user; alternatively, the image to be processed can be captured and sent by a camera. Because the image to be processed may contain sensitive information, the acquisition of the image to be processed can occur within an organization with data access rights to ensure that there is no risk of sensitive information being leaked.

[0068] Optionally, the image to be processed may be a medical image to be desensitized. Thus, the disclosed embodiments can be used to desensitize medical images, allowing a large number of medical images that do not contain sensitive information to be used for model training in deep learning and convolutional neural networks, thereby improving model performance in the medical imaging field and promoting the further development of medical imaging technology.

[0069] S302 , encoding the image to be processed by an encoder included in the image reconstruction model to obtain an encoded image corresponding to the image to be processed.

[0070] The image reconstruction model includes an encoder and a decoder. The encoder is used to encode the image, and the decoder is used to decode the encoded image obtained after the encoding process.

[0071] In this embodiment, the image to be processed may be input into an encoder included in the image reconstruction model, and the image to be processed may be encoded in the encoder to obtain an encoded image corresponding to the image to be processed.

[0072] S303, decoding the encoded image through a decoder included in the image reconstruction model to restore non-sensitive information contained in the image to be processed, and obtain a restored image corresponding to the image to be processed. The image reconstruction model is trained based on a training image without superimposed information.

[0073] The image reconstruction model is trained on training images without superimposed information, allowing it to learn to reconstruct the image content. However, the superimposed information on the image is equivalent to out-of-distribution information, which the image reconstruction model does not learn to reconstruct. Therefore, during the image reconstruction process, the image reconstruction model can recover the image content, that is, it can recover the non-sensitive information on the image, but cannot recover the superimposed information on the image, that is, it cannot recover the sensitive information superimposed on the image.

[0074] The non-sensitive information contained in the image to be processed refers to the information on the image to be processed except for the superimposed sensitive information, that is, the image content on the image to be processed.

[0075] Among them, the restored image corresponding to the image to be processed is the reconstructed image corresponding to the image to be processed. Since compared with the image to be processed, the reconstructed image contains non-sensitive information on the image to be processed but does not contain sensitive information on the image to be processed, the reconstructed image and the image to be processed are similar images.

[0076] In this embodiment, the encoded image can be input into a decoder for decoding, or the encoded image can be further processed and then input into a decoder for decoding. The decoding process can then recover the non-sensitive information contained in the image to be processed, yielding a restored image corresponding to the image to be processed. Thus, the image reconstruction model desensitizes the image to be processed while ensuring the similarity between the restored image and the image to be processed. The desensitization process preserves the image content of the image to be processed, improving the quality of the desensitized image.

[0077] In the disclosed embodiments, an image reconstruction model trained on training images without overlaying information is used for image desensitization. This model can restore the image content without restoring sensitive information overlaid on the image, without compromising image quality or missing characters, thereby improving the image desensitization effect. The image reconstruction model does not require a pre-processed semantic extractor to assist in image reconstruction, thus improving image desensitization efficiency.

[0078] The above steps can be performed on the same device or on different devices.

[0079] In some embodiments, the encoder included in the image reconstruction model is deployed at the transmitting end, and the decoder of the image reconstruction model is deployed at the receiving end. Figure 4 Schematic diagram of the process of the image desensitization method provided according to the embodiment of the present disclosure Figure 2 .like Figure 4 As shown, image desensitization methods include:

[0080] S401: The sending end obtains an image to be processed, on which sensitive information is superimposed.

[0081] S402: The sending end encodes the image to be processed by an encoder included in the image reconstruction model to obtain an encoded image corresponding to the image to be processed.

[0082] S403: The transmitting end sends the encoded image to the receiving end.

[0083] The sending end may send data to the receiving end via a wired or wireless method.

[0084] S404: The receiving end decodes the encoded image through a decoder included in the image reconstruction model to restore the non-sensitive information contained in the image to be processed and obtain a restored image corresponding to the image to be processed. The image reconstruction model is trained based on the training image without superimposed information.

[0085] The implementation principles and technical effects of S401 to S404 may refer to the aforementioned embodiments and will not be described in detail.

[0086] In the embodiments of the present disclosure, in scenarios involving image transmission, to ensure that sensitive information in the image is not leaked, an encoder included in an image reconstruction model can be deployed on the transmitting end, and a decoder included in the same image reconstruction model can be deployed on the receiving end. The image reconstruction model is used to recover non-sensitive information contained in the image to be processed. During the image transmission process, the transmitting end sends the encoded image output by the encoder to the receiving end, and the receiving end decodes the encoded image through the decoder to obtain a recovered image that does not contain sensitive information. Since the encoded image is transmitted, even if an unauthorized third party obtains the encoded image during the transmission process, it is difficult to recover the image because the corresponding decoder is not deployed, thereby improving the security of image transmission.

[0087] In some embodiments, during the image desensitization process, the image reconstruction model and the image denoising and denoising method can be combined to improve the desensitization intensity of the image and improve the image desensitization effect.

[0088] Figure 5 Schematic diagram of the process of the image desensitization method provided according to the embodiment of the present disclosure Figure 3 .like Figure 5 As shown, image desensitization methods include:

[0089] S501: Acquire an image to be processed, where sensitive information is superimposed on the image to be processed.

[0090] S502 , encoding the image to be processed by an encoder included in the image reconstruction model to obtain an encoded image corresponding to the image to be processed.

[0091] The implementation principles and technical effects of S501 to S502 may refer to the aforementioned embodiments and will not be described in detail.

[0092] S503: Add noise to the coded image to obtain a noisy image.

[0093] In this embodiment, after obtaining the encoded image, noise is added to the encoded image to further increase the difficulty of restoring the image to be processed by adding noise, especially to increase the difficulty of restoring sensitive information on the image to be processed, thereby improving the desensitization strength of the image to be processed.

[0094] In one possible implementation, random noise may be added to the encoded image to improve the noise addition effect.

[0095] Furthermore, random Gaussian noise may be added to the encoded image.

[0096] S504: performing noise estimation on the noisy image using a noise estimation model to obtain an estimated noise value.

[0097] The noise estimation model may be a neural network model, and the noise estimation model may be obtained by pre-training.

[0098] In this embodiment, the noisy image can be input into the noise estimation model, and features of the noisy image can be extracted based on the noise estimation model. Noise estimation is performed based on the extracted image features. That is, the noise added to the noisy image is predicted by the noise estimation model to obtain an estimated noise value corresponding to the noisy image. The estimated noise value is the predicted noise value.

[0099] S505: De-noise the noisy image according to the estimated noise value to obtain a de-noised image.

[0100] The data dimension of the estimated noise value may be the same as the data dimension of the image matrix of the noisy image.

[0101] In this embodiment, since the estimated noise value data is in matrix form, the estimated noise value can be removed from the noisy image through matrix operations to obtain a denoised image. Because the estimated noise value is similar to the actual noise value added to the denoised image, the denoised image and the encoded image are similar. This method of adding noise to the encoded image, estimating the noise, and then further denoising the encoded image can interfere with features related to sensitive information in the encoded image, thereby improving the image desensitization effect.

[0102] S506: Input the denoised image into a decoder included in an image reconstruction model for decoding processing to restore non-sensitive information included in the image to be processed and obtain a restored image. The image reconstruction model is trained based on a training image without superimposed information.

[0103] In this embodiment, after the denoised image is obtained, the denoised image is input into a decoder included in the image reconstruction model for decoding processing, so that the image is restored through decoding.

[0104] In the disclosed embodiment, the image reconstruction model does not learn the reconstruction of superimposed information on the image during the training process, and the image reconstruction model has very weak recovery capabilities for superimposed information on the image, that is, the image reconstruction model has very weak recovery capabilities for sensitive information superimposed on the image. In the case where the image reconstruction model has very weak recovery capabilities for sensitive information superimposed on the image, by adding noise to the encoded image, estimating the noise, and then further denoising it, interference is generated on the features on the encoded image, including interference on features on the encoded image related to sensitive information, further increasing the difficulty of recovering the sensitive information superimposed on the image. In this way, the image desensitization strength is effectively improved, and the image desensitization effect is improved. In addition, through noise estimation and denoising, the similarity between the denoised image and the encoded image is ensured. In the case where the image reconstruction image has a strong recovery capability for non-sensitive information contained in the image to be processed, based on the denoised image with a high similarity to the encoded image, a restored image with a high similarity to the image to be processed can be accurately restored, thereby improving the image quality after the image desensitization process.

[0105] Below, we provide Figure 5 Possible implementations of multiple steps in the illustrated embodiment.

[0106] In some embodiments, Figure 5 In the illustrated embodiment, in addition to using the encoder in the image reconstruction model for lossy encoding, the image to be processed can also be encoded using other lossy encoding methods, or a lossless encoding method can be used to encode the image to be processed. Subsequently, the denoised image is decoded using a decoding method corresponding to the encoding method, thereby improving the flexibility of encoding and decoding.

[0107] Among them, lossy coding means that there is information loss in the encoding process of the image, and the difference between the reconstructed result and the original image is visible to the naked eye; lossless coding means that the encoding process of the image follows the Shannon source coding theorem, the compression rate is close to the information entropy of the data, and there is almost no information loss.

[0108] For example, other lossy encoding methods may be a Joint Photographic Experts Group (JPEG) compression method; and for another example, lossless encoding methods may be a Portable Network Graphics (PNG) compression method or a Huffman Coding method.

[0109] In some embodiments, a possible implementation of S503 includes: determining a number of noise addition steps corresponding to the encoded image; and adding multiple steps of noise to the encoded image corresponding to the image to be processed according to the number of noise addition steps to obtain a noisy image. In this manner, by adding multiple steps of noise, the randomness of the noise is increased, thereby improving the noise addition effect and the image desensitization effect.

[0110] In this implementation, the number of noise addition steps corresponding to the encoded image can be randomly determined; alternatively, the number of noise addition steps corresponding to the encoded image can be pre-set, for example, by a professional. Based on the number of noise addition steps, multiple steps of random noise can be added to the encoded image corresponding to the image to be processed. For example, if the number of noise addition steps is t1, then t1 steps of random noise can be added to the encoded image corresponding to the image to be processed. Compared to single-step random noise, multi-step random noise can improve the randomness of the noise. The multi-step noise can be added step by step, or a multi-step noise addition formula can be derived, and the multi-step noise can be added all at once based on the formula, thereby improving the efficiency of adding multi-step noise to the encoded image.

[0111] In one possible implementation of determining the number of denoising steps corresponding to a coded image, the number of denoising steps corresponding to the coded image is determined based on a step threshold corresponding to a noise estimation model; wherein the number of denoising steps is less than or equal to the step threshold. Thus, the number of denoising steps is determined within the capability of the noise estimation model, ensuring the accuracy of noise estimation and improving the denoising effect, thereby increasing the similarity between the denoised image and the coded image, and increasing the similarity between the reconstructed restored image and the processed image.

[0112] In this implementation, a step threshold corresponding to the noise estimation model can be output to prompt the user to select a number of noise addition steps within the step threshold range. The user's input of the number of noise addition steps is then received. Alternatively, the number of noise addition steps corresponding to the encoded image can be randomly determined within the step threshold range corresponding to the noise estimation model, further enhancing the randomness of the noise addition and the noise addition effect.

[0113] In one possible implementation, when adding multi-step noise to the coded image corresponding to the image to be processed, the noise intensity increases as the number of steps increases. For example, when adding t1 steps of noise to the coded image, the noise intensity gradually increases from step 0, with the noise added at step t1 being the highest. In this way, the multi-step noise added to the coded image corresponding to the image to be processed closely resembles a sampling result from a random Gaussian distribution.

[0114] In the process of adding multi-step noise to a coded image corresponding to a processed image according to a number of noise addition steps to obtain a noisy image, one possible implementation method includes adding multi-step random Gaussian noise to the coded image according to the number of noise addition steps and a Gaussian distribution function to obtain the noisy image. Thus, utilizing the Gaussian distribution function ensures that the noise added to the coded image is both random and conforms to the law of Gaussian distribution, thereby improving the efficiency of adding multi-step noise to the coded image.

[0115] In this implementation, a noise addition formula corresponding to the coded image may be determined according to the number of noise addition steps and the Gaussian distribution function, and multiple steps of random noise may be added to the coded image according to the noise addition formula to obtain a noisy image.

[0116] Furthermore, the noise addition formula is as follows:

[0117] Among them, x t1 ~q(x t1 |x0) represents the probability distribution q(x t1 |x0) sampling to get x t1 , x t1 It represents the noisy image obtained by adding t1 steps of random Gaussian noise to the original image x0 (here refers to the encoded image corresponding to the image to be processed); Indicates the mean The variance is Gaussian distribution, is a random sample from a standard Gaussian distribution, β i is a series of fixed values, i∈[1, t1], β i It can be calculated by a manually defined formula. For example, when t1 = 1000, 1000 values ​​can be taken from a to b at equal intervals as β i . a and b are constants, and a is smaller than b.

[0118] It can be seen that the above noise addition formula can directly add multi-step random Gaussian noise to the encoded image, which effectively improves the noise addition efficiency and thus improves the image desensitization efficiency.

[0119] In some embodiments, when the noise added to the encoded image is multi-step noise, the estimated noise value obtained by the noise estimation model includes the estimated noise value corresponding to each step in the number of noise addition steps. A possible implementation of S504 includes: estimating the multi-step noise added to the noisy image using the noise estimation model to obtain the estimated noise value corresponding to each step in the number of noise addition steps; and a possible implementation of S505 includes: performing stepwise denoising on the noisy image according to the estimated noise value corresponding to each step in the number of noise addition steps to obtain a denoised image. In this way, by estimating the noise added at each step and performing stepwise denoising, denoising accuracy is improved, thereby improving the similarity between the denoised image and the encoded image.

[0120] In some embodiments, another possible implementation of S504 includes: obtaining attribute information of the image to be processed; and inputting the attribute information of the image to be processed and the noisy image into a noise estimation model to perform noise estimation and obtain an estimated noise value. Thus, the attribute information of the image to be processed is input into the noise estimation model along with the noisy image as a priori condition for noise estimation, thereby improving the accuracy of noise estimation. This further improves the similarity between the denoised image and the encoded image, the similarity between the restored image and the image to be processed, and the image reconstruction capability of the image reconstruction model.

[0121] In this implementation, input data for the noise estimation model can be obtained based on the attribute information of the image to be processed and the noisy image. For example, the input vector for the noise estimation model can be obtained by encoding the attribute information of the image to be processed and the noisy image. Because the input data carries features related to the attribute information of the image to be processed, the noise estimation process is aided by prior conditions, compared to an approach where only the noisy image is input. Based on the input data, the noise estimation model can more accurately estimate the noise added to the noisy image, thereby obtaining an estimated noise value.

[0122] Furthermore, when the noise added to the encoded image is multi-step noise, the attribute information of the image to be processed and the noisy image are input into the noise estimation model for noise estimation, and the estimated noise value corresponding to each step in the noisy step number can be obtained.

[0123] Furthermore, the attribute information of the image to be processed may include the image type and / or semantic information of the image to be processed. Thus, the image type and semantic information on the image are used as prior conditions for noise estimation, further improving the accuracy of noise estimation.

[0124] When the image to be processed is a medical image, the image type of the image to be processed can be classified according to image content, or according to disease type or lesion level. For example, the image type of the image to be processed is fundus image or fundus lesion level 2 image.

[0125] When the image to be processed is a medical image, the semantic information of the image to be processed may include semantic features related to the disease and the body part.

[0126] When the attribute information of the image to be processed includes the image type of the image to be processed and the semantic information of the image to be processed, the semantic information of the image to be processed can be obtained by performing semantic feature extraction on the image to be processed based on the image type of the image to be processed, so as to improve the accuracy of the semantic information of the image to be processed.

[0127] In some embodiments, based on the attribute information of the image to be processed, which may include the image type of the image to be processed and / or the semantic information of the image to be processed, obtaining the attribute information of the image to be processed may include: obtaining the image type of the image to be processed; and / or, inputting the image to be processed into a feature extraction model for feature extraction to obtain image features corresponding to the image to be processed, inputting the image features into a feature encoding model for feature encoding to obtain the semantic information of the image to be processed.

[0128] In this embodiment, if the attribute information of the image to be processed may include the image type of the image to be processed, the image type of the image to be processed may be obtained from the annotation information of the image to be processed, or the image type input by the user for the image to be processed may be obtained; or, the image type of the image to be processed may be obtained by performing type recognition on the image to be processed. If the attribute information of the image to be processed may include semantic information of the image to be processed, the image to be processed may be input into a feature extraction model to extract semantic features, thereby obtaining image features corresponding to the image to be processed, wherein the image features are related to the semantic information of the image to be processed; the image features may be input into a feature encoding model for feature encoding, thereby obtaining an encoding vector of the image features, wherein the encoding vector is the semantic information extracted from the image to be processed.

[0129] Furthermore, the feature extraction model may be an image classification, detection and / or segmentation model.

[0130] Furthermore, when the attribute information of the image to be processed includes the image type and semantic information of the image to be processed, the image to be processed can be input into a feature extraction model corresponding to the image type of the image to be processed to extract semantic features and obtain image features of the image to be processed. The feature extraction model corresponding to the image type of the image to be processed is trained using training images corresponding to the image type of the image to be processed. Thus, semantic features of the image to be processed are extracted based on the image type of the image to be processed, improving the accuracy of extracting the semantic information of the image to be processed.

[0131] As an example, taking the image to be processed as a medical image, when the image type of the image to be processed is a fundus image, the image to be processed can be input into a fundus image segmentation model and a macular lesion assessment model to extract semantic features.

[0132] In some embodiments, based on any of the foregoing embodiments, the image reconstruction model may employ a variational autoencoder. This is a generative deep learning model with an encoder-decoder structure. The encoder encodes input data as parameter variables of a probability distribution, and the decoder reconstructs the input data based on this probability distribution. The variational autoencoder can improve the encoding and reconstruction of the processed image, thereby enhancing the image quality after desensitization.

[0133] In some embodiments, based on any of the foregoing embodiments, the noise estimation model may adopt a U-shaped neural network to improve the noise estimation accuracy by using the U-shaped neural network.

[0134] In some embodiments, based on any of the aforementioned embodiments, a diffusion model is used for noise addition and noise estimation. The diffusion model includes a forward step and a backward step: in the forward step, the diffusion model adds multiple steps of noise to the input data. In the backward step, a U-shaped neural network is used as a noise estimation model to iteratively estimate the noise added at each step.

[0135] As an example, Figure 6 This is an example diagram of the noise adding process provided by the embodiment of the present disclosure. In the process of adding noise by the diffusion model, the original image x0( Figure 6 Take the puppy image as an example) the first step of noise addition is to get x1, the second step of noise addition is to get x2, ..., and finally the Nth step of noise addition is to get x N The image changes gradually from clear image content to almost indistinguishable image content.

[0136] In some embodiments, the encoder included in the image reconstruction model is deployed at the transmitting end, and the decoder and noise estimation model included in the image reconstruction model are deployed at the receiving end. Thus, image encoding and image noise addition are performed at the transmitting end, and image denoising and image decoding are performed at the receiving end, which improves the image desensitization effect and improves the security of image transmission. Based on this, Figure 7 Schematic diagram of the process of the image desensitization method provided according to the embodiment of the present disclosure Figure 4 .like Figure 7 As shown, image desensitization methods include:

[0137] S701: The sending end obtains an image to be processed, on which sensitive information is superimposed.

[0138] S702 : The sending end encodes the image to be processed by using an encoder included in the image reconstruction model to obtain an encoded image corresponding to the image to be processed.

[0139] S703: The transmitting end adds noise to the coded image to obtain a noisy image.

[0140] S704: The transmitting end sends the noisy image to the receiving end.

[0141] S705: The receiving end performs noise estimation on the noisy image using a noise estimation model to obtain an estimated noise value.

[0142] S706: The receiving end denoises the noisy image according to the estimated noise value to obtain a denoised image.

[0143] S707: The receiving end inputs the denoised image into a decoder for decoding to restore non-sensitive information contained in the image to be processed, thereby obtaining a restored image.

[0144] The implementation principles and technical effects of S701 to S707 may refer to the aforementioned embodiments and will not be described in detail.

[0145] In some embodiments, when the noise added to the encoded image is multi-step noise, the transmitter and receiver can pre-agreed on the number of noise addition steps corresponding to the image to be processed; alternatively, the transmitter can send the number of noise addition steps corresponding to the image to be processed to the receiver. This number of noise addition steps is thus known only to the transmitter and receiver, improving the security of image transmission during image desensitization.

[0146] In some embodiments, when the noise estimation model uses attribute information of the image to be processed as a priori conditions for noise estimation, the transmitting end may send the attribute information of the image to be processed to the receiving end, thereby facilitating the receiving end to perform noise estimation based on the attribute information of the image to be processed.

[0147] In some embodiments, a corresponding desensitization method can be selected according to the desensitization level of the image to be processed to improve the flexibility and diversity of image desensitization, and different desensitization levels and then different desensitization methods can be adopted to adapt to different desensitization needs of users.

[0148] In one possible implementation, when it is determined that the desensitization level of the image to be processed is the first level, the coded image corresponding to the image to be processed may be denoised to obtain a noisy image, and then the noise of the noisy image may be estimated to obtain an estimated noise value, and the noisy image may be denoised according to the estimated noise value to obtain a denoised image, and then the denoised image may be decoded by the decoder in the image reconstruction model to obtain a restored image. In other words, when it is determined that the desensitization level of the image to be processed is the first level, the image may be denoised according to the first level. Figure 5 and Figure 7 The embodiment shown performs image desensitization. Thus, through encoding, denoising, denoising and decoding, higher intensity image desensitization is achieved, thereby improving the image desensitization effect.

[0149] In another possible implementation, when it is determined that the desensitization level of the image to be processed is the second level, the encoded image corresponding to the image to be processed can be input into the decoder of the image reconstruction model for decoding to restore the non-sensitive information contained in the image to be processed and obtain a restored image. In other words, when it is determined that the desensitization level of the image to be processed is the first level, the image to be processed can be restored according to the following method: Figure 3 and Figure 4 The embodiment shown performs image desensitization. Although the desensitization effect of this method is weaker than the desensitization effect of the first level desensitization method, the image reconstruction effect of this method is better.

[0150] Therefore, the embodiment of the present disclosure can provide two desensitization levels, and users can choose the desensitization level according to different needs. For example, if the sender wants a stronger desensitization intensity, use the first level desensitization solution, otherwise use the second level desensitization solution.

[0151] As an example, Figure 8 Examples of transmitted images when the desensitization levels provided in the embodiment of the present disclosure are the first and second levels respectively Figure 1 .like Figure 8 As shown in the figure, the original image, the transmission image when the desensitization level is the first level, and the transmission image when the desensitization level is the second level are given. The transmission image when the desensitization level is the first level is the noise image obtained by adding noise to the coded image corresponding to the image to be processed, and the transmission image when the desensitization level is the second level is the coded image corresponding to the image to be processed. Figure 8 It can be seen that the transmitted image when the desensitization level is the first level is more blurred than the transmitted image when the desensitization level is the second level, and the image content of the original image is even less discernible, thereby improving the security of image transmission.

[0152] As an example, Figure 9 This is an example diagram of the reconstruction effect when the desensitization levels provided in the embodiment of the present disclosure are the first level and the second level respectively. Figure 9 As shown in the figure, the original image, the restored image when the desensitization level is the first level, the reconstruction error when the desensitization level is the first level, the restored image when the desensitization level is the second level, and the reconstruction error when the desensitization level is the second level are shown. In the reconstruction error, the horizontal axis represents the variance of the error, and the vertical axis represents the number of pixels. Figure 9 The values ​​of the horizontal and vertical axes are just examples and will not be described here one by one. Figure 9 It can be seen that in the image desensitized using the desensitization method corresponding to the second level, the optic disc structure and some vascular information of the fundus image can still be observed, and the fundus image outline is also clearly visible. However, in the image desensitized using the desensitization method corresponding to the first level, it is difficult to observe any valid information except for the blurred fundus image outline. Therefore, the restoration degree of the restored image with the second desensitization level is higher than that of the restored image with the first desensitization level, and the reconstruction error with the second desensitization level is lower than that with the first desensitization level.

[0153] As an example, Figure 10 This is an example of a restored image when the desensitization levels provided in the embodiment of the present disclosure are the first level and the second level respectively. Figure 10As shown, the desensitization effect of the explicit watermark and the desensitization effect of the implicit watermark are shown. In the desensitization effect of the explicit watermark, the original fundus image a superimposed with the explicit watermark, the restored image obtained by desensitizing the original fundus image a by the desensitization method corresponding to the first level, the restored image obtained by desensitizing the original fundus image a by the desensitization method corresponding to the second level, the original fundus image b carrying the implicit watermark, the restored image obtained by desensitizing the original fundus image b by the desensitization method corresponding to the first level, the restored image obtained by desensitizing the original fundus image b by the desensitization method corresponding to the second level, the implicit watermark in the original fundus image b, the data after the implicit watermark is desensitized by the desensitization method corresponding to the first level, and the data after the implicit watermark is desensitized by the desensitization method corresponding to the second level.

[0154] As an example, Figure 11 Schematic diagram of the model structure provided in the embodiment of the present disclosure Figure 1 . Take the image to be processed as a medical image as an example, Figure 11 As shown, the model involved in the image desensitization method includes a diffusion model and an automatic variational encoder (equivalent to the image reconstruction model in the aforementioned embodiment), and the diffusion model includes a noise estimator (equivalent to the noise estimation model in the aforementioned embodiment). The variational autoencoder includes an encoder and a decoder. L1 represents the second-level desensitization route, and L2 represents the first-level desensitization route. The specific desensitization process is as follows:

[0155] like Figure 11 As shown in the figure, the image to be processed is the fundus image x. The sending end inputs the fundus image x into the encoder included in the variational autoencoder for encoding to obtain the encoded image. If it is the second level of desensitization, the encoded image is superimposed with T-step Gaussian noise through the diffusion model to obtain a noisy image. After that, the noisy image is sent to the receiver. The receiver inputs the noisy image into the noise estimator for noise estimation and then performs denoising. The denoised image is input into the decoder included in the variational autoencoder to obtain the restored image corresponding to the fundus image x. If it is the first level of desensitization, the encoded image is sent to the receiver, and the receiver inputs the encoded image into the decoder included in the variational autoencoder to obtain the restored image corresponding to the fundus image x

[0156] As an example, Figure 12 Schematic diagram of the model structure provided in the embodiment of the present disclosure Figure 2 . Take the image to be processed as a medical image as an example, Figure 12As shown, the models involved in the image desensitization method include an image feature extractor, an image feature encoder, a diffusion model and an automatic variational encoder (equivalent to the image reconstruction model in the aforementioned embodiment). Among them, the image feature extractor includes feature extraction models corresponding to different image types (equivalent to the feature extraction model in the aforementioned embodiment), such as a fundus image segmentation model and a macular lesion assessment model; the image feature encoder includes an image feature encoding model (equivalent to the feature encoding model in the aforementioned embodiment); the diffusion model includes a noise estimator (equivalent to the noise estimation model in the aforementioned embodiment); and the variational autoencoder includes an encoder and a decoder. L1 represents the second-level desensitization route, and L2 represents the first-level desensitization route. The specific desensitization process is as follows:

[0157] like Figure 12 As shown, in the second level desensitization process, the image to be processed is a fundus image x, and the fundus image x is input into the image feature extractor. In the image feature extractor, semantic features are extracted through the fundus image segmentation model and the macular lesion assessment model to obtain the output data of the image feature extractor; the output data of the image feature extractor is input into the image feature encoder for feature encoding to obtain an image feature encoding vector (equivalent to the encoding vector and semantic information in the aforementioned embodiment); the fundus image x is input into the encoder included in the variational autoencoder to obtain an encoded image; and the encoded image is superimposed with T steps of Gaussian noise through the diffusion model to obtain the noise data x r ; Afterwards, the noise data x r The image feature encoding vector is input into the noise estimator for noise estimation, and then denoising is performed. The denoised image is input into the decoder to obtain the restored image corresponding to the fundus image x.

[0158] like Figure 12 As shown in the first level of desensitization, the fundus image x is input into the encoder included in the variational autoencoder to obtain an encoded image; then, the encoded image is input into the decoder to obtain the restored image corresponding to the fundus image x.

[0159] Below, an example of training a model involved in the image desensitization method is provided. It should be noted that model training and image desensitization can be performed on the same device or on different devices.

[0160] Figure 13 The process diagram of the model training method provided according to the embodiment of the present disclosure is as follows Figure 1 .like Figure 13 As shown, the model training method includes:

[0161] S1301: Acquire a training data set, where the training data set includes training images without superimposed information.

[0162] In this embodiment, a pre-collected training data set may be obtained from a database. In order to enable the trained image reconstruction model to learn the reconstruction of image content and not the reconstruction of information superimposed on the image, the training images in the training data set are required to have no superimposed information.

[0163] S1302: Train the image reconstruction model according to the training data set to obtain a trained image reconstruction model. The image reconstruction model includes an encoder and a decoder. The encoder is used for encoding the image, and the decoder is used for recovering non-sensitive information in the image.

[0164] In this embodiment, the image reconstruction model can be supervised multiple times based on the training data set to obtain a trained image reconstruction model. The i-th training process of the image reconstruction model may include: obtaining a training image for the i-th training from the training data set; inputting the training image for the i-th training into an encoder included in the image reconstruction model for encoding processing to obtain an encoded image corresponding to the training image for the i-th training; inputting the encoded image corresponding to the training image for the i-th training into a decoder included in the image reconstruction model for decoding processing to obtain a restored image corresponding to the training image for the i-th training; adjusting model parameters of the image reconstruction model based on the training image for the i-th training and the restored image corresponding to the training image for the i-th training to obtain the image reconstruction model after the i-th training.

[0165] In which, in the process of adjusting the model parameters of the image reconstruction model according to the training image used in the i-th training and the restored image corresponding to the training image used in the i-th training, the reconstruction loss of the i-th training can be determined according to the training image used in the i-th training and the restored image corresponding to the training image used in the i-th training, and the model parameters of the image reconstruction model can be adjusted according to the reconstruction loss.

[0166] In the disclosed embodiments, an image reconstruction model for recovering non-sensitive information in an image is trained based on training images without information superimposed thereon. This image reconstruction model can be applied to the image desensitization method provided in any of the aforementioned embodiments, improving the image desensitization effect and the quality of the desensitized image. This image reconstruction model does not require a pre-processed semantic extractor, thereby improving the efficiency of image desensitization.

[0167] Figure 14 The process diagram of the model training method provided according to the embodiment of the present disclosure is as follows Figure 2 .like Figure 14 As shown, the model training method includes:

[0168] S1401, obtaining a training data set, where the training data set includes training images without superimposed information;

[0169] S1402: Train the image reconstruction model according to the training data set to obtain a trained image reconstruction model. The image reconstruction model includes an encoder and a decoder. The encoder is used for encoding the image, and the decoder is used for recovering non-sensitive information in the image.

[0170] The implementation principles and technical effects of S1401 to S1402 may refer to the aforementioned embodiments and will not be described in detail.

[0171] S1403: Train the noise estimation model according to the training image and the trained image reconstruction model to obtain the trained noise estimation model.

[0172] In this embodiment, the training image can be encoded using an encoder included in the trained image reconstruction model to obtain an encoded image corresponding to the training image. Noise is then added to the encoded image to obtain training samples for the noise estimation model. The noise estimation model is then trained based on the training samples to obtain a noise estimation model. Thus, the noise estimation model is trained using the trained image reconstruction model, improving the noise estimation performance of the noise estimation model for images encoded and then noisy by the image reconstruction model.

[0173] In the disclosed embodiments, the image reconstruction model and noise estimation model are trained separately: the image reconstruction model is trained first, and then the noise estimation model is trained using the trained image reconstruction model. This improves the image reconstruction model's ability to recover non-sensitive information from the image using training images without superimposed information, while also improving the training effectiveness of the noise estimation model using the image reconstruction model.

[0174] In some embodiments, the noise estimation model is trained multiple times, wherein the i-th training process of the noise estimation model includes: encoding a training image using an encoder in a trained image reconstruction model to obtain an encoded image corresponding to the training image; adding an actual noise value to the encoded image to obtain a noisy image; estimating noise on the noisy image using the noise estimation model to obtain an estimated noise value; and adjusting model parameters of the noise estimation model based on the actual noise value and the estimated noise value to obtain the i-th trained noise estimation model. Thus, utilizing the trained image model and supervised training methods can improve the training effect of the noise estimation model.

[0175] In this embodiment, after adding the actual noise value to the encoded image to obtain the noisy image, the noise image can be input into the noise estimation model for noise estimation to obtain the estimated noise value; based on the difference between the actual noise value and the estimated noise value, the training loss value of the noise estimation model is determined; based on the training loss value, the parameters of the noise estimation model are adjusted to obtain the noise estimation model after the i-th training.

[0176] In the process of adding an actual noise value to a coded image corresponding to a training image to obtain a noisy image, one possible implementation includes: determining a number of noise addition steps corresponding to the coded image; and adding multiple steps of noise to the coded image according to the number of noise addition steps to obtain the noisy image. The process of adding multiple steps of noise to the coded image can be referred to the description of the previous embodiment and will not be repeated here.

[0177] Furthermore, the multi-step noise may be multi-step random Gaussian noise.

[0178] In a possible implementation, the number of noise adding steps corresponding to the encoded image may be preset; or the number of noise adding steps corresponding to the encoded image may be received and input by a user.

[0179] In another possible implementation, the number of noise addition steps corresponding to the coded image corresponding to the training image can be randomly determined based on the step threshold corresponding to the noise estimation model. Through multiple training sessions, the distribution of the randomly determined number of noise addition steps can cover the range specified by the step threshold. For example, if the step threshold is T and the number of randomly determined noise addition steps is t2 each time, after multiple training sessions, the values ​​of t2 will cover the range from 1 to T. This eliminates the need to add noise equal to the step threshold to the coded image during each training session, improving training efficiency and ensuring that the noise estimation model maintains the noise estimation effect on loaded images with noise added at different step numbers.

[0180] In one possible implementation, during training, the noise estimation model can be used to predict the noise added to the noisy image from the previous noisy step to the next noisy step, obtaining an estimated noise value from the previous noisy step to the next noisy step. Based on the estimated noise value from the previous noisy step to the next noisy step and the actual noise value from the previous noisy step to the next noisy step, the model parameters of the noise estimation model are adjusted to obtain a noise estimation model after the i-th training. This indicates that the noise estimation model only needs to perform a single-step noise estimation during a single training process. Since the number of noisy steps can be randomly selected within a step threshold, the single-step noise estimated by the noise estimation model during multiple training sessions can cover the range of the step threshold. This improves the training efficiency of the noise estimation model while also ensuring the noise estimation effect of the trained noise estimation model.

[0181] The noise addition formula for adding multi-step noise of step t2 to the noise estimation model can be expressed as:

[0182]

[0183] Among them, x t2 ~q(x t2 |x0) represents the probability distribution q(x t2 |x0) sampling to get x t2 , x t2represents the noisy image obtained by adding t2 steps of random Gaussian noise to the original image x0 (here refers to the encoded image corresponding to the training image); Indicates the mean The variance is Gaussian distribution, is a random sample from a standard Gaussian distribution, β i is a series of fixed values, i∈[1, T], β i It can be calculated by a manually defined formula, where T is the step threshold. For example, when t1 = 1000, 1000 values ​​can be taken from a to b at equal intervals as β i . a and b are constants, and a is smaller than b.

[0184] In some embodiments, performing noise estimation on a noisy image using a noise estimation model to obtain an estimated noise value may include: obtaining attribute information of a training image; inputting the attribute information and the noisy image into the noise estimation model to perform noise estimation to obtain an estimated noise value.

[0185] Among them, the noise estimation process in this embodiment can refer to the corresponding noise estimation process in the image desensitization method provided in the previous embodiment, and will not be repeated here.

[0186] In some embodiments, the attribute information of the training image includes the image type of the training image and / or the semantic information of the training image. Obtaining the attribute information of the training image includes: obtaining the image type; and / or inputting the training image into a feature extraction model for feature extraction to obtain image features corresponding to the training image, and inputting the image features into a feature encoding model for feature encoding to obtain semantic information.

[0187] Among them, the noise estimation process in this embodiment can refer to the corresponding noise estimation process in the image desensitization method provided in the previous embodiment, and will not be repeated here.

[0188] Figure 15 This is a schematic diagram of the structure of the image desensitization device provided by the embodiment of the present disclosure. Figure 15 As shown, the image desensitization device 1500 includes:

[0189] An acquisition unit 1501 is used to acquire an image to be processed, where sensitive information is superimposed on the image to be processed;

[0190] The encoding unit 1502 is configured to encode the image to be processed by using an encoder included in the image reconstruction model to obtain an encoded image corresponding to the image to be processed;

[0191] The decoding unit 1503 is used to decode the encoded image through the decoder included in the image reconstruction model to restore the non-sensitive information contained in the image to be processed and obtain a restored image corresponding to the image to be processed. The image reconstruction model is trained based on the training image without superimposed information.

[0192] In some embodiments, the image desensitization device 1500 also includes: a noise adding unit 1504, used to add noise to the encoded image to obtain a noisy image; a noise estimation unit 1505, used to estimate the noise of the noisy image through a noise estimation model to obtain an estimated noise value; a denoising unit 1506, used to denoise the noisy image according to the estimated noise value to obtain a denoised image; the decoding unit 1503 includes: a first decoding module (not shown in the figure), used to input the denoised image into the decoder for decoding processing to restore the non-sensitive information contained in the image to be processed to obtain a restored image.

[0193] In some embodiments, the noise addition unit 1504 includes: a step number determination module (not shown in the figure), used to determine the number of noise addition steps corresponding to the encoded image; a noise addition module (not shown in the figure), used to add multiple steps of noise to the encoded image according to the number of noise addition steps to obtain a noisy image.

[0194] In some embodiments, the step determination module includes: a step determination submodule (not shown in the figure), which is used to determine the number of noise-added steps based on the step threshold corresponding to the noise estimation model; wherein the number of noise-added steps is less than or equal to the step threshold.

[0195] In some embodiments, the noise adding module includes: a loading submodule (not shown in the figure) adding multiple steps of random Gaussian noise to the encoded image according to the number of noise adding steps and the Gaussian distribution function to obtain a noisy image.

[0196] In some embodiments, the noise estimation unit 1505 includes: an attribute acquisition module (not shown in the figure), which is used to obtain attribute information of the image to be processed; and a noise estimation module (not shown in the figure), which is used to input the attribute information and the noisy image into a noise estimation model for noise estimation to obtain an estimated noise value.

[0197] In some embodiments, the attribute information includes the image type of the image to be processed and / or the semantic information of the image to be processed, and the attribute acquisition module includes: a type acquisition submodule, used to obtain the image type; and / or a semantic information acquisition submodule, used to input the image to be processed into a feature extraction model for feature extraction, obtain image features corresponding to the image to be processed, input the image features into a feature encoding model for feature encoding, and obtain semantic information.

[0198] In some embodiments, the image desensitization device further includes: a first level determination unit (not shown in the figure), which is used to determine that the desensitization level of the image to be processed is the first level.

[0199] In some embodiments, the image desensitization device also includes: a second level determination unit (not shown in the figure), used to determine that the desensitization level of the image to be processed is the second level; a second decoding module, used to input the encoded image into the decoder for decoding processing to restore the non-sensitive information contained in the image to be processed and obtain a restored image.

[0200] In some embodiments, the encoder is deployed at the transmitting end and the decoder is deployed at the receiving end.

[0201] Figure 15 The image desensitization device provided can execute the above-mentioned corresponding method embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0202] Figure 16 This is a schematic diagram of the structure of the model training device provided in the embodiment of the present disclosure. Figure 16 As shown, the model training device 1600 includes:

[0203] An acquiring unit 1601 is configured to acquire a training data set, where the training data set includes training images without superimposed information.

[0204] The first training unit 1602 is used to train the image reconstruction model according to the training data set to obtain a trained image reconstruction model. The image reconstruction model includes an encoder and a decoder. The encoder is used for encoding processing of the image, and the decoder is used for recovering non-sensitive information in the image.

[0205] In some embodiments, the model training apparatus further includes: a second training unit 1603, configured to train the noise estimation model based on the training image and the trained image reconstruction model to obtain a trained noise estimation model.

[0206] In some embodiments, during the i-th training process of the noise estimation model, the second training unit 1603 includes: an encoding module 16031, which is used to encode the training image through an encoder in the trained image reconstruction model to obtain an encoded image corresponding to the training image; a denoising module 16032, which is used to add an actual noise value to the encoded image to obtain a noisy image; a noise estimation module 16033, which is used to perform noise estimation on the noisy image through the noise estimation model to obtain an estimated noise value; and a parameter adjustment module 16034, which is used to adjust the model parameters of the noise estimation model according to the actual noise value and the estimated noise value to obtain the noise estimation model after the i-th training.

[0207] In some embodiments, the noise addition module includes 16032: a step determination submodule (not shown in the figure), used to determine the number of noise addition steps corresponding to the encoded image; a noise addition submodule (not shown in the figure), used to add multiple steps of noise to the encoded image according to the number of noise addition steps to obtain a noisy image.

[0208] In some embodiments, the step number determination submodule is specifically used to randomly determine the number of noise adding steps according to a step number threshold corresponding to the noise estimation model.

[0209] In some embodiments, the noise estimation module 16033 includes: an attribute acquisition submodule (not shown in the figure), which is used to obtain attribute information of the training image; and a noise estimation submodule (not shown in the figure), which is used to input the attribute information and the noisy image into the noise estimation model for noise estimation to obtain an estimated noise value.

[0210] In some embodiments, the attribute information includes the image type of the training image and / or the semantic information of the training image, and the attribute acquisition submodule is specifically used to: obtain the image type; and / or, input the training image into a feature extraction model for feature extraction to obtain image features corresponding to the training image, input the image features into a feature encoding model for feature encoding to obtain semantic information.

[0211] Figure 16 The model training device provided can execute the above-mentioned corresponding method embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0212] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the solution provided by any of the above embodiments.

[0213] According to an embodiment of the present disclosure, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the solution provided by any of the above embodiments.

[0214] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0215] Figure 171 is a schematic block diagram of an example electronic device 1700 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0216] like Figure 17 As shown, the electronic device 1700 includes a computing unit 1701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1702 or a computer program loaded from the storage unit 808 into a random access memory (RAM) 1703. Various programs and data required for the operation of the device 1700 can also be stored in the RAM 1703. The computing unit 1701, the ROM 1702, and the RAM 1703 are connected to each other via a bus 1704. An input / output (I / O) interface 1705 is also connected to the bus 1704.

[0217] Various components in device 1700 are connected to I / O interface 1705, including an input unit 1706, such as a keyboard, mouse, etc.; an output unit 1707, such as various types of displays, speakers, etc.; a storage unit 1708, such as a magnetic disk, optical disk, etc.; and a communication unit 1709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1709 allows device 1700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0218] The computing unit 1701 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 1701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1701 performs the various methods and processes described above, such as the image desensitization method. For example, in some embodiments, the image desensitization method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1700 via the ROM 1702 and / or the communication unit 1709. When the computer program is loaded into the RAM 803 and executed by the computing unit 1701, one or more steps of the image desensitization method described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the image desensitization method in any other appropriate manner (eg, by means of firmware).

[0219] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0220] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0221] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0222] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0223] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0224] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0225] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of this disclosure can be achieved, and this document is not limited here.

[0226] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. An image desensitization method, comprising: Acquiring an image to be processed, where sensitive information is superimposed on the image to be processed; Encoding the image to be processed by an encoder included in the image reconstruction model to obtain an encoded image corresponding to the image to be processed; The encoded image is decoded by a decoder included in the image reconstruction model to restore the non-sensitive information contained in the image to be processed, thereby obtaining a restored image corresponding to the image to be processed. The image reconstruction model is trained based on a training image without superimposed information.

2. The image desensitization method according to claim 1, wherein: The decoding process of the encoded image by the decoder included in the image reconstruction model to restore the non-sensitive information included in the image to be processed and obtain the restored image corresponding to the image to be processed includes: Noising the encoded image to obtain a noisy image; performing noise estimation on the noisy image using a noise estimation model to obtain an estimated noise value; Denoising the noisy image according to the estimated noise value to obtain a denoised image; The denoised image is input into the decoder for decoding to restore the non-sensitive information contained in the image to be processed, thereby obtaining the restored image.

3. The image desensitization method according to claim 2, wherein: The step of adding noise to the coded image to obtain a noisy image includes: Determining the number of noise addition steps corresponding to the encoded image; Adding multiple steps of noise to the coded image according to the number of noise adding steps to obtain the noisy image.

4. The image desensitization method according to claim 3, wherein: The determining the number of noise addition steps corresponding to the encoded image includes: Determining the number of noise addition steps according to a step number threshold corresponding to the noise estimation model; The number of noise addition steps is less than or equal to the step threshold.

5. The image desensitization method according to claim 3, wherein: The adding multi-step noise to the coded image according to the number of noise adding steps to obtain the noisy image includes: According to the number of noise addition steps and the Gaussian distribution function, multiple steps of random Gaussian noise are added to the encoded image to obtain the noisy image.

6. The image desensitization method according to claim 2, wherein: The performing noise estimation on the noisy image by using a noise estimation model to obtain an estimated noise value includes: Acquiring attribute information of the image to be processed; The attribute information and the noisy image are input into the noise estimation model to perform noise estimation to obtain the estimated noise value.

7. The image desensitization method according to claim 6, wherein: The attribute information includes the image type of the image to be processed and / or semantic information of the image to be processed, and obtaining the attribute information of the image to be processed includes: Obtaining the image type; And / or, the image to be processed is input into a feature extraction model for feature extraction to obtain image features corresponding to the image to be processed, and the image features are input into a feature encoding model for feature encoding to obtain the semantic information.

8. The image desensitization method according to any one of claims 2 to 7, before adding noise to the encoded image to obtain the noisy image, further comprising: Determine that the desensitization level of the image to be processed is the first level.

9. The image desensitization method according to any one of claims 1 to 7, wherein: The decoding process of the encoded image by the decoder included in the image reconstruction model to restore the non-sensitive information included in the image to be processed and obtain the restored image corresponding to the image to be processed includes: Determining that the desensitization level of the image to be processed is the second level; The encoded image is input into the decoder for decoding to restore the non-sensitive information contained in the image to be processed to obtain the restored image.

10. The image desensitization method according to any one of claims 1 to 7, wherein: The encoder is deployed at the transmitting end, and the decoder is deployed at the receiving end.

11. A model training method comprising: Acquire a training data set, wherein the training data set includes training images without superimposed information; The image reconstruction model is trained according to the training data set to obtain a trained image reconstruction model, wherein the image reconstruction model includes an encoder and a decoder, wherein the encoder is used for encoding processing of the image, and the decoder is used for recovering non-sensitive information in the image.

12. The model training method according to claim 11, further comprising: The noise estimation model is trained according to the training image and the trained image reconstruction model to obtain the trained noise estimation model.

13. The model training method according to claim 12, wherein: The i-th training process of the noise estimation model includes: In the trained image reconstruction model, encoding the training image by the encoder to obtain an encoded image corresponding to the training image; Adding an actual noise value to the encoded image to obtain a noisy image; performing noise estimation on the noisy image using a noise estimation model to obtain an estimated noise value; According to the actual noise value and the estimated noise value, the model parameters of the noise estimation model are adjusted to obtain the noise estimation model after the i-th training.

14. The model training method according to claim 13, wherein: Adding an actual noise value to the coded image to obtain a noisy image includes: Determining the number of noise addition steps corresponding to the encoded image; Adding multiple steps of noise to the coded image according to the number of noise adding steps to obtain the noisy image.

15. The model training method according to claim 14, wherein: The determining the number of noise addition steps corresponding to the encoded image includes: The number of noise adding steps is randomly determined according to a step number threshold corresponding to the noise estimation model.

16. The model training method according to claim 13, wherein: The performing noise estimation on the noisy image by using a noise estimation model to obtain an estimated noise value includes: Acquiring attribute information of the training image; The attribute information and the noisy image are input into the noise estimation model to perform noise estimation to obtain the estimated noise value.

17. The model training method according to claim 16, wherein: The attribute information includes the image type of the training image and / or semantic information of the training image, and obtaining the attribute information of the training image includes: Obtaining the image type; And / or, the training image is input into a feature extraction model for feature extraction to obtain image features corresponding to the training image, and the image features are input into a feature encoding model for feature encoding to obtain the semantic information.

18. An image desensitization device, comprising: an acquisition unit, configured to acquire an image to be processed, wherein the image to be processed is superimposed with sensitive information; an encoding unit, configured to encode the image to be processed by an encoder included in an image reconstruction model to obtain an encoded image corresponding to the image to be processed; A decoding unit is used to decode the encoded image through a decoder included in the image reconstruction model to restore the non-sensitive information contained in the image to be processed and obtain a restored image corresponding to the image to be processed, wherein the image reconstruction model is trained based on a training image without superimposed information.

19. A model training device comprising: an acquiring unit, configured to acquire a training data set, wherein the training data set includes training images without superimposed information; The first training unit is used to train the image reconstruction model according to the training data set to obtain a trained image reconstruction model, wherein the image reconstruction model includes an encoder and a decoder, the encoder is used for encoding processing of the image, and the decoder is used for recovering non-sensitive information in the image.

20. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the image desensitization method described in any one of claims 1 to 10, or to enable the at least one processor to execute the model training method described in any one of claims 11 to 17.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the image desensitization method according to any one of claims 1 to 10, or the computer instructions are used to cause the computer to execute the model training method according to any one of claims 11 to 17.

22. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the image desensitization method according to any one of claims 1 to 10, or the computer program implements the steps of the model training method according to any one of claims 11 to 17.

Images