A method, apparatus, and device for processing an image

Abstract

An embodiment of this specification discloses an image processing method, apparatus, and device. The method includes: obtaining a target image to be processed, where the target image includes privacy information of a target user; selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions; for each divided region, respectively performing the following processing to obtain three preselected steganographic images corresponding to each region: using a first steganographic encoder to write the RGB information in the target image into the region in a steganographic manner; using a second steganographic encoder to write the NIR information in the target image into the region in a steganographic manner; using a third steganographic encoder to write the RGB information and NIR information in the target image into the region in a steganographic manner; determining a steganographic image for steganographically writing the target image into the three-dimensional image based on the preselected steganographic images corresponding to the multiple different regions, and performing service processing on the target service based on the steganographic image.

Description

Technical Field

[0001] This document relates to the field of computer technology, and particularly to an image processing method, apparatus, and device. Background Art

[0002] In recent years, biometric technology has developed rapidly. Currently, biometrics has entered people's production and life, including face-swiping payment, face-swiping access control, and face-swiping attendance, etc. However, while the biometric system provides convenience for users, it also frequently collects users' biometric data. And after these biometric data are collected, operations such as transmission, processing, and storage are also carried out, and there is a great risk of privacy leakage in each link. Once the user's privacy information is leaked, their property and information security will be greatly threatened.

[0003] The privacy protection ability has become an important ability of the biometric system. Generally, privacy protection processing can be carried out through image encryption. Specifically, simple linear operations are used to encrypt or perform row-column confusion on the image containing the user's privacy information. However, the atomic operations of the above methods are simple, the process is single, and it is very easy to be cracked by means such as brute force. In addition, privacy protection processing can also be carried out through deep learning. Specifically, a deep learning model (such as a neural network model, etc.) is trained and used to perform privacy protection processing on the image containing the user's privacy information to obtain the image after privacy protection. However, a model for restoring the original image can be trained through model training to restore the image processed by privacy protection, resulting in the leakage of privacy information. Based on this, there is a need to provide an image privacy processing solution with higher security and stronger privacy protection ability. Summary of the Invention

[0004] The purpose of the embodiments of this specification is to provide an image privacy processing solution with higher security and stronger privacy protection ability.

[0005] In order to achieve the above technical solution, the embodiments of this specification are implemented as follows:

[0006] A method for processing an image provided in an embodiment of this specification, the method includes: obtaining a target image to be processed, where the target image includes privacy information of a target user. Selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions. For each of the divided regions, performing the following processing respectively to obtain three preselected stego-images corresponding to each region: Using a first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the region in a steganographic manner. Using a second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the region in a steganographic manner. Using a third stego-encoder obtained by pre-training through a model to write the RGB information and NIR information in the target image into the region in a steganographic manner. Based on the preselected stego-images corresponding to the multiple different regions, determining a stego-image for steganographically writing the target image into the three-dimensional image, and performing service processing on a target service based on the stego-image.

[0007] An image processing apparatus provided in an embodiment of this specification, the apparatus includes: an image acquisition module for obtaining a target image to be processed, where the target image includes privacy information of a target user. A three-dimensional image acquisition module for selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions. A steganography module for, for each of the divided regions, performing the following processing respectively to obtain three preselected stego-images corresponding to each region: Using a first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the region in a steganographic manner. Using a second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the region in a steganographic manner. Using a third stego-encoder obtained by pre-training through a model to write the RGB information and NIR information in the target image into the region in a steganographic manner. A processing module for determining a stego-image for steganographically writing the target image into the three-dimensional image based on the preselected stego-images corresponding to the multiple different regions, and performing service processing on a target service based on the stego-image.

[0008] An image processing device provided by an embodiment of this specification, the image processing device includes: a processor; and a memory arranged to store computer-executable instructions, the executable instructions, when executed, cause the processor to: obtain a target image to be processed, where the target image includes privacy information of a target user. Select a corresponding three-dimensional image for the target image, and divide the three-dimensional image into multiple different regions. For each of the divided regions, perform the following processing respectively to obtain three preselected stego-images corresponding to each region: Use a first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the region in a steganographic manner. Use a second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the region in a steganographic manner. Use a third stego-encoder obtained by pre-training through a model to write the RGB information and NIR information in the target image into the region in a steganographic manner. Based on the preselected stego-images corresponding to the multiple different regions, determine a stego-image for steganographically embedding the target image into the three-dimensional image, and perform business processing on a target business based on the stego-image.

[0009] An embodiment of this specification also provides a storage medium, the storage medium is used to store computer-executable instructions, and the executable instructions, when executed by a processor, implement the following process: obtain a target image to be processed, where the target image includes privacy information of a target user. Select a corresponding three-dimensional image for the target image, and divide the three-dimensional image into multiple different regions. For each of the divided regions, perform the following processing respectively to obtain three preselected stego-images corresponding to each region: Use a first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the region in a steganographic manner. Use a second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the region in a steganographic manner. Use a third stego-encoder obtained by pre-training through a model to write the RGB information and NIR information in the target image into the region in a steganographic manner. Based on the preselected stego-images corresponding to the multiple different regions, determine a stego-image for steganographically embedding the target image into the three-dimensional image, and perform business processing on a target business based on the stego-image. Description of the Drawings

[0010] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 This is an embodiment of a method for processing an image in this specification;

[0012] Figure 2 This is another embodiment of a method for processing an image in this specification;

[0013] Figure 3 This is yet another embodiment of a method for processing an image in this specification;

[0014] Figure 4 This is a schematic structural diagram of an image processing system in this specification;

[0015] Figure 5 This is yet another embodiment of a method for processing an image in this specification;

[0016] Figure 6 This is an embodiment of an image processing device in this specification;

[0017] Figure 7 This is an embodiment of an image processing device in this specification. Detailed implementation manners

[0018] The embodiments of this specification provide a method, device, and device for processing an image.

[0019] In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this specification.

[0020] Embodiment 1

[0021] As Figure 1 shown, the embodiments of this specification provide a method for processing an image. The execution subject of this method can be a terminal device or a server. Among them, the terminal device can be a certain terminal device such as a mobile phone or a tablet computer, or a computer device such as a notebook computer or a desktop computer, or it can also be an IoT device (specifically such as a smart watch, a vehicle-mounted device, etc.). The server can be an independent server, or a server cluster composed of multiple servers. The server can be a background server such as a financial service or an online shopping service, or a background server of a certain application program. The method can specifically include the following steps:

[0022] In step S102, a target image to be processed is obtained, and the target image includes the privacy information of the target user.

[0023] Among them, the target image can be any image containing the privacy information of the target user. The target image can be, for example, a facial image, a fingerprint image, or an iris image, etc., which can be specifically set according to the actual situation. The target user can be any user, such as the owner of the above-mentioned execution subject, or a user who needs to perform a target service. The target service can be any service, such as a biometric service, a payment service, etc., which can be specifically set according to the actual situation. The privacy information of the user can include various types, such as the user's name, the number of the certificate proving the user's identity, the residential address, the mobile phone number, the user's biometric information (specifically, such as the user's fingerprint information, facial information, iris information, etc.), etc., which can be specifically set according to the actual situation, and the embodiments of this specification do not limit this.

[0024] In practice, in recent years, biometric technology has developed rapidly. At present, biometrics has entered people's production and life, including face payment, face access control, and face attendance, etc. However, while the biometric system provides convenience for users, it also frequently collects users' biometric data. And after these biometric data are collected, operations such as transmission, processing, and storage are also carried out, and there is a great risk of privacy leakage in each link. Once the privacy information of the user is leaked, their property and information security will be greatly threatened.

[0025] The privacy protection ability has become an important ability of the biometric system. Generally, privacy protection processing can be carried out through image encryption. Specifically, simple linear operations are used to encrypt the image containing the user's privacy information or perform operations such as row-column confusion, etc. However, the atomic operations of the above methods are simple, the process is single, and it is very easy to be cracked by methods such as brute force cracking. In addition, privacy protection processing can also be carried out through deep learning. Specifically, a deep learning model (such as a neural network model, etc.) is trained and used to perform privacy protection processing on the image containing the user's privacy information to obtain the image after privacy protection. However, a model for restoring the original image can be trained through model training to restore the image after privacy protection processing, resulting in the leakage of privacy information. Based on this, an image privacy processing solution with higher security and stronger privacy protection ability is needed. The embodiments of this specification provide an implementable technical solution, which can specifically include the following content:

[0026] When a user (i.e., the target user) needs to execute a specified service (such as a payment service, a login service, etc.), the execution mechanism of the above-specified service can be triggered. During the execution of the above-specified service, the identity of the target user is often verified. Therefore, it is necessary to obtain relevant information of the target user. For example, it is possible to obtain an image including information such as the number of a document that can prove the user's identity, the residential address, the mobile phone number, and the user's biometric information. The obtained target image can be analyzed to determine whether it contains the privacy information of the target user. If it contains the privacy information of the target user, the above image can be obtained and the obtained image can be used as the target image.

[0027] For example, before executing the above-specified service, it is necessary to identify the identity of the target user. At this time, the corresponding device can start the camera component therein and can collect the user biometric image of the target user through the camera component (specifically, for example, the facial image, fingerprint image, iris image, etc. of the user can be collected), and the user biometric image can be used as the target image.

[0028] In step S104, a corresponding three-dimensional image is selected for the target image, and the three-dimensional image is divided into multiple different regions.

[0029] Among them, the three-dimensional image selected for the target image can be any three-dimensional image, and it can be a three-dimensional image associated with the target image. For example, the three-dimensional image can be the three-dimensional facial image of a certain user, and the target image can be the two-dimensional image of the user, or it can be a three-dimensional image unrelated to the target image, etc. It can be specifically set according to the actual situation, and the embodiments of this specification do not limit this.

[0030] In implementation, considering that there is no risk of privacy leakage in three-dimensional images, therefore, three-dimensional images can be used as the carrier of steganographic objects. After obtaining the target image containing the privacy information of the target user, the target image can be analyzed. Based on the obtained analysis result, the corresponding three-dimensional image can be selected for the target image according to the preset image selection rule. Then, the three-dimensional image can be divided into multiple different regions by a preset image segmentation method. In practical applications, the entire region of the three-dimensional image can be divided to obtain multiple different regions, or the local region of the three-dimensional image can be divided to obtain multiple different regions, which can be specifically set according to the actual situation. In addition, the method of dividing the three-dimensional image into multiple different regions can include various types. For example, the three-dimensional image can be randomly divided to obtain multiple different divided regions. For another example, the three-dimensional image can be evenly divided to obtain N (N is a positive integer greater than or equal to 2) equal parts of the three-dimensional image. For yet another example, a model for image segmentation of the three-dimensional image (for the convenience of subsequent narration, it can be called the first model) can be pre-trained. Then, the first model is used to divide the region of the three-dimensional image. Specifically, the first model can be constructed by various different algorithms. For example, the first model can be constructed by a neural network algorithm, or the first model can be constructed by a random forest algorithm, etc., which can be specifically set according to the actual situation, and the embodiments of this specification do not limit this. The initial architecture of the first model can be constructed by a preset algorithm. Then, different images containing the privacy information of the user can be obtained to form a two-dimensional sample image, and three-dimensional sample images can be selected (where the two-dimensional sample image and the three-dimensional sample image can be related or unrelated), and the above two-dimensional sample image and three-dimensional sample image can be used as the training sample images of the model. Then, a corresponding loss function can be set, and the obtained two-dimensional sample image, three-dimensional sample image, and the loss function can be used. At the same time, some regions can be randomly selected in the three-dimensional sample image, and then the first model can be trained to obtain the trained first model. The trained first model can select the region for steganographic processing for the two-dimensional sample image (that is, the region in the three-dimensional sample image), and the region of the three-dimensional sample image can be divided based on the selected region. After that, the above target image and three-dimensional image can be input into the above trained first model. The first model divides the three-dimensional image into regions where the target image can be steganographically processed, thereby obtaining multiple different regions, or the first model can determine the local region in the three-dimensional image where the target image can be steganographically processed, and then the local region can be further divided to obtain multiple different regions, etc., which can be specifically set according to the actual situation, and the embodiments of this specification do not limit this.

[0031] In step S106, for each divided region, the following processing is respectively performed to obtain three preselected stego-images corresponding to each region: use the first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the above region in a steganographic manner; use the second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the above region in a steganographic manner; use the third stego-encoder obtained by pre-training through a model to write the RGB information and the NIR information in the target image into the above region in a steganographic manner.

[0032] Among them, the first stego-encoder can be an encoder for writing the RGB information in a certain image into a certain region of a certain three-dimensional image in a steganographic manner, the second stego-encoder can be an encoder for writing the NIR information in a certain image into a certain region of a certain three-dimensional image in a steganographic manner, and the third stego-encoder can be an encoder for writing the RGB information and the NIR information in a certain image into a certain region of a certain three-dimensional image in a steganographic manner. The first stego-encoder, the second stego-encoder, and the third stego-encoder can be constructed through a variety of different algorithms respectively. For example, the above stego-encoders (including the first stego-encoder, the second stego-encoder, and the third stego-encoder) can be constructed through a neural network algorithm, or the above stego-encoders can be constructed through the HUGO (Highly Undetectable stego) algorithm, etc. It can be specifically set according to the actual situation, and the embodiments of this specification do not make any limitations in this regard.

[0033] In implementation, considering that general images may contain RGB information and NIR information, the user's privacy information may be carried in the RGB information, may be carried in the NIR information, or may be carried in both the RGB information and the NIR information. Based on this, the initial architectures of the first steganographic encoder, the second steganographic encoder, and the third steganographic encoder can be constructed respectively through a preset algorithm. Then, training sample images composed of the privacy information of different users can be obtained, the RGB information in the training sample images can be obtained, and three-dimensional sample images can be selected (where the training sample images and the three-dimensional sample images may or may not be related). One region can be randomly selected from the multiple different regions divided in the three-dimensional sample images, and the first steganographic encoder can be trained through a preset corresponding loss function. At the same time, another region can be selected, and the first steganographic encoder can be trained through a preset corresponding loss function, and so on. Finally, the trained first steganographic encoder can be obtained, and through the trained first steganographic encoder, the RGB information in the training sample images can be written into the selected region through information hiding. The NIR information in the training sample images can be obtained, and three-dimensional sample images can be selected (where the training sample images and the three-dimensional sample images may or may not be related). One region can be randomly selected from the multiple different regions divided in the three-dimensional sample images, and the second steganographic encoder can be trained through a preset corresponding loss function. At the same time, another region can be selected, and the second steganographic encoder can be trained through a preset corresponding loss function, and so on. Finally, the trained second steganographic encoder can be obtained, and through the trained second steganographic encoder, the NIR information in the training sample images can be written into the selected region through information hiding. The RGB information and the NIR information in the training sample images can be obtained, and three-dimensional sample images can be selected (where the training sample images and the three-dimensional sample images may or may not be related). One region can be randomly selected from the multiple different regions divided in the three-dimensional sample images, and the third steganographic encoder can be trained through a preset corresponding loss function. At the same time, another region can be selected, and the third steganographic encoder can be trained through a preset corresponding loss function, and so on. Finally, the trained third steganographic encoder can be obtained, and through the trained third steganographic encoder, the RGB information and the NIR information in the training sample images can be written into the selected region through information hiding.

[0034] After obtaining the target image and multiple different regions divided from the three-dimensional image, for any one of the divided regions, the RGB information in the target image can be written into the above region in a steganographic manner using the first steganographic encoder obtained by the above training. At the same time, for the above region, the NIR information in the target image can be written into the above region in a steganographic manner using the second steganographic encoder obtained by the above training. At the same time, for the above region, the RGB information and NIR information in the target image can be written into the above region in a steganographic manner using the third steganographic encoder obtained by the above training. Thus, for this region, three preselected steganographic images can be obtained. Then, by selecting another region, the above process can be repeated to obtain three preselected steganographic images corresponding to this region. Until the above process is completed for all regions, three preselected steganographic images corresponding to each region in multiple different regions can be obtained.

[0035] In step S108, based on the preselected steganographic images corresponding to multiple different regions, a steganographic image for steganographically embedding the target image into the three-dimensional image is determined, and business processing for the target service is performed based on this steganographic image.

[0036] In implementation, through the above processing, preselected steganographic images corresponding to multiple different regions can be obtained. One preselected steganographic image can be selected from the multiple preselected steganographic images. The region corresponding to the selected preselected steganographic image can be determined. The determined region and other regions outside the determined region can be spliced to form a spliced three-dimensional image. For example, if the three-dimensional image is divided into four regions, denoted as region 1, region 2, region 3, and region 4, and the region corresponding to the selected preselected steganographic image is region 1 (actually the region obtained by steganographically embedding the RGB information / NIR information / RGB information and NIR information of the target image into region 1), then region 2, region 3, and region 4 can be spliced with the region obtained by steganographically embedding the RGB information / NIR information / RGB information and NIR information of the target image into region 1 to form a spliced three-dimensional image. The spliced three-dimensional image can be used as the steganographic image for steganographically embedding the target image into the three-dimensional image.

[0037] The steganographic image can be used for subsequent business processing. Specifically, for example, if the target business is a payment business, when performing the payment business, the identity of the user needs to be identified. At this time, the steganographic image can be used to calculate the similarity with the benchmark user biometric image (which may not contain privacy information, that is, the user biometric image after privacy protection) pre-stored locally (or on the server). If the obtained similarity value is greater than the preset similarity threshold, the result of the biometric identification of the target user is passed, and at this time, the target business (i.e., the payment business) can continue to be processed. If the obtained similarity value is less than the preset similarity threshold, the result of the biometric identification of the target user is failed, and the processing of the target business is terminated. In practical applications, the above processing process is only an optional method and may also include various different processing methods, which can be specifically set according to the actual situation.

[0038] An embodiment of this specification provides a method for processing an image. By obtaining a target image to be processed, where the target image includes the privacy information of the target user; selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions; for each divided region, the following processing is respectively performed to obtain three preselected steganographic images corresponding to each region: using a first steganographic encoder to write the RGB information in the target image into the region in a steganographic manner; using a second steganographic encoder to write the NIR information in the target image into the region in a steganographic manner; using a third steganographic encoder to write the RGB information and NIR information in the target image into the region in a steganographic manner; based on the preselected steganographic images corresponding to multiple different regions, determining the steganographic image for steganographically writing the target image into the three-dimensional image, and performing business processing on the target business based on the steganographic image. In this way, not only can the privacy information of the RGB information in the image be desensitized, but also the privacy information of the NIR information in the image can be desensitized, that is, through steganographic processing, the RGB information and NIR information are steganographically written into the three-dimensional image without the risk of privacy leakage, and at the same time, the steganographic processing of the RGB information and NIR information is completed. This processing method does not damage the structure of the image and is friendly to image compression, thereby reducing the occupancy of bandwidth and storage by biometric identification. Moreover, this solution only needs to store the three-dimensional image, that is, only less than 1 / 3 of the original bandwidth and storage are used to complete the privacy protection processing, not only does not increase the image volume, but instead greatly reduces the image volume. In addition, in terms of steganography technology, different from the traditional steganographic processing of the entire information, this solution proposes a local-global steganographic method, that is, steganographic processing is performed through the local region of the three-dimensional image. On the one hand, the quality of steganography is improved, and on the other hand, the attacker cannot determine which region the steganographic processing is performed on, so that the privacy information is difficult to be leaked, and the security of steganography is improved.

[0039] Embodiment Two

[0040] As Figure 2 shown, an embodiment of this specification provides a method for processing an image. The execution subject of this method can be a terminal device or a server. Among them, the terminal device can be a certain terminal device such as a mobile phone, a tablet computer, etc., or a computer device such as a laptop computer or a desktop computer, or it can also be an IoT device (specifically such as a smart watch, a vehicle-mounted device, etc.). The server can be an independent server or a server cluster composed of multiple servers. The server can be a background server such as a financial service or an online shopping service, or a background server of a certain application program, etc. The method can specifically include the following steps:

[0041] In step S202, a training sample image and a corresponding three-dimensional sample image are obtained, and the training sample image includes the privacy information of the user.

[0042] Among them, there can be multiple training sample images. The privacy information of the same user can be included in multiple training sample images, or different users' privacy information can be included in different training sample images, etc., which can be specifically set according to the actual situation. There can be multiple three-dimensional sample images. The number of three-dimensional sample images can be determined according to the number of training sample images. The number of three-dimensional sample images can be less than the number of training sample images. The three-dimensional sample image can be a three-dimensional image of a certain user. The training sample image can also include the image of this user, or it can not include the image of this user, etc.

[0043] In implementation, with the consent of the user, images containing the user's privacy information can be obtained from multiple different users, and the obtained images including the user's privacy information can be used as training sample images. Or, images containing the user's privacy information can be obtained from a specified database, and the obtained images can be used as training sample images, etc., which can be specifically set according to the actual situation. In addition, the three-dimensional sample image can also be obtained according to the actual situation. The three-dimensional sample image can be related to the training sample image or unrelated to the training sample image. For example, with the consent of the user, the user can provide a three-dimensional sample image related to the training sample image, or a three-dimensional sample image can be randomly selected from a specified database, etc., which can be specifically set according to the actual situation. This specification embodiment does not make any limitations on this.

[0044] In step S204, the three-dimensional sample image is divided into multiple different sample image regions.

[0045] In implementation, the entire region or part of the three-dimensional sample image can be randomly divided according to the actual situation to obtain multiple different sample image regions. In addition, multiple different other methods can also be included to implement the regional division of the three-dimensional sample image. For example, an average division can be performed. For example, the three-dimensional sample image can be divided into four different sample image regions, which can be the region of the upper left part of the three-dimensional image, the region of the lower left part of the three-dimensional image, the region of the upper right part of the three-dimensional image, and the region of the lower right part of the three-dimensional image, etc.

[0046] In step S206, for each divided sample image region, the following processing is respectively performed: The first steganographic encoder and the corresponding first steganographic decoder are jointly trained through the RGB information in the training sample image and the sample image region to obtain the trained first steganographic encoder. The first steganographic decoder is used to perform a restoration process on the steganographically processed training sample image; the second steganographic encoder and the corresponding second steganographic decoder are jointly trained through the NIR information in the training sample image and the sample image region to obtain the trained second steganographic encoder. The second steganographic decoder is used to perform a restoration process on the steganographically processed training sample image; the third steganographic encoder and the corresponding third steganographic decoder are jointly trained through the RGB information, NIR information, and the sample image region in the training sample image to obtain the trained third steganographic encoder. The third steganographic decoder is used to perform a restoration process on the steganographically processed training sample image.

[0047] Among them, any one of the first steganographic encoder, the first steganographic decoder, the second steganographic encoder, the second steganographic decoder, the third steganographic encoder, and the third steganographic decoder can be constructed based on multiple different methods. For example, it can be constructed based on U-Net, which is constructed by a fully connected network. U-Net presents a structure similar to the letter "U". It consists of a contracting path on the left side and an expansive path on the right side. The contracting path can be constructed by a convolutional neural network. The structure of 2 convolutional layers and 1 max pooling layer can be repeatedly adopted. After each pooling operation, the dimension of the data will increase. In the expansive path, first perform 1 deconvolution operation to halve the dimension of the data. Then, splice and crop it corresponding to the contracting path to obtain the corresponding feature data. Based on the above feature data, recombine new feature data, and then use 2 convolutional layers for feature extraction, and repeat the above structure. In the final output layer, use 2 convolutional layers to map the high-dimensional feature data into low-dimensional output data. U-Net can be specifically divided into two parts: upsampling and downsampling. The downsampling part mainly uses continuous convolutional pooling layers to extract feature information in the data and gradually maps the feature information to a high dimension. There is rich feature information of the entire data at the highest dimension of the entire network. U-Net does not need to directly pool the data and directly upsample it to the output data with the same size as the original data. Instead, through deconvolution processing, the high-dimensional features are mapped to a low dimension again. In order to enhance the accuracy of segmentation during the mapping process, the data with the same dimension in the contracting network at the same dimension will be fused. Since the dimension will become twice the original dimension during the fusion process, it is necessary to perform convolution processing again to ensure that the dimension after processing is the same as the dimension before the fusion operation, so that after another deconvolution processing, it can be secondarily fused with the data at the same dimension until the dimension is the same as the original data and then the output data is output. The structures of the encoder and decoder in this embodiment can be composed of U-Net with a certain number of network layers. Specifically, for example, it can be composed of U-Net with 8 or 10 network layers, etc., which can be specifically set according to the actual situation. For another example, it can be constructed through a multi-layer perceptron MLP. In MLP, in addition to the input layer and the output layer, it can have multiple hidden layers in the middle. The simplest MLP only contains one hidden layer, that is, a three-layer structure. The layers of MLP are fully connected. The bottom layer of MLP is the input layer, the middle is the hidden layer, and the last is the output layer. Specifically, the above-mentioned encoders and decoders can be constructed by a three-layer MLP, which can be specifically set according to the actual situation.

[0048] In implementation, steganography processing can be separately performed on the RGB information, NIR information, and RGB information + NIR information in the training sample image. Specifically, one sample image region can be selected from the above-mentioned multiple sample image regions. Then, the RGB information in the training sample image and this sample image region can be input into the first steganography encoder to obtain output data (i.e., the steganography-processed training sample image, which is the data obtained after writing the RGB information in the training sample image into this sample image region through information steganography). The first steganography decoder can be used to perform restoration processing on the above output data. Then, the corresponding loss value can be calculated through a preset loss function. Based on the calculated loss value, it can be determined whether the above first steganography encoder and the corresponding first steganography decoder converge. If they converge, the trained first steganography encoder and the trained first steganography decoder are obtained. If they do not converge, the parameters in the first steganography encoder and the first steganography decoder are adjusted, and the first steganography encoder and the corresponding first steganography decoder are continuously trained based on the training sample image and the multiple sample image regions. The sample image region can be replaced, and the above processing can be repeatedly executed until the first steganography encoder and the corresponding first steganography decoder converge, obtaining the trained first steganography encoder and the trained first steganography decoder.

[0049] Similarly, one sample image region can also be selected from the above-mentioned multiple sample image regions based on the above processing process. Then, the NIR information in the training sample image and this sample image region can be input into the second steganography encoder to obtain output data (i.e., the steganography-processed training sample image, which is the data obtained after writing the NIR information in the training sample image into this sample image region through information steganography). The second steganography decoder can be used to perform restoration processing on the above output data. Then, the corresponding loss value can be calculated through a preset loss function. Based on the calculated loss value, it can be determined whether the above second steganography encoder and the corresponding second steganography decoder converge. If they converge, the trained second steganography encoder and the trained second steganography decoder are obtained. If they do not converge, the parameters in the second steganography encoder and the second steganography decoder are adjusted, and the second steganography encoder and the corresponding second steganography decoder are continuously trained based on the training sample image and the multiple sample image regions. The sample image region can be replaced, and the above processing can be repeatedly executed until the second steganography encoder and the corresponding second steganography decoder converge, obtaining the trained second steganography encoder and the trained second steganography decoder.

[0050] Similarly, a sample image region can be selected from the above-mentioned multiple sample image regions based on the above processing process. Then, the RGB information, NIR information, and the sample image region in the training sample image can be input into the third steganographic encoder to obtain output data (i.e., the steganographically processed training sample image, which is the data obtained after writing the RGB information and NIR information in the training sample image into the sample image region through information steganography). The third steganographic decoder can be used to restore the above output data. Then, the corresponding loss value can be calculated through a preset loss function. Based on the calculated loss value, it can be determined whether the above third steganographic encoder and the corresponding third steganographic decoder converge. If they converge, the trained third steganographic encoder and the trained third steganographic decoder are obtained. If they do not converge, the parameters in the third steganographic encoder and the third steganographic decoder are adjusted, and the third steganographic encoder and the corresponding third steganographic decoder are continuously trained based on the training sample image and the multiple sample image regions. The sample image region can be replaced, and the above processing can be repeatedly executed until the third steganographic encoder and the corresponding third steganographic decoder converge, and the trained third steganographic encoder and the trained third steganographic decoder are obtained.

[0051] It should be noted that the weights of the three encoders, namely the first steganographic encoder, the second steganographic encoder, and the third steganographic encoder, are independent of each other, and the features of the intermediate network layers of the above three encoders interact with each other.

[0052] There are various specific processing methods for jointly training the first steganographic encoder and the corresponding first steganographic decoder through the RGB information in the training sample image and the above sample image region in step S206. The following provides a specific processing method, which can refer to the processing in steps A2 to A6 below.

[0053] In step A2, the RGB information in the training sample image and the sample image region are input into the first steganographic encoder to obtain the steganographically processed training sample image.

[0054] In step A4, the steganographically processed training sample image is input into the first steganographic decoder to restore the steganographically processed training sample image through the first steganographic decoder, and the RGB information in the reconstructed training sample image is obtained.

[0055] In step A6, based on the RGB information in the training sample image, the steganographically processed training sample image, the RGB information in the reconstructed training sample image, and a preset loss function, determine whether the first steganographic encoder and the first steganographic decoder converge. If not, continue to train the first steganographic encoder and the first steganographic decoder based on the training sample image and the sample image region until the first steganographic encoder and the first steganographic decoder converge, and obtain the trained first steganographic encoder.

[0056] The specific processing method of jointly training the second steganographic encoder and the corresponding second steganographic decoder through the NIR information in the training sample image and the above sample image region in step S206 can be various. The following provides a specific processing method, which can refer to the processing in steps B2 to B6 below.

[0057] In step B2, input the NIR information in the training sample image and the sample image region into the second steganographic encoder to obtain the steganographically processed training sample image.

[0058] In step B4, input the steganographically processed training sample image into the second steganographic decoder to restore the steganographically processed training sample image through the second steganographic decoder, and obtain the NIR information in the reconstructed training sample image.

[0059] In step B6, based on the NIR information in the training sample image, the steganographically processed training sample image, the NIR information in the reconstructed training sample image, and a preset loss function, determine whether the second steganographic encoder and the second steganographic decoder converge. If not, continue to train the second steganographic encoder and the second steganographic decoder based on the training sample image and the sample image region until the second steganographic encoder and the second steganographic decoder converge, and obtain the trained second steganographic encoder.

[0060] The specific processing method of jointly training the third steganographic encoder and the corresponding third steganographic decoder through the RGB information and NIR information in the training sample image and the above sample image region in step S206 can be various. The following provides a specific processing method, which can refer to the processing in steps C2 to C6 below.

[0061] In step C2, input the RGB information and NIR information in the training sample image, and the sample image region into the third steganographic encoder to obtain the steganographically processed training sample image.

[0062] In step C4, the steganographically processed training sample image is input into the third steganographic decoder to restore the steganographically processed training sample image through the third steganographic decoder, and the RGB information and NIR information in the reconstructed training sample image are obtained.

[0063] In step C6, based on the RGB information and NIR information in the training sample image, the steganographically processed training sample image, the RGB information and NIR information in the reconstructed training sample image, and a preset loss function, it is determined whether the third steganographic encoder and the third steganographic decoder converge. If not, the third steganographic encoder and the third steganographic decoder are continuously trained based on the training sample image and the sample image region until the third steganographic encoder and the third steganographic decoder converge, and the trained third steganographic encoder is obtained.

[0064] Among them, the above-mentioned loss function can be determined in a variety of different ways. For example, corresponding loss functions can be set based on the first steganographic encoder, the first steganographic decoder, the second steganographic encoder, the second steganographic decoder, the third steganographic encoder, and the third steganographic decoder respectively. Or a loss function corresponding to the input data and the final output data can be set, etc. Or, a suitable loss function can be set for the above joint training according to the actual situation, which can be specifically set according to the actual situation, and the embodiments of this specification do not limit this. In this embodiment, the loss function can be the maximum value of the similarity between the three-dimensional image constructed from the steganographically processed training sample image and the three-dimensional sample image, and the maximum value of the similarity between the reconstructed RGB information / reconstructed NIR information / reconstructed RGB information and reconstructed NIR information and the RGB information / NIR information / RGB information and NIR information in the training sample image (the three-dimensional image constructed from the steganographically processed training sample image can effectively restore the RGB information / NIR information / RGB information and NIR information in the training sample image). The loss functions in the training processes of the above three parts can generally be expressed as:

[0065]

[0066] Among them, L psnr is used to constrain that the relative error between the three-dimensional images before and after steganography should be as small as possible, that is, the regions of the three-dimensional images before and after steganography are visually consistent. L reconstruction is used to constrain that the three-dimensional image after steganography can effectively restore the embedded RGB information, NIR information, RGB information + NIR information, that is, the maximum value of the similarity between the reconstructed information (reconstructed RGB information, reconstructed NIR information, reconstructed RGB information + reconstructed NIR information) and the original information (RGB information, NIR information, RGB information + NIR information). I RGBrepresents the RGB information in the training sample image, I NIR represents the NIR information in the training sample image, I RGB-NIR represents the RGB information and the NIR information in the training sample image represents the 3D image obtained after secretly writing the RGB information into the 3D image represents the 3D image obtained after secretly writing the NIR information into the 3D image represents the 3D image obtained after secretly writing the RGB information and the NIR information into the 3D image, I′ RGB represents the reconstructed RGB information, I′ NIR represents the reconstructed NIR information, I′ RGB-NIR represents the reconstructed RGB information and the reconstructed NIR information

[0067] In step S208, obtain the target image to be processed, where the target image includes the privacy information of the target user

[0068] Among them, the user's privacy information can be user biometric information, and the user biometric information can include multiple types, such as the user's fingerprint information, palmprint information, facial information, or iris information, etc. Specifically, it can be set according to the actual situation, and the embodiments of this specification do not limit this

[0069] In step S210, select a corresponding 3D image for the target image and divide the 3D image into multiple different regions

[0070] In implementation, for example, the 3D image can be divided into four different regions, and the four different regions are respectively the region in the upper left part of the 3D image, the region in the lower left part of the 3D image, the region in the upper right part of the 3D image, and the region in the lower right part of the 3D image

[0071] In step S212, for each divided region, perform the following processing respectively to obtain three preselected stego-images corresponding to each region: use the first stego-encoder obtained by pre-training through a model to secretly write the RGB information in the target image into this region in a steganographic manner; use the second stego-encoder obtained by pre-training through a model to secretly write the NIR information in the target image into this region in a steganographic manner; use the third stego-encoder obtained by pre-training through a model to secretly write the RGB information and the NIR information in the target image into this region in a steganographic manner

[0072] In implementation, if the 3D image is divided into four different regions, through the above processing, each region can obtain three preselected stego-images, so a total of 12 preselected stego-images can be obtained

[0073] In step S214, obtain the compression volumes of the preselected stego-images corresponding to multiple different regions, and the peak signal-to-noise ratio (PSNR) of the images of the multiple different regions.

[0074] In step S216, based on the compression volumes of the preselected stego-images corresponding to multiple different regions, the peak signal-to-noise ratio (PSNR) of the images of the multiple different regions, and the corresponding weights, respectively determine the stego-scores of the preselected stego-images corresponding to each region. The stego-score is used to characterize the degree of non-identifiability after the information of the target image is written into the corresponding region.

[0075] In implementation, for example, the stego-score of the preselected stego-image corresponding to each region can be calculated by the following formula

[0076] score = 0.25 * (50.0 - s) + 0.75 * size

[0077] where score represents the stego-score, s represents the PSNR of the image of a certain region, size represents the compression volume of the preselected stego-image corresponding to that region, and 0.25 and 0.75 respectively represent the weights. Through the above formula, the stego-scores of the preselected stego-images corresponding to each region can be calculated. The larger the obtained stego-score, the more difficult it is to identify the information of the target image written into the corresponding region.

[0078] In step S218, obtain the first region corresponding to the preselected stego-image with a stego-score greater than the preset threshold, and splice the first region with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

[0079] Among them, the preset threshold can be set according to the actual situation, and the embodiments of this specification do not limit this.

[0080] In practical applications, the region corresponding to the preselected stego-image with the largest stego-score can be selected as the first region, etc.

[0081] The process of splicing the first region with the regions other than the region corresponding to the first region among the multiple different regions in step S218 to obtain a spliced three-dimensional image can be various. The following provides an optional processing method, which can specifically include the processing of step D2 and step D4.

[0082] In step D2, perform color correction processing on the image of the first region to obtain the first corrected first region.

[0083] In implementation, generally, the images of the regions where steganography is performed in the three-dimensional image may be distorted, so that others can easily identify the regions where steganography is performed, resulting in the user's privacy information being at risk. Therefore, the above regions can be repaired. Specifically, color correction processing can be performed on the images of the first region. According to the actual situation, color correction rules can be preset. Specifically, for example, the colors in the image can be deepened, etc. The color correction processing of the images of the first region can be performed based on the color correction rules, etc., and can be specifically set according to the actual situation.

[0084] The processing of step D2 above can be various. The following provides an optional processing method, which can specifically include the processing of steps D22 to D26 below.

[0085] In step D22, based on the color information of the first region and the color information of the regions other than the region corresponding to the first region among the multiple different regions, Gaussian parameters are determined.

[0086] Among them, the Gaussian parameters can be represented by the filter size of the Gaussian smoothing filter, that is, the color adjustment processing in this embodiment is implemented by the Gaussian smoothing filter. In practical applications, the Gaussian parameters can also be characterized by other indicators or parameters, and the color adjustment processing can also be implemented by other methods, which can be specifically set according to the actual situation. The embodiments of this specification do not limit this.

[0087] In implementation, the following formula can be used to calculate the Gaussian parameters

[0088] m pp =Mean(I pp ),m ori =Mean(I ori )

[0089] k size =0.05*||m pp -m ori ||2

[0090] Among them, I pp represents the color information of the first region, I ori represents the color information of the regions other than the region corresponding to the first region among the multiple different regions, Mean represents the average value, m pp represents the average value of the color information of the first region (that is, the average value of the pixels in the first region), m ori represents the average value of the color information of the regions other than the region corresponding to the first region among the multiple different regions (that is, the average value of the pixels in the regions other than the region corresponding to the first region among the multiple different regions), k sizeRepresents the Gaussian parameter (i.e., the filter size of the Gaussian smoothing filter).

[0091] In step D24, based on the Gaussian parameter, the color information of the first region, and the color information of the regions other than the region corresponding to the first region among the multiple different regions, Gaussian adjustment is respectively performed on the first region and the regions other than the region corresponding to the first region among the multiple different regions, to obtain the adjusted color information of the first region and the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions.

[0092] In implementation, the following formula can be used to calculate the adjusted color information of the first region and the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions

[0093] I ppblur =Guassian(I pp ,(k szie ,k size ))

[0094] I oriblur =Guassian(I ori ,(k szie ,k size ))

[0095] Wherein, I ppblur represents the adjusted color information of the first region, I oriblur represents the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions, Guassian(I pp ,(k szie ,k size )) represents inputting the color information of the first region into the filter of the Gaussian smoothing filter with the above Gaussian parameter for Gaussian adjustment.

[0096] In step D26, based on the color information of the first region, the adjusted color information of the first region, and the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions, color correction processing is performed on the image of the first region to obtain the first region after the first correction.

[0097] In implementation, the first region after the first correction can be calculated through the following formula

[0098] I ppnew =I pp *(I ppblur / I oriblur )

[0099] Wherein, I ppnew represents the first region after the first correction.

[0100] In step D4, the first corrected first region is stitched with the regions other than the region corresponding to the first region among the multiple different regions to obtain a stitched three-dimensional image.

[0101] For the specific processing procedure of the above step D4, reference can be made to the foregoing relevant content, which will not be elaborated herein.

[0102] The processing of the above step D4 can be various. The following provides an optional processing method, which specifically may include the processing of the following step D42 and step D44.

[0103] In step D42, correction processing is performed on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the second corrected first region;

[0104] In implementation, since simple color correction cannot solve some texture distortion problems, based on this, the correction algorithm of the image can be preset according to the actual situation. This correction algorithm can perform correction processing on the image texture, shape, and spatial relationship corresponding to the image. There can be multiple such correction algorithms. Specifically, for example, a corresponding neural network model can be constructed through a neural network algorithm, and a large number of sample data are used to train this neural network model to obtain the trained neural network model. Then, the image of the first region can be input into this neural network model, and through this neural network model, correction processing can be performed on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the second corrected first region. In practical applications, there can also be multiple different algorithms, which can be specifically set according to the actual situation, and the embodiments of this specification do not limit this.

[0105] The processing of the above step D42 can be various. The following further provides an optional processing method, which specifically may include the following: The image of the first region is input into a pre-trained image enhancement model, and through the image enhancement model, correction processing is performed on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the second corrected first region. The image enhancement model is a model obtained after model training based on image samples.

[0106] In implementation, the image enhancement model can be constructed through an image enhancement algorithm, etc., and can be obtained after model training using image samples. Through the image enhancement model, gray-scale transformation, image smoothing processing, image sharpening processing, filtering processing, color enhancement processing, etc. can be performed on the image of the first region, so that the image texture, shape, and spatial relationship corresponding to the image of the first region are corrected, and finally the second corrected first region is obtained.

[0107] In step D44, the second corrected first region is spliced with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

[0108] In step S220, based on the spliced three-dimensional image, a stego image in which the target image is hidden in the three-dimensional image is determined.

[0109] In implementation, when the resolution of the three-dimensional image is lower than a preset threshold, the spliced three-dimensional image can be determined as the stego image in which the target image is hidden in the three-dimensional image.

[0110] The processing of the above step S220 can be various. Hereinafter, an optional processing method is provided, which may specifically include the processing of step E2 and step E4.

[0111] In step E2, the spliced three-dimensional image is subjected to image compression processing to obtain a compressed three-dimensional image.

[0112] In implementation, an image compression algorithm such as JPEG can be used to perform image compression processing on the spliced three-dimensional image to obtain a compressed three-dimensional image. Since the texture structure of the spliced three-dimensional image (i.e., the three-dimensional image after steganography processing) is still the texture structure of the three-dimensional image, it is very friendly to image compression. As long as less than 1 / 3 of the original data volume is transmitted and stored, the occupancy of bandwidth and storage can be greatly reduced. In this way, not only can the privacy protection of the target image be completed, but also the image volume is not increased, but instead the image volume is greatly reduced.

[0113] In step E4, based on the compressed three-dimensional image, a stego image in which the target image is hidden in the three-dimensional image is determined.

[0114] In implementation, the compressed three-dimensional image can be determined as the stego image in which the target image is hidden in the three-dimensional image.

[0115] In addition, the resolution of the spliced three-dimensional image can also be adjusted as follows: the resolution of the spliced three-dimensional image is adjusted to be the same as the resolution of the three-dimensional image, and based on the adjusted spliced three-dimensional image, a stego image in which the target image is hidden in the three-dimensional image is determined.

[0116] In step S222, business processing is performed on the target service based on the stego image.

[0117] In implementation, if the target image is a user biometric image for biometric identification, biometric identification processing can be performed on the target user based on the stego image.

[0118] In step S224, the target image is deleted.

[0119] An embodiment of this specification provides an image processing method. By obtaining a target image to be processed, where the target image includes privacy information of a target user; selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions; for each divided region, the following processing is respectively performed to obtain three preselected steganographic images corresponding to each region: using a first steganographic encoder to write the RGB information in the target image into the region in a steganographic manner; using a second steganographic encoder to write the NIR information in the target image into the region in a steganographic manner; using a third steganographic encoder to write the RGB information and NIR information in the target image into the region in a steganographic manner; based on the preselected steganographic images corresponding to multiple different regions, determining a steganographic image in which the target image is steganographically written into the three-dimensional image, and performing service processing on the target service based on the steganographic image. In this way, not only can the privacy information of the RGB information in the image be desensitized, but also the privacy information of the NIR information in the image can be desensitized, that is, through steganographic processing, the RGB information and NIR information are steganographically written into a three-dimensional image without the risk of privacy leakage, and at the same time, the steganographic processing of the RGB information and NIR information is completed. This processing method does not damage the structure of the image and is friendly to image compression, thereby reducing the occupation of bandwidth and storage by biometrics. Moreover, this solution only needs to store the three-dimensional image, that is, as long as less than 1 / 3 of the original bandwidth and storage are used, the privacy protection processing can be completed. Not only does it not increase the image volume, but it also greatly reduces the image volume. In addition, in terms of steganography technology, different from the traditional steganographic processing of the entire information, this solution proposes a local-global steganographic method, that is, steganographic processing is performed through the local region of the three-dimensional image. On the one hand, the quality of steganography is improved, and on the other hand, the attacker cannot determine which region the steganographic processing is performed on, so that the privacy information is difficult to be leaked, and the security of steganography is improved.

[0120] Embodiment III

[0121] As Figure 3 shown, an embodiment of this specification provides an image processing method. This method can be jointly executed by a terminal device and a server. Among them, the terminal device can be a certain terminal device such as a mobile phone, a tablet computer, etc., and can also be a computer device such as a laptop computer or a desktop computer, or can also be an IoT device (specifically such as a smart watch, a vehicle-mounted device, etc.). The server can be an independent server, or can also be a server cluster composed of multiple servers, etc. The server can be a background server such as a financial service or an online shopping service, or can also be a background server of a certain application program, etc. Its system architecture can be as Figure 4 shown, and this method can specifically include the following steps:

[0122] In step S302, the server obtains the training sample image and the corresponding three-dimensional sample image, and the training sample image includes the user's privacy information.

[0123] In step S304, the server divides the three-dimensional sample image into multiple different sample image regions.

[0124] In step S306, for each divided sample image region, the server respectively performs the following processing: jointly trains the first steganographic encoder and the corresponding first steganographic decoder through the RGB information in the training sample image and the sample image region to obtain the trained first steganographic encoder, and the first steganographic decoder is used to perform restoration processing on the steganographically processed training sample image; jointly trains the second steganographic encoder and the corresponding second steganographic decoder through the NIR information in the training sample image and the sample image region to obtain the trained second steganographic encoder, and the second steganographic decoder is used to perform restoration processing on the steganographically processed training sample image; jointly trains the third steganographic encoder and the corresponding third steganographic decoder through the RGB information, NIR information in the training sample image, and the sample image region to obtain the trained third steganographic encoder, and the third steganographic decoder is used to perform restoration processing on the steganographically processed training sample image.

[0125] The specific processing method of jointly training the first steganographic encoder and the corresponding first steganographic decoder through the RGB information in the training sample image and the above sample image region in step S306 to obtain the trained first steganographic encoder can be various. The following provides a specific processing method, which can be referred to the following content: The server inputs the RGB information in the training sample image and the sample image region into the first steganographic encoder to obtain the steganographically processed training sample image; the server inputs the steganographically processed training sample image into the first steganographic decoder to perform restoration processing on the steganographically processed training sample image through the first steganographic decoder to obtain the RGB information in the reconstructed training sample image; the server determines whether the first steganographic encoder and the first steganographic decoder converge based on the RGB information in the training sample image, the steganographically processed training sample image, the RGB information in the reconstructed training sample image, and a preset loss function. If not, continue to train the first steganographic encoder and the first steganographic decoder based on the training sample image and the sample image region until the first steganographic encoder and the first steganographic decoder converge to obtain the trained first steganographic encoder.

[0126] In the above step S306, the specific method of jointly training the second steganographic encoder and the corresponding second steganographic decoder by using the NIR information in the training sample image and the above sample image region to obtain the trained second steganographic encoder can be various. The following provides a specific method, which can be referred to as follows: The server inputs the NIR information and the sample image region in the training sample image into the second steganographic encoder to obtain the steganographically processed training sample image; The server inputs the steganographically processed training sample image into the second steganographic decoder to perform a restoration process on the steganographically processed training sample image through the second steganographic decoder to obtain the NIR information in the reconstructed training sample image; The server determines whether the second steganographic encoder and the second steganographic decoder converge based on the NIR information in the training sample image, the steganographically processed training sample image, the NIR information in the reconstructed training sample image, and a preset loss function. If not, the second steganographic encoder and the second steganographic decoder are continuously trained based on the training sample image and the sample image region until the second steganographic encoder and the second steganographic decoder converge to obtain the trained second steganographic encoder.

[0127] In the above step S306, the specific method of jointly training the third steganographic encoder and the corresponding third steganographic decoder by using the RGB information, NIR information in the training sample image and the above sample image region to obtain the trained third steganographic encoder can be various. The following provides a specific method, which can be referred to as follows: The server inputs the RGB information, NIR information in the training sample image, and the sample image region into the third steganographic encoder to obtain the steganographically processed training sample image; The server inputs the steganographically processed training sample image into the third steganographic decoder to perform a restoration process on the steganographically processed training sample image through the third steganographic decoder to obtain the RGB information and NIR information in the reconstructed training sample image; The server determines whether the third steganographic encoder and the third steganographic decoder converge based on the RGB information and NIR information in the training sample image, the steganographically processed training sample image, the RGB information and NIR information in the reconstructed training sample image, and a preset loss function. If not, the third steganographic encoder and the third steganographic decoder are continuously trained based on the training sample image and the sample image region until the third steganographic encoder and the third steganographic decoder converge to obtain the trained third steganographic encoder.

[0128] In step S308, the server sends the trained first steganographic encoder, the trained second steganographic encoder, and the trained third steganographic encoder to the terminal device.

[0129] In step S310, the terminal device obtains the target image to be processed, and the target image includes the privacy information of the target user.

[0130] Among them, the user's privacy information can be the user's biological information, and the user's biological information can include various types, such as the user's fingerprint information, palmprint information, facial information, or iris information, etc. Specifically, it can be set according to the actual situation, and the embodiments of this specification do not limit this.

[0131] In step S312, the terminal device selects a corresponding three-dimensional image for the target image and divides the three-dimensional image into multiple different regions.

[0132] In implementation, for example, the three-dimensional image can be divided into four different regions, which are the region of the upper left part of the three-dimensional image, the region of the lower left part of the three-dimensional image, the region of the upper right part of the three-dimensional image, and the region of the lower right part of the three-dimensional image.

[0133] In step S314, for each divided region, the terminal device respectively performs the following processing to obtain three preselected stego-images corresponding to each region: using a first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the region in a steganographic manner; using a second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the region in a steganographic manner; using a third stego-encoder obtained by pre-training through a model to write the RGB information and NIR information in the target image into the region in a steganographic manner.

[0134] In step S316, the terminal device obtains the compression volume of the preselected stego-images corresponding to multiple different regions, and the peak signal-to-noise ratio PSNR of the images of multiple different regions.

[0135] In step S318, the terminal device respectively determines the stego-score of the preselected stego-image corresponding to each region based on the compression volume of the preselected stego-images corresponding to multiple different regions, the peak signal-to-noise ratio PSNR of the images of multiple different regions, and the corresponding weights. The stego-score is used to characterize the degree of non-identifiability after the information of the target image is written into the corresponding region.

[0136] In step S320, the terminal device obtains the first region corresponding to the preselected stego-image with a stego-score greater than the preset threshold, and splices the first region with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

[0137] In the above step S320, the process of splicing the first region with the regions other than the region corresponding to the first region among the multiple different regions to obtain the spliced three-dimensional image can be various. The following provides an optional processing method, which may specifically include the following: The terminal device performs color correction processing on the image of the first region to obtain the first region after the first correction; The terminal device splices the first region after the first correction with the regions other than the region corresponding to the first region among the multiple different regions to obtain the spliced three-dimensional image.

[0138] The above-mentioned color correction processing of the image of the first region by the terminal device can be various. The following provides an optional processing method, which may specifically include the following: The terminal device determines the Gaussian parameters based on the color information of the first region and the color information of the regions other than the region corresponding to the first region among the multiple different regions; The terminal device performs Gaussian adjustment on the first region and the regions other than the region corresponding to the first region among the multiple different regions respectively based on the Gaussian parameters, the color information of the first region, and the color information of the regions other than the region corresponding to the first region among the multiple different regions to obtain the adjusted color information of the first region and the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions; The terminal device performs color correction processing on the image of the first region based on the color information of the first region, the adjusted color information of the first region, and the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions to obtain the first region after the first correction.

[0139] The above-mentioned region splicing process can be various. The following provides an optional processing method, which may specifically include the following: The terminal device corrects the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the first region after the second correction; The terminal device splices the first region after the second correction with the regions other than the region corresponding to the first region among the multiple different regions to obtain the spliced three-dimensional image.

[0140] The above-mentioned correction processing of the image texture, shape, and spatial relationship corresponding to the image of the first region can be various. The following provides another optional processing method, which may specifically include the following: The terminal device inputs the image of the first region into a pre-trained image enhancement model, and the image enhancement model corrects the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the first region after the second correction. The image enhancement model is a model obtained after model training based on image samples.

[0141] In step S322, the terminal device determines the stego image obtained by implicitly writing the target image into the three-dimensional image based on the spliced three-dimensional image.

[0142] The processing in step S322 can be diversified. Here is another optional processing method, which may specifically include the following: The terminal device performs image compression processing on the spliced three-dimensional image to obtain a compressed three-dimensional image; the terminal device determines a steganographic image in which the target image is hidden in the three-dimensional image based on the compressed three-dimensional image.

[0143] In step S324, the terminal device sends the steganographic image to the server.

[0144] In step S326, the server performs biometric recognition processing on the target user based on the pre-stored reference user biometric image and the steganographic image.

[0145] In step S328, the terminal device receives the recognition result of the biometric recognition processing on the target user and deletes the target image.

[0146] For the specific processing procedures of the above steps S302 to S328, please refer to the relevant content above and will not be elaborated here.

[0147] An embodiment of this specification provides an image processing method. By obtaining a target image to be processed, where the target image includes privacy information of a target user; selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions; for each divided region, the following processing is respectively performed to obtain three preselected steganographic images corresponding to each region: using a first steganographic encoder to write the RGB information in the target image into the region in a steganographic manner; using a second steganographic encoder to write the NIR information in the target image into the region in a steganographic manner; using a third steganographic encoder to write the RGB information and NIR information in the target image into the region in a steganographic manner; based on the preselected steganographic images corresponding to multiple different regions, determining a steganographic image for steganographically embedding the target image into the three-dimensional image, and performing service processing on the target service based on the steganographic image. In this way, not only can the privacy information of the RGB information in the image be desensitized, but also the privacy information of the NIR information in the image can be desensitized, that is, through steganography processing, the RGB information and NIR information are steganographically embedded into a three-dimensional image without the risk of privacy leakage, and at the same time, the steganography processing of the RGB information and NIR information is completed. This processing method does not damage the structure of the image and is friendly to image compression, thereby reducing the occupancy of bandwidth and storage by biometrics. Moreover, this solution only needs to store the three-dimensional image, that is, privacy protection processing can be completed by using less than 1 / 3 of the original bandwidth and storage. It not only does not increase the image volume, but instead greatly reduces the image volume. In addition, in terms of steganography technology, different from the traditional steganography processing of the entire information, this solution proposes a local-global steganography method, that is, steganography processing is performed through local regions of the three-dimensional image. On the one hand, the quality of steganography is improved, and on the other hand, it is difficult for an attacker to determine which region the steganography is performed on, so that the privacy information is difficult to be leaked, and the security of steganography is improved.

[0148] Embodiment 4

[0149] This embodiment will elaborate in detail on an image processing method provided by an embodiment of the present invention in combination with a specific application scenario. The corresponding application scenario is an application scenario of biometrics (such as face recognition, etc.).

[0150] As Figure 5 shown, the execution subject of this method can be a terminal device and a server. Among them, the terminal device can be a certain terminal device such as a mobile phone, a tablet computer, etc., or a computer device such as a laptop computer or a desktop computer, or, it can also be an IoT device (specifically such as a smart watch, a vehicle-mounted device, etc.). The server can be an independent server, or a server cluster composed of multiple servers, etc. The server can be a background server such as a financial service or an online shopping service, or a background server of a certain application program, etc. This method can specifically include the following steps:

[0151] In step S502, the server obtains a training sample image and a corresponding three-dimensional sample image, where the training sample image includes the user's privacy information.

[0152] In step S504, the server divides the three-dimensional sample image into multiple different sample image regions.

[0153] In step S506, for each divided sample image region, the server respectively performs the following processing: jointly trains the first steganographic encoder and the corresponding first steganographic decoder through the RGB information in the training sample image and the sample image region to obtain the trained first steganographic encoder, and the first steganographic decoder is used to perform a restoration process on the steganographically processed training sample image; jointly trains the second steganographic encoder and the corresponding second steganographic decoder through the NIR information in the training sample image and the sample image region to obtain the trained second steganographic encoder, and the second steganographic decoder is used to perform a restoration process on the steganographically processed training sample image; jointly trains the third steganographic encoder and the corresponding third steganographic decoder through the RGB information, NIR information in the training sample image, and the sample image region to obtain the trained third steganographic encoder, and the third steganographic decoder is used to perform a restoration process on the steganographically processed training sample image.

[0154] The specific processing method of jointly training the first steganographic encoder and the corresponding first steganographic decoder through the RGB information in the training sample image and the above sample image region in step S506 to obtain the trained first steganographic encoder can be various. The following provides a specific processing method, which can be referred to as follows: The server inputs the RGB information in the training sample image and the sample image region into the first steganographic encoder to obtain the steganographically processed training sample image; the server inputs the steganographically processed training sample image into the first steganographic decoder to perform a restoration process on the steganographically processed training sample image through the first steganographic decoder to obtain the RGB information in the reconstructed training sample image; the server determines whether the first steganographic encoder and the first steganographic decoder converge based on the RGB information in the training sample image, the steganographically processed training sample image, the RGB information in the reconstructed training sample image, and a preset loss function. If not, continue to train the first steganographic encoder and the first steganographic decoder based on the training sample image and the sample image region until the first steganographic encoder and the first steganographic decoder converge to obtain the trained first steganographic encoder.

[0155] In the above step S506, the specific method of jointly training the second steganographic encoder and the corresponding second steganographic decoder by using the NIR information in the training sample image and the above sample image region to obtain the trained second steganographic encoder can be various. The following provides a specific method, which can be referred to as follows: The server inputs the NIR information and the sample image region in the training sample image into the second steganographic encoder to obtain the steganographically processed training sample image; The server inputs the steganographically processed training sample image into the second steganographic decoder to perform a restoration process on the steganographically processed training sample image through the second steganographic decoder, so as to obtain the NIR information in the reconstructed training sample image; The server determines whether the second steganographic encoder and the second steganographic decoder converge based on the NIR information in the training sample image, the steganographically processed training sample image, the NIR information in the reconstructed training sample image, and a preset loss function. If not, continue to train the second steganographic encoder and the second steganographic decoder based on the training sample image and the sample image region until the second steganographic encoder and the second steganographic decoder converge to obtain the trained second steganographic encoder.

[0156] In the above step S506, the specific method of jointly training the third steganographic encoder and the corresponding third steganographic decoder by using the RGB information, NIR information in the training sample image and the above sample image region to obtain the trained third steganographic encoder can be various. The following provides a specific method, which can be referred to as follows: The server inputs the RGB information, NIR information in the training sample image, and the sample image region into the third steganographic encoder to obtain the steganographically processed training sample image; The server inputs the steganographically processed training sample image into the third steganographic decoder to perform a restoration process on the steganographically processed training sample image through the third steganographic decoder, so as to obtain the RGB information and NIR information in the reconstructed training sample image; The server determines whether the third steganographic encoder and the third steganographic decoder converge based on the RGB information and NIR information in the training sample image, the steganographically processed training sample image, the RGB information and NIR information in the reconstructed training sample image, and a preset loss function. If not, continue to train the third steganographic encoder and the third steganographic decoder based on the training sample image and the sample image region until the third steganographic encoder and the third steganographic decoder converge to obtain the trained third steganographic encoder.

[0157] In step S508, the server sends the trained first steganographic encoder, the trained second steganographic encoder, and the trained third steganographic encoder to the terminal device.

[0158] In step S510, the terminal device obtains a biometric request of the target user, where the biometric request includes a target image to be processed, and the target image includes the user biometric information of the target user.

[0159] Among them, the user biometric information can include various types, such as the fingerprint information, palmprint information, facial information, or iris information of the user, etc. Specifically, it can be set according to the actual situation, and the embodiments of this specification do not limit this.

[0160] In step S512, the terminal device selects a corresponding three-dimensional image for the target image and divides the three-dimensional image into multiple different regions.

[0161] In step S514, for each divided region, the terminal device respectively performs the following processing to obtain three preselected stego-images corresponding to each region: using a first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the region in a steganographic manner; using a second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the region in a steganographic manner; using a third stego-encoder obtained by pre-training through a model to write the RGB information and NIR information in the target image into the region in a steganographic manner.

[0162] In step S516, the terminal device obtains the compression volume of the preselected stego-images corresponding to multiple different regions, and the peak signal-to-noise ratio PSNR of the images of multiple different regions.

[0163] In step S518, the terminal device respectively determines the stego-score of the preselected stego-image corresponding to each region based on the compression volume of the preselected stego-images corresponding to multiple different regions, the peak signal-to-noise ratio PSNR of the images of multiple different regions, and the corresponding weights. The stego-score is used to characterize the degree of non-identifiability after the information of the target image is written into the corresponding region.

[0164] In step S520, the terminal device obtains the first region corresponding to the preselected stego-image with a stego-score greater than a preset threshold.

[0165] In step S522, the terminal device determines the Gaussian parameter based on the color information of the first region and the color information of the regions other than the region corresponding to the first region among multiple different regions.

[0166] In step S524, the terminal device performs Gaussian adjustment on the first region and the regions other than the region corresponding to the first region among the multiple different regions respectively based on Gaussian parameters, color information of the first region, and color information of the regions other than the region corresponding to the first region among the multiple different regions, so as to obtain adjusted color information of the first region and adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions.

[0167] In step S526, the terminal device performs color correction processing on the image of the first region based on the color information of the first region, the adjusted color information of the first region, and the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions, so as to obtain the first region after the first correction.

[0168] In step S528, the terminal device inputs the image of the first region after the first correction into a pre-trained image enhancement model, and performs correction processing on the image texture, shape, and spatial relationship corresponding to the image of the first region through the image enhancement model, so as to obtain the first region after the second correction. The image enhancement model is a model obtained after model training based on image samples.

[0169] In step S530, the terminal device splices the first region after the second correction with the regions other than the region corresponding to the first region among the multiple different regions, so as to obtain a spliced three-dimensional image.

[0170] In step S532, the terminal device performs image compression processing on the spliced three-dimensional image, so as to obtain a compressed three-dimensional image.

[0171] In step S534, the terminal device determines a stego image in which a target image is hidden in the three-dimensional image based on the compressed three-dimensional image.

[0172] In step S536, the terminal device sends the stego image to the server.

[0173] In step S538, the server performs restoration processing on the stego image through a stego decoder corresponding to the stego encoder corresponding to the stego image, so as to obtain a reconstructed target image, and performs biometric processing on the target user based on a pre-stored reference user biometric image and the reconstructed target image.

[0174] In step S540, the terminal device receives the recognition result of the biometric processing on the target user and deletes the target image.

[0175] For the specific processing procedures of the above steps S502 to S540, reference may be made to the above relevant content, which will not be elaborated here.

[0176] An embodiment of this specification provides an image processing method. By obtaining a target image to be processed, where the target image includes privacy information of a target user; selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions; for each divided region, performing the following processing respectively to obtain three preselected stego images corresponding to each region: writing the RGB information in the target image into the region in a steganographic manner using a first steganographic encoder; writing the NIR information in the target image into the region in a steganographic manner using a second steganographic encoder; writing the RGB information and NIR information in the target image into the region in a steganographic manner using a third steganographic encoder; determining a stego image for steganographically embedding the target image into the three-dimensional image based on the preselected stego images corresponding to multiple different regions, and performing service processing on the target service based on the stego image. In this way, not only can the privacy information of the RGB information in the image be desensitized, but also the privacy information of the NIR information in the image can be desensitized, that is, through steganographic processing, the RGB information and NIR information are steganographically embedded into a three-dimensional image without the risk of privacy leakage, and at the same time, the steganographic processing of the RGB information and NIR information is completed. This processing method does not damage the structure of the image and is friendly to image compression, thereby reducing the occupancy of bandwidth and storage by biometric recognition. Moreover, this solution only needs to store the three-dimensional image, that is, privacy protection processing can be completed as long as less than 1 / 3 of the original bandwidth and storage are used. It not only does not increase the image volume, but also greatly reduces the image volume. In addition, in terms of steganography technology, different from the traditional steganographic processing of the entire information, this solution proposes a local-global steganographic method, that is, steganographic processing is performed through local regions of the three-dimensional image. On the one hand, the quality of steganography is improved, and on the other hand, it is difficult for an attacker to determine which region the steganographic processing is performed on, so that the privacy information is difficult to be leaked, and the security of steganography is improved.

[0177] Embodiment 5

[0178] Based on the same idea, an embodiment of this specification also provides an image processing device, as Figure 6 shown.

[0179] The image processing device includes: an image acquisition module 601, a three-dimensional image acquisition module 602, a steganography module 603, and a processing module 604, where:

[0180] The image acquisition module 601 acquires a target image to be processed, and the target image includes privacy information of a target user;

[0181] The three-dimensional image acquisition module 602 selects a corresponding three-dimensional image for the target image and divides the three-dimensional image into multiple different regions;

[0182] The steganography module 603 performs the following processing for each of the divided regions respectively to obtain three preselected steganographic images corresponding to each region:

[0183] Use the first steganographic encoder obtained by pre-training with a model to write the RGB information in the target image into the region in a steganographic manner;

[0184] Use the second steganographic encoder obtained by pre-training with a model to write the NIR information in the target image into the region in a steganographic manner;

[0185] Use the third steganographic encoder obtained by pre-training with a model to write the RGB information and NIR information in the target image into the region in a steganographic manner;

[0186] The processing module 604 determines a steganographic image for steganographically writing the target image into the three-dimensional image based on the preselected steganographic images corresponding to the multiple different regions, and performs service processing on the target service based on the steganographic image.

[0187] In the embodiments of this specification, the target image is a user biometric image for biometric identification.

[0188] The processing module 604 performs biometric identification processing on the target user based on the steganographic image.

[0189] The device further includes:

[0190] A deletion module for deleting the target image.

[0191] In the embodiments of this specification, the processing module 604 includes:

[0192] An image sending unit for sending the steganographic image to a server, where the steganographic image is used to trigger the server to perform biometric identification processing on the target user based on a pre-stored reference user biometric image and the steganographic image;

[0193] A result receiving unit for receiving the biometric identification result of the biometric identification processing on the target user sent by the server.

[0194] In the embodiments of this specification, the processing module 604 includes:

[0195] An information acquisition unit for acquiring the compression volume of the preselected steganographic images corresponding to multiple different regions, and the peak signal-to-noise ratio PSNR of the images of multiple different regions.

[0196] A score determination unit determines the steganography scores corresponding to each region for the preselected stego-images corresponding to multiple different regions respectively based on the compression volumes of the preselected stego-images corresponding to the multiple different regions, the peak signal-to-noise ratio (PSNR) of the images of the multiple different regions, and the corresponding weights, where the steganography scores are used to characterize the degree of non-identifiability after the information of the target image is written into the corresponding regions;

[0197] A splicing unit acquires the first regions corresponding to the preselected stego-images with steganography scores greater than a preset threshold, and splices the first regions with the regions other than the regions corresponding to the first regions among the multiple different regions to obtain a spliced three-dimensional image;

[0198] A stego-image determination unit determines a stego-image in which the target image is steganographically embedded into the three-dimensional image based on the spliced three-dimensional image.

[0199] In an embodiment of the present specification, the stego-image determination unit adjusts the resolution of the spliced three-dimensional image to be the same as the resolution of the three-dimensional image, and determines a stego-image in which the target image is steganographically embedded into the three-dimensional image based on the adjusted spliced three-dimensional image.

[0200] In an embodiment of the present specification, the splicing unit performs color correction processing on the image of the first region to obtain a first corrected first region; and splices the first corrected first region with the regions other than the regions corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

[0201] In an embodiment of the present specification, the splicing unit determines Gaussian parameters based on the color information of the first region and the color information of the regions other than the regions corresponding to the first region among the multiple different regions; respectively performs Gaussian adjustment on the first region and the regions other than the regions corresponding to the first region among the multiple different regions based on the Gaussian parameters, the color information of the first region, and the color information of the regions other than the regions corresponding to the first region among the multiple different regions to obtain the adjusted color information of the first region and the adjusted color information of the regions other than the regions corresponding to the first region among the multiple different regions; and performs color correction processing on the image of the first region based on the color information of the first region, the adjusted color information of the first region, and the adjusted color information of the regions other than the regions corresponding to the first region among the multiple different regions to obtain a first corrected first region.

[0202] In the embodiments of this specification, the splicing unit corrects the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the first region after the second correction; and splices the first region after the second correction with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

[0203] In the embodiments of this specification, the splicing unit inputs the image of the first region into a pre-trained image enhancement model, and the image enhancement model corrects the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the first region after the second correction. The image enhancement model is a model obtained after model training based on image samples.

[0204] In the embodiments of this specification, the device further includes:

[0205] A sample acquisition module, which acquires training sample images and corresponding three-dimensional sample images, and the training sample images include the privacy information of the user;

[0206] A sample region division module, which divides the three-dimensional sample image into multiple different sample image regions;

[0207] A training module, which respectively performs the following processing for each of the divided sample image regions:

[0208] Through the RGB information in the training sample image and the sample image region, the first steganographic encoder and the corresponding first steganographic decoder are jointly trained to obtain a trained first steganographic encoder. The first steganographic decoder is used to perform restoration processing on the steganographically processed training sample image;

[0209] Through the NIR information in the training sample image and the sample image region, the second steganographic encoder and the corresponding second steganographic decoder are jointly trained to obtain a trained second steganographic encoder. The second steganographic decoder is used to perform restoration processing on the steganographically processed training sample image;

[0210] Through the RGB information, NIR information in the training sample image, and the sample image region, the third steganographic encoder and the corresponding third steganographic decoder are jointly trained to obtain a trained third steganographic encoder. The third steganographic decoder is used to perform restoration processing on the steganographically processed training sample image.

[0211] In the embodiments of this specification, the training module includes:

[0212] A first input unit that inputs the RGB information in the training sample image and the sample image region into the first steganographic encoder to obtain a steganographically processed training sample image;

[0213] A second input unit that inputs the steganographically processed training sample image into the first steganographic decoder to perform a restoration process on the steganographically processed training sample image through the first steganographic decoder to obtain the RGB information in the reconstructed training sample image;

[0214] A first training unit that determines whether the first steganographic encoder and the first steganographic decoder converge based on the RGB information in the training sample image, the steganographically processed training sample image, the RGB information in the reconstructed training sample image, and a preset loss function. If not, continue to train the first steganographic encoder and the first steganographic decoder based on the training sample image and the sample image region until the first steganographic encoder and the first steganographic decoder converge to obtain a trained first steganographic encoder.

[0215] In the embodiments of this specification, the loss function is composed of the maximum value of the similarity between the three-dimensional image constructed by the steganographically processed training sample image and the three-dimensional sample image, and the maximum value of the similarity between the reconstructed RGB information and the RGB information in the training sample image.

[0216] In the embodiments of this specification, the three-dimensional image acquisition module 602 divides the three-dimensional image into four different regions, which are respectively the region of the upper left part of the three-dimensional image, the region of the lower left part of the three-dimensional image, the region of the upper right part of the three-dimensional image, and the region of the lower right part of the three-dimensional image.

[0217] In the embodiments of this specification, the steganographic image determination unit performs image compression processing on the spliced three-dimensional image to obtain a compressed three-dimensional image; based on the compressed three-dimensional image, determines a steganographic image in which the target image is steganographically written into the three-dimensional image.

[0218] An embodiment of this specification provides an image processing apparatus. By obtaining a target image to be processed, where the target image includes privacy information of a target user, selecting a corresponding three-dimensional image for the target image, and dividing the three-dimensional image into multiple different regions, for each divided region, the following processing is respectively performed to obtain three preselected steganographic images corresponding to each region: writing the RGB information in the target image into the region in a steganographic manner using a first steganographic encoder; writing the NIR information in the target image into the region in a steganographic manner using a second steganographic encoder; writing the RGB information and NIR information in the target image into the region in a steganographic manner using a third steganographic encoder; determining a steganographic image for steganographically embedding the target image into the three-dimensional image based on the preselected steganographic images corresponding to the multiple different regions, and performing service processing on the target service based on the steganographic image. In this way, not only can the privacy information of the RGB information in the image be desensitized, but also the privacy information of the NIR information in the image can be desensitized, that is, through steganographic processing, the RGB information and NIR information are steganographically embedded into a three-dimensional image without the risk of privacy leakage, and at the same time, the steganographic processing of the RGB information and NIR information is completed. This processing method does not damage the structure of the image and is friendly to image compression, thereby reducing the occupancy of bandwidth and storage by biometrics. Moreover, this solution only needs to store the three-dimensional image, that is, only less than 1 / 3 of the original bandwidth and storage are used to complete the privacy protection processing. Not only does it not increase the image volume, but it also greatly reduces the image volume. In addition, in terms of steganography technology, different from the traditional steganographic processing of the entire information, this solution proposes a local-global steganographic method, that is, steganographic processing is performed through the local region of the three-dimensional image. On the one hand, the quality of steganography is improved, and on the other hand, the attacker cannot determine in which region the steganographic processing is performed, so that the privacy information is difficult to be leaked, improving the security of steganography.

[0219] Embodiment VI

[0220] The above is the image processing apparatus provided by the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide an image processing device, as Figure 7 shown.

[0221] The image processing device may be a terminal device or a server provided in the above embodiments, etc.

[0222] Image processing devices can vary significantly depending on their configuration or performance. They can include one or more processors 701 and a memory 702. The memory 702 can store one or more stored application programs or data. Among them, the memory 702 can be short-term storage or persistent storage. The application programs stored in the memory 702 can include one or more modules (not shown in the figure), and each module can include a series of computer-executable instructions in the image processing device. Further, the processor 701 can be set to communicate with the memory 702 and execute a series of computer-executable instructions in the memory 702 on the image processing device. The image processing device can also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input / output interfaces 705, and one or more keyboards 706.

[0223] Specifically, in this embodiment, the image processing device includes a memory and one or more programs. One or more of the programs are stored in the memory, and one or more of the programs can include one or more modules. Each module can include a series of computer-executable instructions in the image processing device and is configured to be executed by one or more processors. The one or more programs include the following computer-executable instructions for:

[0224] Obtain a target image to be processed, where the target image includes the privacy information of the target user;

[0225] Select a corresponding three-dimensional image for the target image and divide the three-dimensional image into multiple different regions;

[0226] For each of the divided regions, perform the following processing respectively to obtain three preselected stego images corresponding to each region:

[0227] Use a first steganographic encoder obtained by pre-training a model to write the RGB information in the target image into the region in a steganographic manner;

[0228] Use a second steganographic encoder obtained by pre-training a model to write the NIR information in the target image into the region in a steganographic manner;

[0229] Use a third steganographic encoder obtained by pre-training a model to write the RGB information and NIR information in the target image into the region in a steganographic manner;

[0230] Based on the preselected stego images corresponding to the multiple different regions, determine a stego image for steganographically embedding the target image into the three-dimensional image, and perform business processing on the target service based on the stego image.

[0231] In the embodiments of this specification, the target image is a user biometric image for biometric identification.

[0232] Performing business processing on the target service based on the steganographic image includes:

[0233] Performing biometric identification processing on the target user based on the steganographic image;

[0234] The method further includes:

[0235] Deleting the target image.

[0236] In the embodiments of this specification, performing biometric identification processing on the target user based on the steganographic image includes:

[0237] Sending the steganographic image to a server, where the steganographic image is used to trigger the server to perform biometric identification processing on the target user based on a pre-stored reference user biometric image and the steganographic image;

[0238] Receiving the biometric identification result of performing biometric identification processing on the target user sent by the server.

[0239] In the embodiments of this specification, determining the steganographic image for steganographically writing the target image into the three-dimensional image based on the preselected steganographic images corresponding to multiple different regions includes:

[0240] Obtaining the compression volume of the preselected steganographic images corresponding to multiple different regions, and the peak signal-to-noise ratio (PSNR) of the images of multiple different regions;

[0241] Based on the compression volume of the preselected steganographic images corresponding to multiple different regions, the peak signal-to-noise ratio (PSNR) of the images of multiple different regions, and the corresponding weights, respectively determining the steganographic scores of the preselected steganographic images corresponding to each region, where the steganographic scores are used to characterize the degree of non-identifiability after the information of the target image is written into the corresponding region;

[0242] Obtaining the first region corresponding to the preselected steganographic image with a steganographic score greater than a preset threshold, and splicing the first region with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image;

[0243] Based on the spliced three-dimensional image, determining the steganographic image for steganographically writing the target image into the three-dimensional image.

[0244] In the embodiments of this specification, determining the steganographic image for steganographically writing the target image into the three-dimensional image based on the spliced three-dimensional image includes:

[0245] Adjust the resolution of the stitched three-dimensional image to be the same as that of the three-dimensional image, and based on the adjusted stitched three-dimensional image, determine a stego image obtained by steganographically writing the target image into the three-dimensional image.

[0246] In the embodiments of this specification, the stitching the first region with regions other than the region corresponding to the first region among the multiple different regions to obtain a stitched three-dimensional image includes:

[0247] Perform color correction processing on the image of the first region to obtain a first corrected first region;

[0248] Stitch the first corrected first region with regions other than the region corresponding to the first region among the multiple different regions to obtain a stitched three-dimensional image.

[0249] In the embodiments of this specification, the performing color correction processing on the image of the first region to obtain a first corrected first region includes:

[0250] Determine Gaussian parameters based on the color information of the first region and the color information of regions other than the region corresponding to the first region among the multiple different regions;

[0251] Based on the Gaussian parameters, the color information of the first region, and the color information of regions other than the region corresponding to the first region among the multiple different regions, perform Gaussian adjustment on the first region and regions other than the region corresponding to the first region among the multiple different regions respectively to obtain the adjusted color information of the first region and the adjusted color information of regions other than the region corresponding to the first region among the multiple different regions;

[0252] Based on the color information of the first region, the adjusted color information of the first region, and the adjusted color information of regions other than the region corresponding to the first region among the multiple different regions, perform color correction processing on the image of the first region to obtain a first corrected first region.

[0253] In the embodiments of this specification, the stitching the first region with regions other than the region corresponding to the first region among the multiple different regions to obtain a stitched three-dimensional image includes:

[0254] Perform correction processing on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain a second corrected first region;

[0255] Stitch the second corrected first region with regions other than the region corresponding to the first region among the multiple different regions to obtain a stitched three-dimensional image.

[0256] In the embodiments of this specification, the correction processing of the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the second corrected first region includes:

[0257] Input the image of the first region into a pre-trained image enhancement model, and the image enhancement model corrects the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the second corrected first region. The image enhancement model is a model obtained after model training based on image samples.

[0258] In the embodiments of this specification, it further includes:

[0259] Obtain training sample images and corresponding three-dimensional sample images, where the training sample images include user privacy information;

[0260] Divide the three-dimensional sample images into multiple different sample image regions;

[0261] For each of the divided sample image regions, perform the following processing respectively:

[0262] Jointly train the first steganographic encoder and the corresponding first steganographic decoder through the RGB information in the training sample image and the sample image region to obtain the trained first steganographic encoder. The first steganographic decoder is used to perform restoration processing on the steganographically processed training sample image;

[0263] Jointly train the second steganographic encoder and the corresponding second steganographic decoder through the NIR information in the training sample image and the sample image region to obtain the trained second steganographic encoder. The second steganographic decoder is used to perform restoration processing on the steganographically processed training sample image;

[0264] Jointly train the third steganographic encoder and the corresponding third steganographic decoder through the RGB information, NIR information, and the sample image region in the training sample image to obtain the trained third steganographic encoder. The third steganographic decoder is used to perform restoration processing on the steganographically processed training sample image.

[0265] In the embodiments of this specification, the joint training of the first steganographic encoder and the corresponding first steganographic decoder through the RGB information in the training sample image and the sample image region to obtain the trained first steganographic encoder includes:

[0266] Input the RGB information in the training sample image and the sample image region into the first steganographic encoder to obtain the steganographically processed training sample image;

[0267] Input the steganographically processed training sample image into the first steganographic decoder to perform restoration processing on the steganographically processed training sample image through the first steganographic decoder, so as to obtain the RGB information in the reconstructed training sample image;

[0268] Based on the RGB information in the training sample image, the steganographically processed training sample image, the RGB information in the reconstructed training sample image, and a preset loss function, determine whether the first steganographic encoder and the first steganographic decoder converge. If not, continue to train the first steganographic encoder and the first steganographic decoder based on the training sample image and the sample image region until the first steganographic encoder and the first steganographic decoder converge, and obtain the trained first steganographic encoder.

[0269] In the embodiments of this specification, the loss function consists of the maximum similarity between the three-dimensional image constructed by the steganographically processed training sample image and the three-dimensional sample image, and the maximum similarity between the reconstructed RGB information and the RGB information in the training sample image.

[0270] In the embodiments of this specification, the dividing the three-dimensional image into multiple different regions includes:

[0271] Divide the three-dimensional image into four different regions, which are respectively the region of the upper left part of the three-dimensional image, the region of the lower left part of the three-dimensional image, the region of the upper right part of the three-dimensional image, and the region of the lower right part of the three-dimensional image.

[0272] In the embodiments of this specification, the determining the steganographic image obtained by steganographically writing the target image into the three-dimensional image based on the spliced three-dimensional image includes:

[0273] Perform image compression processing on the spliced three-dimensional image to obtain a compressed three-dimensional image;

[0274] Based on the compressed three-dimensional image, determine the steganographic image obtained by steganographically writing the target image into the three-dimensional image.

[0275] An embodiment of this specification provides an image processing device. By obtaining a target image to be processed, where the target image includes privacy information of a target user; selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions; for each divided region, performing the following processing respectively to obtain three preselected stego images corresponding to each region: writing the RGB information in the target image into the region in a steganographic manner using a first steganographic encoder; writing the NIR information in the target image into the region in a steganographic manner using a second steganographic encoder; writing the RGB information and NIR information in the target image into the region in a steganographic manner using a third steganographic encoder; determining a stego image for steganographically embedding the target image into the three-dimensional image based on the preselected stego images corresponding to multiple different regions, and performing service processing on the target service based on the stego image. In this way, not only can the privacy information of the RGB information in the image be desensitized, but also the privacy information of the NIR information in the image can be desensitized, that is, through steganographic processing, the RGB information and NIR information are steganographically embedded into a three-dimensional image without the risk of privacy leakage, and at the same time, the steganographic processing of the RGB information and NIR information is completed. This processing method does not damage the structure of the image and is friendly to image compression, thereby reducing the occupancy of bandwidth and storage by biometric recognition. Moreover, this solution only needs to store the three-dimensional image, that is, it can complete the privacy protection process as long as it uses less than 1 / 3 of the original bandwidth and storage, not only does not increase the image volume, but instead greatly reduces the image volume. In addition, in terms of steganography technology, different from the traditional steganographic processing of the entire information, this solution proposes a local-global steganographic method, that is, steganographic processing is performed through the local regions of the three-dimensional image. On the one hand, the quality of steganography is improved, and on the other hand, the attacker cannot determine in which region the steganographic processing is performed, so that the privacy information is difficult to be leaked, and the security of steganography is improved.

[0276] Embodiment Seven

[0277] Further, based on the above Figures 1 to 5 shown method, one or more embodiments of this specification further provide a storage medium for storing computer-executable instruction information. In a specific embodiment, the storage medium can be a USB flash drive, an optical disc, a hard disk, etc. When the computer-executable instruction information stored in the storage medium is executed by a processor, the following process can be implemented:

[0278] Obtain a target image to be processed, where the target image includes privacy information of a target user;

[0279] Select a corresponding three-dimensional image for the target image and divide the three-dimensional image into multiple different regions;

[0280] For each of the divided regions, the following processing is performed respectively to obtain three preselected stego-images corresponding to each of the regions:

[0281] Use the first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the region in a steganographic manner;

[0282] Use the second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the region in a steganographic manner;

[0283] Use the third stego-encoder obtained by pre-training through a model to write the RGB information and NIR information in the target image into the region in a steganographic manner;

[0284] Based on the preselected stego-images corresponding to the multiple different regions, determine the stego-image for steganographically embedding the target image into the three-dimensional image, and perform service processing on the target service based on the stego-image.

[0285] In the embodiments of this specification, the target image is a user biometric image for biometric identification,

[0286] The performing service processing on the target service based on the stego-image includes:

[0287] Perform biometric identification processing on the target user based on the stego-image;

[0288] The method further includes:

[0289] Delete the target image.

[0290] In the embodiments of this specification, the performing biometric identification processing on the target user based on the stego-image includes:

[0291] Send the stego-image to the server, and the stego-image is used to trigger the server to perform biometric identification processing on the target user based on the pre-stored reference user biometric image and the stego-image;

[0292] Receive the biometric identification result of performing biometric identification processing on the target user sent by the server.

[0293] In the embodiments of this specification, the determining the stego-image for steganographically embedding the target image into the three-dimensional image based on the preselected stego-images corresponding to the multiple different regions includes:

[0294] Obtain the compression volume of the preselected stego-images corresponding to the multiple different regions, and the peak signal-to-noise ratio PSNR of the images of the multiple different regions;

[0295] Based on the compressed volumes of the preselected stego-images corresponding to multiple different regions, the peak signal-to-noise ratio (PSNR) of the images of the multiple different regions, and the corresponding weights, determine the stego scores of the preselected stego-images corresponding to each region respectively, where the stego scores are used to characterize the degree of non-identifiability after the information of the target image is written into the corresponding regions;

[0296] Obtain the first region corresponding to the preselected stego-image with a stego score greater than a preset threshold, and splice the first region with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image;

[0297] Based on the spliced three-dimensional image, determine the stego-image obtained by steganographically embedding the target image into the three-dimensional image.

[0298] In the embodiments of this specification, the determining the stego-image obtained by steganographically embedding the target image into the three-dimensional image based on the spliced three-dimensional image includes:

[0299] Adjust the resolution of the spliced three-dimensional image to be the same as the resolution of the three-dimensional image, and based on the adjusted spliced three-dimensional image, determine the stego-image obtained by steganographically embedding the target image into the three-dimensional image.

[0300] In the embodiments of this specification, the splicing the first region with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image includes:

[0301] Perform color correction processing on the image of the first region to obtain the first region after the first correction;

[0302] Splice the first region after the first correction with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

[0303] In the embodiments of this specification, the performing color correction processing on the image of the first region to obtain the first region after the first correction includes:

[0304] Based on the color information of the first region and the color information of the regions other than the region corresponding to the first region among the multiple different regions, determine the Gaussian parameters;

[0305] Based on the Gaussian parameters, the color information of the first region, and the color information of the regions other than the region corresponding to the first region among the multiple different regions, perform Gaussian adjustment on the first region and the regions other than the region corresponding to the first region among the multiple different regions respectively to obtain the adjusted color information of the first region and the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions;

[0306] Based on the color information of the first region, the adjusted color information of the first region, and the adjusted color information of the regions other than the region corresponding to the first region among the multiple different regions, perform color correction processing on the image of the first region to obtain the first corrected first region.

[0307] In the embodiments of the present specification, the splicing of the indicated first region with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image includes:

[0308] Perform correction processing on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the second corrected first region;

[0309] Splice the second corrected first region with the regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

[0310] In the embodiments of the present specification, the performing correction processing on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain the second corrected first region includes:

[0311] Input the image of the first region into a pre-trained image enhancement model, and perform correction processing on the image texture, shape, and spatial relationship corresponding to the image of the first region through the image enhancement model to obtain the second corrected first region, where the image enhancement model is a model obtained after model training based on image samples.

[0312] In the embodiments of the present specification, it further includes:

[0313] Obtain a training sample image and a corresponding three-dimensional sample image, where the training sample image includes the user's privacy information;

[0314] Divide the three-dimensional sample image into multiple different sample image regions;

[0315] For each divided sample image region, respectively perform the following processing:

[0316] Through the RGB information in the training sample image and the sample image region, jointly train the first steganographic encoder and the corresponding first steganographic decoder to obtain the trained first steganographic encoder, where the first steganographic decoder is used to perform restoration processing on the steganographically processed training sample image;

[0317] Jointly train the second steganographic encoder and the corresponding second steganographic decoder based on the NIR information and the sample image region in the training sample image to obtain the trained second steganographic encoder, where the second steganographic decoder is used to perform restoration processing on the steganographically processed training sample image;

[0318] Jointly train the third steganographic encoder and the corresponding third steganographic decoder based on the RGB information and NIR information in the training sample image, and the sample image region, to obtain the trained third steganographic encoder, where the third steganographic decoder is used to perform restoration processing on the steganographically processed training sample image.

[0319] In the embodiments of this specification, the joint training of the first steganographic encoder and the corresponding first steganographic decoder based on the RGB information and the sample image region in the training sample image to obtain the trained first steganographic encoder includes:

[0320] Input the RGB information and the sample image region in the training sample image into the first steganographic encoder to obtain the steganographically processed training sample image;

[0321] Input the steganographically processed training sample image into the first steganographic decoder to perform restoration processing on the steganographically processed training sample image through the first steganographic decoder, and obtain the RGB information in the reconstructed training sample image;

[0322] Based on the RGB information in the training sample image, the steganographically processed training sample image, the RGB information in the reconstructed training sample image, and a preset loss function, determine whether the first steganographic encoder and the first steganographic decoder converge. If not, continue to train the first steganographic encoder and the first steganographic decoder based on the training sample image and the sample image region until the first steganographic encoder and the first steganographic decoder converge to obtain the trained first steganographic encoder.

[0323] In the embodiments of this specification, the loss function is composed of the maximum similarity between the three-dimensional image constructed by the steganographically processed training sample image and the three-dimensional sample image, and the maximum similarity between the reconstructed RGB information and the RGB information in the training sample image.

[0324] In the embodiments of this specification, the dividing the three-dimensional image into multiple different regions includes:

[0325] The three-dimensional image is divided into four different regions, namely, the region of the upper left part of the three-dimensional image, the region of the lower left part of the three-dimensional image, the region of the upper right part of the three-dimensional image, and the region of the lower right part of the three-dimensional image.

[0326] In an embodiment of the present specification, determining a stego image in which the target image is hidden in the three-dimensional image based on the spliced three-dimensional image includes:

[0327] Performing image compression processing on the spliced three-dimensional image to obtain a compressed three-dimensional image;

[0328] Based on the compressed three-dimensional image, determining a stego image in which the target image is hidden in the three-dimensional image.

[0329] An embodiment of the present specification provides a storage medium. By obtaining a target image to be processed, where the target image includes privacy information of a target user; selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions; for each divided region, respectively performing the following processing to obtain three preselected stego images corresponding to each region: writing the RGB information in the target image into the region in a steganographic manner using a first steganographic encoder; writing the NIR information in the target image into the region in a steganographic manner using a second steganographic encoder; writing the RGB information and the NIR information in the target image into the region in a steganographic manner using a third steganographic encoder; determining a stego image in which the target image is hidden in the three-dimensional image based on the preselected stego images corresponding to the multiple different regions, and performing service processing on the target service based on the stego image. In this way, not only can the privacy information of the RGB information in the image be desensitized, but also the privacy information of the NIR information in the image can be desensitized, that is, through steganographic processing, the RGB information and the NIR information are hidden in the three-dimensional image without the risk of privacy leakage. At the same time, the steganographic processing of the RGB information and the NIR information is completed. This processing method does not damage the structure of the image and is friendly to image compression, thereby reducing the occupancy of bandwidth and storage by biometric recognition. Moreover, this solution only needs to store the three-dimensional image, that is, only less than 1 / 3 of the original bandwidth and storage are used to complete the privacy protection processing. Not only does it not increase the image volume, but it greatly reduces the image volume. In addition, in terms of steganography technology, different from the traditional steganographic processing of the entire information, this solution proposes a local-global steganographic method, that is, steganographic processing is performed through the local region of the three-dimensional image. On the one hand, the quality of steganography is improved, and on the other hand, the attacker cannot determine in which region the steganographic processing is performed, so that the privacy information is difficult to be leaked, and the security of steganography is improved.

[0330] The foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0331] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many improvements to method flows today can be regarded as direct improvements to hardware circuit structures. Almost all designers obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement to a method flow cannot be implemented using a hardware entity module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logical function is determined by the user programming the device. Designers can program a digital system "integrated" on a single PLD by themselves, without having to ask a chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a Hardware Description Language (HDL), and there is not just one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones currently are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that by simply performing a little logical programming on the method flow using the above-mentioned several hardware description languages and programming it into an integrated circuit, it is easy to obtain the hardware circuit that implements the logical method flow.

[0332] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to make the controller implement the same function in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.

[0333] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0334] For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0335] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0336] The embodiments of this specification are described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of this specification. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable serial-parallel devices for fraud cases to generate a machine, such that the instructions executed by the processors of the computer or other programmable serial-parallel devices for fraud cases generate means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0337] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable serial-parallel device for fraud cases to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0338] These computer program instructions can also be loaded onto a computer or other programmable serial-parallel device for fraud cases, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0339] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0340] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0341] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0342] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0343] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, one or more embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0344] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0345] Each embodiment in this specification is described in a progressive manner. For the parts that are the same or similar among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, reference can be made to the partial description of the method embodiment.

[0346] The above is only the embodiment of this specification and is not intended to limit this application. For those skilled in the art, various changes and modifications can be made to this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included within the scope of the claims of this specification.

Claims

1. A method for processing an image, the method comprising: Obtaining a target image to be processed, where the target image includes privacy information of a target user; Selecting a corresponding three-dimensional image for the target image and dividing the three-dimensional image into multiple different regions; For each of the divided regions, performing the following processing respectively to obtain three preselected stego-images corresponding to each region: Writing the RGB information in the target image into the region in a steganographic manner using a first steganographic encoder obtained by pre-training through a model; Writing the NIR information in the target image into the region in a steganographic manner using a second steganographic encoder obtained by pre-training through a model; Writing the RGB information and NIR information in the target image into the region in a steganographic manner using a third steganographic encoder obtained by pre-training through a model; Based on the preselected stego-images corresponding to the multiple different regions, determining a stego-image for steganographically writing the target image into the three-dimensional image, and performing business processing on a target business based on the stego-image.

2. The method according to claim 1, wherein the target image is a user biometric image for biometric identification, The performing business processing on the target business based on the stego-image includes: Performing biometric identification processing on the target user based on the stego-image; The method further includes: Deleting the target image.

3. The method according to claim 2, wherein the performing biometric identification processing on the target user based on the stego-image includes: Sending the stego-image to a server, where the stego-image is used to trigger the server to perform biometric identification processing on the target user based on a pre-stored reference user biometric image and the stego-image; Receiving a biometric identification result of performing biometric identification processing on the target user sent by the server.

4. The method according to claim 1, wherein the determining a stego-image for steganographically writing the target image into the three-dimensional image based on the preselected stego-images corresponding to the multiple different regions includes: Obtaining the compression volume of the preselected stego-images corresponding to the multiple different regions, and the peak signal-to-noise ratio PSNR of the images of the multiple different regions; Based on the compression volume of the preselected stego-images corresponding to the multiple different regions, the peak signal-to-noise ratio PSNR of the images of the multiple different regions, and corresponding weights, respectively determining the steganographic scores of the preselected stego-images corresponding to each region, where the steganographic scores are used to characterize the degree of non-identifiability after the information of the target image is written into the corresponding region; Obtaining a first region corresponding to a preselected stego-image with a steganographic score greater than a preset threshold, and splicing the first region with the regions other than the regions corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image; Based on the spliced three-dimensional image, determining a stego-image for steganographically writing the target image into the three-dimensional image.

5. The method according to claim 4, wherein the determining a stego-image for steganographically writing the target image into the three-dimensional image based on the spliced three-dimensional image includes: Adjust the resolution of the spliced three-dimensional image to be the same as that of the three-dimensional image, and based on the adjusted spliced three-dimensional image, determine a stego image obtained by steganographically embedding the target image into the three-dimensional image.

6. The method according to claim 4 or 5, wherein the splicing the first region with regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image comprises: Perform color correction processing on the image of the first region to obtain a first corrected first region; Splice the first corrected first region with regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

7. The method according to claim 6, wherein the performing color correction processing on the image of the first region to obtain a first corrected first region comprises: Determine Gaussian parameters based on the color information of the first region and the color information of regions other than the region corresponding to the first region among the multiple different regions; Based on the Gaussian parameters, the color information of the first region, and the color information of regions other than the region corresponding to the first region among the multiple different regions, perform Gaussian adjustment on the first region and regions other than the region corresponding to the first region among the multiple different regions respectively, to obtain the adjusted color information of the first region and the adjusted color information of regions other than the region corresponding to the first region among the multiple different regions; Based on the color information of the first region, the adjusted color information of the first region, and the adjusted color information of regions other than the region corresponding to the first region among the multiple different regions, perform color correction processing on the image of the first region to obtain a first corrected first region.

8. The method according to claim 4 or 5, wherein the splicing the first region with regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image comprises: Perform correction processing on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain a second corrected first region; Splice the second corrected first region with regions other than the region corresponding to the first region among the multiple different regions to obtain a spliced three-dimensional image.

9. The method according to claim 8, wherein the performing correction processing on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain a second corrected first region comprises: Input the image of the first region into a pre-trained image enhancement model, and through the image enhancement model, perform correction processing on the image texture, shape, and spatial relationship corresponding to the image of the first region to obtain a second corrected first region, where the image enhancement model is a model obtained by training based on image samples.

10. The method according to claim 5, wherein the determining, based on the spliced three-dimensional image, a stego image obtained by steganographically embedding the target image into the three-dimensional image comprises: Perform image compression processing on the spliced three-dimensional image to obtain a compressed three-dimensional image; Based on the compressed three-dimensional image, determine a stego image in which the target image is hidden in the three-dimensional image.

11. The method according to claim 1, wherein the method further comprises: Obtain a training sample image and a corresponding three-dimensional sample image, where the training sample image includes user privacy information; Divide the three-dimensional sample image into multiple different sample image regions; For each of the divided sample image regions, perform the following processing respectively: Through the RGB information in the training sample image and the sample image region, jointly train the first steganographic encoder and the corresponding first steganographic decoder to obtain a trained first steganographic encoder, and the first steganographic decoder is used to perform restoration processing on the steganographically processed training sample image; Through the NIR information in the training sample image and the sample image region, jointly train the second steganographic encoder and the corresponding second steganographic decoder to obtain a trained second steganographic encoder, and the second steganographic decoder is used to perform restoration processing on the steganographically processed training sample image; Through the RGB information, NIR information in the training sample image, and the sample image region, jointly train the third steganographic encoder and the corresponding third steganographic decoder to obtain a trained third steganographic encoder, and the third steganographic decoder is used to perform restoration processing on the steganographically processed training sample image.

12. The method according to claim 11, wherein the step of jointly training the first steganographic encoder and the corresponding first steganographic decoder through the RGB information in the training sample image and the sample image region to obtain a trained first steganographic encoder includes: Input the RGB information in the training sample image and the sample image region into the first steganographic encoder to obtain a steganographically processed training sample image; Input the steganographically processed training sample image into the first steganographic decoder to perform restoration processing on the steganographically processed training sample image through the first steganographic decoder to obtain the RGB information in the reconstructed training sample image; Based on the RGB information in the training sample image, the steganographically processed training sample image, the RGB information in the reconstructed training sample image, and a preset loss function, determine whether the first steganographic encoder and the first steganographic decoder converge. If not, continue to train the first steganographic encoder and the first steganographic decoder based on the training sample image and the sample image region until the first steganographic encoder and the first steganographic decoder converge to obtain a trained first steganographic encoder.

13. The method according to claim 12, wherein the loss function is composed of the maximum similarity between the three-dimensional image constructed by the steganographically processed training sample image and the three-dimensional sample image, and the maximum similarity between the reconstructed RGB information and the RGB information in the training sample image.

14. The method according to claim 1, wherein the dividing the three-dimensional image into a plurality of different regions includes: Dividing the three-dimensional image into four different regions, namely, the region of the upper left part of the three-dimensional image, the region of the lower left part of the three-dimensional image, the region of the upper right part of the three-dimensional image, and the region of the lower right part of the three-dimensional image.

15. An image processing apparatus, the apparatus comprising: An image acquisition module that acquires a target image to be processed, where the target image includes privacy information of a target user; A three-dimensional image acquisition module that selects a corresponding three-dimensional image for the target image and divides the three-dimensional image into a plurality of different regions; A steganography module that, for each of the divided regions, respectively performs the following processing to obtain three preselected steganographic images corresponding to each of the regions: Writing the RGB information in the target image into the region in a steganographic manner using a first steganographic encoder obtained by pre-training through a model; Writing the NIR information in the target image into the region in a steganographic manner using a second steganographic encoder obtained by pre-training through a model; Writing the RGB information and the NIR information in the target image into the region in a steganographic manner using a third steganographic encoder obtained by pre-training through a model; A processing module that determines a steganographic image for steganographically writing the target image into the three-dimensional image based on the preselected steganographic images corresponding to the plurality of different regions, and performs service processing on a target service based on the steganographic image.

16. An image processing device, the image processing device comprising: A processor; And A memory arranged to store computer-executable instructions, the executable instructions, when executed, causing the processor to: Acquire a target image to be processed, where the target image includes privacy information of a target user; Select a corresponding three-dimensional image for the target image and divide the three-dimensional image into a plurality of different regions; For each of the divided regions, respectively perform the following processing to obtain three preselected steganographic images corresponding to each of the regions: Writing the RGB information in the target image into the region in a steganographic manner using a first steganographic encoder obtained by pre-training through a model; Writing the NIR information in the target image into the region in a steganographic manner using a second steganographic encoder obtained by pre-training through a model; Writing the RGB information and the NIR information in the target image into the region in a steganographic manner using a third steganographic encoder obtained by pre-training through a model; Determine a steganographic image for steganographically writing the target image into the three-dimensional image based on the preselected steganographic images corresponding to the plurality of different regions, and perform service processing on a target service based on the steganographic image.

17. A storage medium for storing computer-executable instructions, the executable instructions, when executed by a processor, implement the following process: Acquire a target image to be processed, where the target image includes privacy information of a target user; Select a corresponding three-dimensional image for the target image, and divide the three-dimensional image into multiple different regions; For each of the divided regions, perform the following processing respectively to obtain three preselected stego-images corresponding to each of the regions: Use a first stego-encoder obtained by pre-training through a model to write the RGB information in the target image into the region in a steganographic manner; Use a second stego-encoder obtained by pre-training through a model to write the NIR information in the target image into the region in a steganographic manner; Use a third stego-encoder obtained by pre-training through a model to write the RGB information and NIR information in the target image into the region in a steganographic manner; Based on the preselected stego-images corresponding to the multiple different regions, determine a stego-image for steganographically embedding the target image into the three-dimensional image, and perform business processing on the target service based on the stego-image.

Images