Image processing method, image model training method, device, medium and equipment

Abstract

Embodiments of the present specification disclose an image processing method, an image model training method, an apparatus, a storage medium and an equipment. The method comprises: acquiring a biological recognition image of each user in a plurality of users, and extracting an image feature of the biological recognition image; performing feature fusion processing on the image feature of the biological recognition image of each user, to generate a fused image feature corresponding to the biological recognition image of each user, the fused image feature containing associated features between the image features of the biological recognition images of the plurality of users; and generating a desensitized image corresponding to the biological recognition image of each user based on the fused image feature.

Description

Technical Field

[0001] This specification relates to the field of machine learning technology, and in particular to an image processing method, an image model training method, an apparatus, a storage medium, and a device. Background Technology

[0002] Biometric technology has been applied in many scenarios in recent years, such as facial recognition payment, fingerprint attendance, and iris scanners. Protecting the privacy of users' biometric images has become a key concern.

[0003] In related technical solutions, data encryption technology is used to protect the privacy of users' biometric images. For example, the biometric image is encrypted during image transmission or storage, and then decrypted during the computation phase. However, this technical solution has low security and is easily cracked by attackers, making it difficult to guarantee the security of users' privacy information in most scenarios.

[0004] Therefore, increasing the cost of attacks for attackers and improving the security of privacy protection for biometric images have become urgent technical challenges. Summary of the Invention

[0005] This specification provides an image processing method, an image model training method, an apparatus, a storage medium, and a device that can increase the attack cost for attackers and improve the security of privacy protection for biometric images.

[0006] Firstly, embodiments of this specification provide an image processing method, including:

[0007] Acquire biometric images of each user from multiple users, and extract image features from the biometric images;

[0008] The image features of the biometric images of each user are subjected to feature fusion processing to generate fused image features corresponding to the biometric images of each user. The fused image features include the correlation features between the image features of the biometric images of the multiple users.

[0009] Based on the fused image features corresponding to the biometric images of each user, a desensitized image corresponding to the biometric image of each user is generated.

[0010] Secondly, embodiments of this specification provide an image model training method, wherein the image model includes an image desensitization sub-model, and the method includes:

[0011] Acquire biometric images of each user from multiple users, and extract image features from each user's biometric images;

[0012] Feature fusion processing is performed on the image features of each user's biometric image to generate fused image features corresponding to each user's biometric image. The fused image features include correlation features between the image features of the biometric images of the multiple users.

[0013] The fused image features corresponding to each user are input into the image desensitization sub-model to obtain the desensitized image corresponding to the biometric image of each user.

[0014] Based on the desensitized image and the biometric image, the model loss of the image model is determined, and the model loss includes the desensitization loss;

[0015] The image model is trained based on the model loss.

[0016] Thirdly, embodiments of this specification provide an image processing apparatus, including:

[0017] The feature extraction module is used to acquire biometric images of each user among multiple users and extract image features from the biometric images;

[0018] The feature fusion module is used to perform feature fusion processing on the image features of the biometric images of each user to generate fused image features corresponding to the biometric images of each user. The fused image features include the correlation features between the image features of the biometric images of the multiple users.

[0019] The desensitization processing module is used to generate desensitized images corresponding to the biometric images of each user based on the fused image features corresponding to the biometric images of each user.

[0020] Fourthly, embodiments of this specification provide an image model training apparatus, wherein the image model includes an image desensitization sub-model, and the apparatus includes:

[0021] The feature extraction module is used to acquire biometric images of each user among multiple users and extract the image features of each user's biometric images;

[0022] The feature fusion module is used to perform feature fusion processing on the image features of each user's biometric image to generate fused image features corresponding to each user's biometric image. The fused image features include correlation features between the image features of the biometric images of the multiple users.

[0023] The image desensitization module is used to input the fused image features corresponding to each user into the image desensitization sub-model to obtain the desensitized image corresponding to the biometric image of each user.

[0024] The loss determination module is used to determine the model loss of the image model based on the desensitized image and the biometric image, wherein the model loss includes the desensitization loss;

[0025] The model training module is used to train the image model based on the model loss.

[0026] Fifthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method described above.

[0027] Sixthly, embodiments of this specification provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of the method described above.

[0028] In a seventh aspect, embodiments of this specification provide an electronic device, including: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method described above.

[0029] According to the technical solution of the embodiments of this specification, multiple biometric images of multiple users are used for joint desensitization, so that the biometric images of each user are interconnected and dependent on each other during the desensitization stage. Therefore, the desensitized images of these multiple users also need to be used for desensitization during the anti-desensitization stage, thereby increasing the attack cost for attackers (from obtaining a single image sample to obtaining multiple image samples, increasing the difficulty of sample acquisition) and improving the security of privacy protection of biometric images. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the implementation environment of an image processing method provided according to an embodiment of this specification;

[0032] Figure 2 This is a schematic flowchart of an image processing method provided according to an embodiment of this specification;

[0033] Figure 3 This is a schematic diagram of the feature fusion process provided in the embodiments of this specification;

[0034] Figure 4 This is a flowchart illustrating an image model training method provided according to an embodiment of this specification.

[0035] Figure 5 This is a schematic diagram of the process for determining a user risk coefficient according to the embodiments provided in this specification;

[0036] Figure 6 This is a flowchart illustrating another image model training method provided according to an embodiment of this specification;

[0037] Figure 7 This is a schematic diagram of the structure of the image processing apparatus provided according to the embodiments of this specification;

[0038] Figure 8 This is a schematic diagram of the image model training device provided according to the embodiments of this specification;

[0039] Figure 9 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this specification. Detailed Implementation

[0040] To make the features and advantages of this specification more apparent and understandable, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative effort are within the scope of protection of this specification.

[0041] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0042] Biometric images: including but not limited to various biometric images that can be used for identity authentication, such as faces, fingerprints, irises, etc.

[0043] Privacy risk assessment: refers to predicting the risk of leakage of each user's biometric image.

[0044] Multi-user joint desensitization: Unlike related technical solutions that desensitize a single image for a single user, the multi-user joint desensitization method considers multiple images from multiple users and performs joint desensitization based on the relationships between the multiple images.

[0045] In related technical solutions, image de-identification algorithms are used to protect the privacy of biometric images. For example, a de-identification model and a de-identification model are trained on a user's biometric image. During image transmission or storage, the de-identification model encrypts and de-identifies the user's biometric image to protect privacy, while the de-identification model reverses the de-identification during the computation phase. However, in this technical solution, once an attacker obtains a large amount of original and de-identified data, they can train a de-identification model, thereby enabling large-scale theft of user privacy.

[0046] Based on the above, this specification provides an image processing method and an image model training method. According to the technical solution of this specification, multiple biometric images from multiple users are used for joint desensitization, making the biometric images of each user interconnected and dependent during the desensitization stage. Therefore, the desensitized images of these users are also needed for desensitization during the desensitization stage, thereby increasing the attack cost for attackers (from obtaining a single image sample to obtaining multiple image samples, increasing the difficulty of sample acquisition) and improving the security of privacy protection for biometric images.

[0047] The technical solutions of the embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0048] Figure 1 This is a schematic diagram illustrating the implementation environment of an image processing method provided in the embodiments of this specification.

[0049] See Figure 1 The implementation environment may include terminal 110 and server 140.

[0050] Terminal 110 is connected to server 140 via a wireless or wired network. Optionally, terminal 110 may be a smartphone, tablet, laptop, desktop computer, smartwatch, etc., but is not limited to these. Terminal 110 has applications installed and running that support image processing methods.

[0051] Server 140 is a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Server 140 provides background services for applications running on terminal 110.

[0052] Those skilled in the art will understand that the number of terminals described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more, in which case other terminals may also be included in the above implementation environment. This specification does not limit the number of terminals or the type of device in the embodiments.

[0053] After introducing the implementation environment of the embodiments of this specification, the application scenarios of the embodiments of this specification will be described below in conjunction with the above implementation environment. In the following description, the terminal is also the terminal 110 in the above implementation environment, and the server is also the server 140 in the above implementation environment. The technical solutions provided by the embodiments of this specification can be applied to image desensitization processing, such as face recognition image desensitization, fingerprint recognition image desensitization, and iris recognition image desensitization.

[0054] Taking the technical solution provided in the embodiments of this specification as an example in a face recognition scenario, with user authorization, face recognition images of each user from multiple users within a predetermined time period are obtained, and image features of each user's face recognition images are extracted; feature fusion processing is performed on the image features of each user's face recognition images to generate fused image features corresponding to each user's face recognition images, and the fused image features contain the correlation features between the image features of multiple users' face recognition images; based on the fused image features corresponding to each user's face recognition images, desensitized images corresponding to each user's face recognition images are generated.

[0055] It should be noted that the above description is based on the application of the technical solution provided in the embodiments of this specification to a face recognition scenario. The technical solution provided in the embodiments of this specification can also be applied to other appropriate image desensitization processing scenarios. The implementation process is the same as the above description and belongs to the same inventive concept, so it will not be repeated here.

[0056] It should be noted that the steps in the image processing method in the example embodiments of this specification may be partially executed by the client, partially executed by the server, or entirely executed by the server or entirely by the client. This specification does not impose any special limitations on this.

[0057] based on Figure 1 The implementation environment shown below will be combined with... Figures 2-4 This specification provides a detailed description of the image processing methods provided in the embodiments. It should be noted that the above-described implementation environments are shown only to facilitate understanding of the spirit and principles of this specification, and the embodiments are not limited in any way. Rather, the embodiments can be applied to any applicable scenario.

[0058] Figure 2This is a schematic flowchart illustrating an image processing method provided in an embodiment of this specification. This image processing method can be executed by a device with computing capabilities, such as a terminal device or a server. Figure 2 As shown, the image processing method in the embodiments of this specification may include the following steps S210 to S230.

[0059] In step S210, biometric images of each user among multiple users are acquired, and image features of the biometric images are extracted.

[0060] In an example embodiment, the biometric image can be a face recognition image or a fingerprint recognition image, etc. With user authorization, biometric images of each user within a predetermined time period are acquired, and image features of each acquired user's biometric image are extracted. Taking a face recognition scenario as an example, if 10 authorized users perform face recognition within 5 minutes, face recognition images of the 10 authorized users within 5 minutes are acquired, and image features of each acquired user's face recognition image are extracted.

[0061] It should be noted that although facial recognition images or fingerprint recognition images have been used in the description, those skilled in the art should understand that biometric images can also be other suitable images, such as iris recognition images or retinal recognition images, which are also within the scope of the embodiments in this specification.

[0062] Furthermore, in the example embodiment, image features of the biometric image are extracted using a feature extraction model. The feature extraction model can be a convolutional neural network model, or other suitable feature extraction models such as SIFT (Scale Invariant Feature Transform) or HOG (Histogram of Oriented Gradients) feature extraction models, which are also within the scope of the embodiments described in this specification.

[0063] In step S220, feature fusion processing is performed on the image features of each user's biometric image to generate fused image features corresponding to each user's biometric image. The fused image features contain the correlation features between the image features of multiple users' biometric images.

[0064] In the example embodiment, feature fusion processing refers to fusing image features between biometric images of different users. Feature fusion processing may include one or more of the following methods: weighted fusion processing, feature addition processing, and feature multiplication processing. Feature fusion processing is performed on the image features of biometric images of different users among the aforementioned multiple users to generate fused image features corresponding to the biometric images of each user. The fused image features contain the correlation features between the image features of the biometric images of multiple users.

[0065] For example, the image features of each user's biometric image can be weighted and fused with the image features of other users' biometric images to generate fused image features corresponding to each user's biometric image. The fused image features corresponding to each user's biometric image contain the correlation features between the image features of that user and the image features of other users' biometric images.

[0066] Suppose there are three users, A, B, and C. The image features of the biometric images of users A, B, and C are fused to generate fused image features corresponding to the biometric images of users A, B, and C. The fused image features of user A's biometric image include user A's image features, the correlation features between user A and user B's image features, and the correlation features between user A and user C's image features.

[0067] It should be noted that although the above feature fusion processing method has been used as an example, those skilled in the art should understand that feature fusion processing can also be other appropriate fusion processing, such as feature splicing processing, which is also within the scope of the embodiments in this specification.

[0068] Furthermore, in some example embodiments, feature fusion processing is performed on the image features of each user's biometric image using a feature fusion processing model. For example, the feature weights of the image features of each user's biometric image are determined, and the image features of each user's biometric image are weighted and fused using a convolutional neural network to generate fused image features corresponding to each user's biometric image. The feature weights can be determined based on the privacy leakage risk of the user's biometric image; the higher the privacy leakage risk of the biometric image, the greater the feature weight.

[0069] It should be noted that although the feature fusion model is illustrated using a convolutional neural network as an example, those skilled in the art should understand that the feature fusion processing model can also be other suitable models, such as the Transformer model, etc., and the embodiments in this specification do not impose any special limitations on this.

[0070] In step S230, based on the fused image features corresponding to the biometric images of each user, a desensitized image corresponding to the biometric image of each user is generated.

[0071] In the example embodiment, image desensitization refers to a technique that encrypts, transforms, or modifies sensitive parts of an image, such as facial features. Image desensitization processing is performed on the fused image features corresponding to each user's biometric image to generate desensitized images corresponding to each user's biometric image.

[0072] For example, the fused image features corresponding to the biometric images of each user are input into the image desensitization model; through the image desensitization model, desensitized images corresponding to the biometric images of each user are generated. The image desensitization model can be a neural network model, such as a convolutional neural network model.

[0073] It should be noted that although the image desensitization model is described using a convolutional neural network as an example, those skilled in the art should understand that the image desensitization model can also be other suitable machine learning models such as the Transformer model, and the embodiments in this specification do not impose any special limitations on it.

[0074] according to Figure 2 The technical solution in the example embodiment uses multiple biometric images from multiple users for joint desensitization, making the biometric images of each user interconnected and dependent during the desensitization stage. Therefore, the desensitized images of these users are also needed for desensitization during the desensitization stage, thereby increasing the attack cost for attackers (from obtaining a single image sample to obtaining multiple image samples, increasing the difficulty of sample acquisition) and improving the security of privacy protection for biometric images.

[0075] Furthermore, in some example embodiments, the image processing method further includes: acquiring multiple associated desensitized images corresponding to the target desensitized image of the target user; and, based on the multiple associated desensitized images, performing desensitization processing on the target desensitized image using a desensitization model to obtain a biometric image corresponding to the target desensitized image. The desensitization model can be a neural network model, such as a convolutional neural network model. For example, suppose the target desensitized image of the target user corresponds to 10 associated desensitized images of 10 users; acquire 10 associated desensitized images of the 10 users corresponding to the target desensitized image; and, based on the 10 associated desensitized images, perform desensitization processing on the target desensitized image using a desensitization model to obtain a biometric image of the target user corresponding to the target desensitized image.

[0076] It should be noted that although the desensitization model is illustrated using a convolutional neural network as an example, those skilled in the art should understand that the desensitization model can also be other suitable machine learning models such as the Transformer model, and the embodiments in this specification do not impose any special limitations on it.

[0077] According to the technical solution in the above example embodiment, since multiple biometric images of multiple users, such as N images, are used for joint desensitization during the desensitization stage, the biometric images of each user are interconnected and dependent on each other during the desensitization stage. Therefore, in the anti-desensitization stage, it is necessary to find N associated desensitized images corresponding to each desensitized image and perform corresponding anti-desensitization processing, which increases the attack cost for attackers (not only do they need to obtain a single desensitized sample, but they also need to obtain the corresponding associated desensitized samples), thereby improving the security of privacy protection for biometric images.

[0078] Figure 3 This is a schematic diagram of the feature fusion process provided in the embodiments of this specification.

[0079] Reference Figure 3 As shown, in step S310, the user risk coefficient corresponding to each user is determined. The user risk coefficient represents the feature weight of the image features of each user's biometric image.

[0080] In the example embodiment, the user risk coefficient represents the magnitude of the privacy leakage risk of a user's biometric image, and can further represent the feature weight of the image features of each user's biometric image. For example, if the user risk coefficient is large, it indicates that the user's biometric image has a greater risk of privacy leakage, requiring a greater degree of fusion processing of the image features of the user's biometric image; therefore, the feature weight of the image features of the user's biometric image is large. Conversely, if the user risk coefficient is small, it indicates that the user's biometric image has a smaller risk of privacy leakage, requiring only a smaller degree of fusion processing of the image features of the user's biometric image; therefore, the feature weight of the image features of the user's biometric image is small.

[0081] Furthermore, the biometric characteristics of each user within a predetermined period are determined, including biometric frequency characteristics and scene frequency characteristics of each biometric scenario; based on the biometric characteristics, the user risk coefficient corresponding to each user is determined.

[0082] For example, suppose the biometric scenarios include high-risk scenarios, medium-risk scenarios, and low-risk scenarios. If the biometric frequency of a user in a predetermined period is c1, and the biometric frequencies in high-risk, medium-risk, and low-risk scenarios are c2, c3, and c4, then the user risk coefficient is obtained by weighting the user's biometric frequency and the biometric frequency of each biometric scenario.

[0083] In step S320, the image features of the biometric images corresponding to each user are weighted and fused based on the user risk coefficients corresponding to each user.

[0084] In the example embodiment, the image features of the biometric images corresponding to each user are weighted and fused using the user risk coefficient corresponding to each user as the weight to generate the fused image features corresponding to the biometric images of each user. For example, the image features of the biometric images of each user are weighted and fused using the fully connected layer of the neural network model to generate the fused image features corresponding to the biometric images of each user.

[0085] For example, using the user risk coefficient corresponding to each user as the weight, the image features of each user's biometric image among multiple users are weighted and fused with the image features of other users' biometric images to generate fused image features corresponding to each user's biometric image. The fused image features corresponding to each user's biometric image contain the correlation features between the image features of that user and the image features of other users' biometric images.

[0086] according to Figure 3 The technical solution in the example embodiment, on the one hand, considers that multiple images often come from multiple users, and the privacy leakage risks of multiple users are inconsistent (generally, the more frequent the biometric identification and the more open the scenario in which the biometric identification takes place, the higher the privacy leakage risk). Therefore, it is necessary to perceive the privacy leakage risk of different users, i.e., the user risk coefficient, to provide prior knowledge for subsequent adaptive multi-image desensitization. On the other hand, by performing weighted fusion processing on the image features of the biometric images corresponding to each user according to the user risk coefficient, the image features of the biometric images with higher user risk coefficients can be enhanced, while the image features of the biometric images with lower user risk coefficients can be weakened. This allows for deeper fusion processing of the biometric images of users with higher privacy leakage risks. Furthermore, since deeper fusion processing can be performed on the biometric images of users with higher privacy leakage risks, the security of privacy protection of biometric images can be further improved.

[0087] Furthermore, since the user risk coefficient for privacy leakage changes over time, in some example embodiments, the biometric features of each user are periodically acquired; the feature similarity between the user's biometric features in the current period and the biometric features in the previous period is determined; if the feature similarity is less than a predetermined threshold, the user risk coefficient corresponding to the user is adjusted based on the biometric features in the current period.

[0088] According to the technical solution in the above example embodiment, on the one hand, when the feature similarity between the biometric features of the current period and the biometric features of the previous period is less than a predetermined threshold, the user risk coefficient corresponding to the user is adjusted based on the biometric features of the current period, so that the user risk coefficient can be updated in a timely manner when the user risk coefficient changes significantly; on the other hand, the image features of the user's biometric image are weighted and fused according to the updated user risk coefficient, thereby further improving the security of privacy protection of biometric images.

[0089] Figure 4 This is a schematic flowchart illustrating an image model training method provided according to embodiments of this specification. This image model training method can be executed by a device with computing capabilities, such as a server or terminal device. Figure 4 As shown, the image model training method in the embodiments of this specification may include the following steps S410 to S450.

[0090] Reference Figure 4 As shown, in step S410, the biometric images of each user among multiple users are acquired, and the image features of each user's biometric image are extracted.

[0091] In the example embodiment, the biometric image can be a face recognition image or a fingerprint recognition image, etc. With user authorization, biometric images of each user from multiple users within a predetermined time period are acquired. Image features of the biometric image are extracted using a feature extraction model. The feature extraction model can be a convolutional neural network model, or other suitable feature extraction models such as SIFT (Scale Invariant Feature Transform) or HOG (Histogram of Oriented Gradients) feature extraction models, etc., which are also within the scope of the embodiments in this specification.

[0092] In step S420, the image features of each user's biometric image are fused to generate fused image features corresponding to each user's biometric image. The fused image features contain the correlation features between the image features of multiple users' biometric images.

[0093] In the example embodiment, feature fusion processing refers to fusing image features from biometric images of different users. Feature fusion processing can include one or more of the following methods: weighted fusion processing, feature addition processing, and feature multiplication processing. For example, the image model includes a feature fusion sub-model, which performs feature fusion processing on the image features of each user's biometric images. For instance, a convolutional neural network can be used to perform weighted fusion processing on the image features of each user's biometric images to generate fused image features corresponding to each user's biometric images.

[0094] It should be noted that although the feature fusion sub-model is described using a convolutional neural network as an example, those skilled in the art should understand that the feature fusion sub-model can also be other suitable models, such as the Transformer model, etc., and the embodiments in this specification do not impose any special limitations on it.

[0095] In step S430, the fused image features corresponding to each user are input into the image desensitization sub-model of the image model to obtain the desensitized image corresponding to the biometric image of each user.

[0096] In an example embodiment, the image model includes an image desensitization sub-model, which can be a neural network model, such as a convolutional neural network model. The input of the image desensitization sub-model is multiple fused image features after feature fusion processing, and the output is multiple corresponding desensitized images. For example, assuming the image desensitization sub-model is a convolutional neural network model, the fused image features corresponding to each user are input into the image desensitization sub-model of the image model to obtain desensitized images corresponding to the biometric images of each user.

[0097] It should be noted that although the image desensitization model is described using a convolutional neural network as an example, those skilled in the art should understand that the image desensitization model can also be other suitable machine learning models such as the Transformer model, and the embodiments in this specification do not impose any special limitations on it.

[0098] In step S440, the model loss of the image model is determined based on the desensitized image and the biometric image, and the model loss includes the desensitization loss.

[0099] In an example embodiment, the desensitization loss is used to make the user's desensitized image inconsistent with the original biometric image. The desensitization loss is determined by the difference between the user's desensitized image and the original biometric image. For example, the feature differences between the image features of the user's corresponding desensitized image and the image features of the biometric image are determined, and the desensitization loss of the image model is determined based on these feature differences.

[0100] In step S450, the image model is trained based on the model loss.

[0101] In the example embodiment, the model loss includes desensitization loss, and the image model is trained using gradient descent based on the model loss to adjust the model parameters of the image model.

[0102] For example, the ADAM (Adaptive Moment Estimation) optimizer is used to train an image model until it converges. The ADAM optimizer considers both the first moment estimate (mean of the gradient) and the second moment estimate (uncentered variance of the gradient) to calculate the update step size for model training, and then trains the image model based on this update step size. By training the image model using the ADAM optimizer, it can adaptively adjust from both the mean and squared gradient perspectives, rather than being directly determined by the current gradient, thus improving model training efficiency.

[0103] according to Figure 4 The technical solution in the example embodiment uses multiple biometric images from multiple users for joint desensitization, making the biometric images of each user interconnected and dependent during the desensitization stage. Therefore, the desensitized images of these users must also be used for desensitization during the desensitization stage, thereby increasing the attack cost for attackers (from obtaining a single image sample to obtaining multiple image samples, increasing the difficulty of sample acquisition) and improving the security of privacy protection for biometric images.

[0104] Furthermore, in the example embodiment, the model loss also includes a desensitization loss ranking loss, which is used to ensure that the forward ranking of the desensitization losses is consistent with the reverse ranking of the user risk coefficients. Training the image model based on the model loss includes: determining the risk coefficient ranking of multiple users' user risk coefficients based on the size of the user risk coefficients; determining the desensitization loss ranking based on the size of the desensitization losses; determining the desensitization loss ranking loss based on the desensitization loss ranking and the risk coefficient ranking; and training the image model based on the desensitization losses and the desensitization loss ranking loss.

[0105] For example, assume there are three users, A, B, and C, and the order of the user risk coefficients is User A > User B > User C. Then, the order of the corresponding desensitization losses is User A < User B < User C. Based on the magnitudes of the desensitization losses of the three users A, B, and C and the magnitudes of the user risk coefficients, determine the desensitization sorting loss corresponding to these three users. For example, if the desensitization loss of User A is greater than that of User B, the desensitization sorting loss is larger. Train the image model based on the desensitization loss and the desensitization sorting loss.

[0106] According to the technical solution in the above exemplary embodiment, by setting the desensitization sorting loss, it is possible to make the positive sorting of the magnitudes of the desensitization losses consistent with the reverse sorting of the magnitudes of the user risk coefficients. That is to say, the desensitization loss of the biometric image corresponding to the user with a larger user risk coefficient is smaller, so that the desensitization effect of the biometric image corresponding to the user with a larger user risk coefficient is better.

[0107] In addition, in the exemplary embodiment, the image model further includes an image reverse desensitization sub-model, and the model loss of the image model includes a reverse desensitization loss. The method for training the image model includes: inputting the desensitized images of multiple users into the image reverse desensitization sub-model to obtain the reverse desensitized images corresponding to each user; determining the reverse desensitization loss based on the difference between the reverse desensitized image of the user and the biometric image; and training the image model based on the desensitization loss and the reverse desensitization loss.

[0108] According to the technical solution in the above exemplary embodiment, since multiple biometric images of multiple users are used for joint desensitization in the desensitization stage, the biometric images of each user in the desensitization stage are mutually correlated and dependent. Therefore, in the reverse desensitization stage, it is necessary to find N associated desensitized images corresponding to each desensitized image for corresponding reverse desensitization processing, which increases the attack cost for the attacker (not only needing to obtain a single desensitized sample, but also needing to obtain the corresponding associated desensitized sample), and improves the security of privacy protection for biometric images.

[0109] Furthermore, during de-identification processing, replacing a portion of the de-identified images significantly worsens the de-identification effect, increasing the attacker's cost. Therefore, in the example embodiment, the model loss also includes multi-image dependency loss. Multi-image dependency loss represents the model loss when at least one image from multiple users' de-identified images is replaced or missing. For example, the more images replaced or missing, the greater the multi-image dependency loss. The above-mentioned training of the image model based on the model loss includes: determining the corresponding multi-image dependency loss when at least one image from multiple users' de-identified images is replaced or missing, and training the image model based on the de-identification loss, de-identification loss, and multi-image dependency loss. For example, suppose there are 10 users, and at least one user's de-identified image is replaced or missing. The multi-image dependency loss is determined based on the number of replaced or missing de-identified images, and the image model is trained based on the de-identification loss, de-identification loss, and multi-image dependency loss.

[0110] According to the technical solution in the above example embodiment, by setting a multi-image dependency loss, if a part of the images in multiple images is missing or replaced during the de-identification process, the de-identification effect will be significantly worse, thereby further increasing the attack cost for attackers.

[0111] Furthermore, considering that multiple biometric images often come from multiple users, and the privacy leakage risk of multiple users is inconsistent (generally, the more frequent the biometric recognition and the more open the biometric recognition scenario, the higher the privacy leakage risk), it is necessary to perceive the privacy leakage risk of different users, i.e., the user risk coefficient, to provide prior knowledge for subsequent adaptive multi-image desensitization. Figure 5 This is a schematic diagram of the process for determining a user risk coefficient according to the embodiments provided in this specification.

[0112] Reference Figure 5 As shown, in step S510, the biometric features of each user among multiple users within a predetermined time period are determined.

[0113] In the example embodiment, biometric features include biometric frequency features and scene frequency features for each biometric scenario. For example, biometric features include biometric images of the user in the current time period, such as one week, the number of biometric attempts, and the number of times each biometric scenario is performed. Biometric scenarios include high-risk scenarios, medium-risk scenarios, and low-risk scenarios.

[0114] In step S520, based on biometric features, the user risk coefficient for each user is determined through a risk coefficient determination sub-model.

[0115] In the example embodiment, the user risk coefficient represents the feature weight of the image features of each user's biometric image. The image model also includes a risk coefficient determination sub-model, which includes a feature encoder and a biometric feature prediction module. For example, the input to the feature encoder is the user's biometric images, biometric counts, and the number of times each biometric scenario is performed within the current time period (e.g., one week), and the output of the feature encoder is the biometric feature. The input to the biometric feature prediction model is the user's biometric feature within the current time period, and the output is the user risk coefficient for the next time period.

[0116] Furthermore, the loss function of the biometric feature prediction model is determined. The loss function includes the biometric number prediction loss and the biometric scene number prediction loss. Based on this loss function, the risk coefficient determination sub-model is optimized and trained by the ADAM optimizer until the risk coefficient determination sub-model converges.

[0117] According to the technical solution in the above example embodiment, based on biometric features, the risk coefficient of each user is determined by a risk coefficient determination sub-model. This can predict the privacy leakage risk of a user's biometric image by combining the number of biometric recognitions and the number of biometric recognition scenarios, thereby accurately predicting the privacy leakage risk of a user's biometric image.

[0118] Furthermore, in the example embodiment, the image features of the biometric images corresponding to each user and the user risk coefficient are input into the above-mentioned feature fusion sub-model. Based on the user risk coefficient corresponding to each user, the image features of the biometric images of each user are weighted and fused through the feature fusion sub-model.

[0119] According to the technical solution in the above example embodiments, on the one hand, by performing weighted fusion processing on the image features of biometric images corresponding to each user based on the user risk coefficient, the image features of biometric images with a high user risk coefficient can be enhanced, while the image features of biometric images with a low user risk coefficient can be weakened. This allows for deeper fusion processing of biometric images of users with a high risk of privacy leakage. On the other hand, since deeper fusion processing is possible for biometric images of users with a high risk of privacy leakage, the security of privacy protection for biometric images can be further improved.

[0120] Furthermore, in the example embodiment, the image model also includes a risk coefficient adjustment sub-model. The image model training method further includes: determining the user's first biometric feature in the current period and the second biometric feature in the previous period; inputting the first biometric feature and the second biometric feature into the risk coefficient adjustment sub-model to obtain the feature similarity between the first biometric feature and the second biometric feature; if the feature similarity is less than a predetermined threshold, adjusting the user risk coefficient corresponding to the user based on the biometric feature in the current period.

[0121] According to the technical solution in the above example embodiment, when the feature similarity between the biometric features of the current period and the biometric features of the previous period is less than a predetermined threshold, the user risk coefficient corresponding to the user is adjusted based on the biometric features of the current period. This allows for timely updates to the user risk coefficient when there are significant changes. The updated user risk coefficient is then used to perform weighted fusion processing on the image features of the user's biometric image, thereby further improving the security of privacy protection for biometric images.

[0122] Figure 6 This is a flowchart illustrating another image model training method provided according to an embodiment of this specification.

[0123] To increase the cost of attacks for attackers and thus raise the level of privacy protection, this specification proposes a biometric image privacy protection method based on multi-user joint perception desensitization. The core idea of ​​this method is to use multiple biometric images from multiple users for joint desensitization, making the biometric images of each user interconnected and dependent during the desensitization stage. Similarly, the desensitized images of these users are also needed in the desensitization stage, thereby increasing the cost for attackers (from obtaining a single image sample to obtaining multiple image samples, increasing the difficulty of the attack). Below, in conjunction with... Figure 6 This paper provides a detailed explanation of the method for protecting the privacy of biometric images.

[0124] Reference Figure 6 As shown, in step S610, the user risk coefficient of privacy leakage of the user's biometric image is predicted based on the user's historical biometric behavior.

[0125] In the example embodiment, a method of simultaneously desensitizing multiple biometric images is used to increase the cost for attackers. Considering that multiple biometric images often come from multiple users, and the privacy leakage risks of multiple users are inconsistent (generally, the more frequent the biometric recognition and the more open the scenario in which the biometric recognition takes place, the higher the privacy leakage risk), it is necessary to perceive the privacy leakage risk of different users, i.e., the user risk coefficient, to provide prior knowledge for subsequent adaptive multi-image desensitization.

[0126] Furthermore, the training of the user privacy risk perception model, i.e., the risk coefficient determination sub-model, based on biometric frequency and biometric scenarios, includes the following parts:

[0127] (1) Model structure: The model structure consists of three parts. The first part is the basic feature encoder, the second part is the biometric frequency prediction module, and the third part is the biometric scene statistics module.

[0128] (2) Input and output: The input of the basic feature encoder is the biometric image of the user in the current period, such as the current week, the number of biometrics in the current period, and the number of biometrics in various scenarios (divided into high-risk, medium-risk, and low-risk scenarios). The output is the biometric features, which include biometric frequency features and scene frequency features of each biometric scenario. The input of the biometric frequency prediction module is the biometric frequency features of the current period, and the output is the biometric frequency of the next period, such as the number of biometrics in the next week. The input of the biometric scene statistics module is the scene frequency features of the current period, and the output is the scene frequency features of the next period, such as the number of biometrics in various scenarios in the next week.

[0129] (3) Loss function: The loss function consists of two parts. The first part is the loss for predicting the number of biometric identifications, and the second part is the loss for predicting the number of biometric identifications in various scenarios.

[0130] (4) Training method: Based on the above model structure and loss function, the ADAM optimizer is used to optimize and train the model until the model converges;

[0131] (5) Calculation of user risk coefficient: Input the user’s current period data, such as the current week, into the user privacy risk perception model trained above to obtain the number of biometric identifications c1 in the next period, such as the next week, and the number of biometric identifications c2, c3 and c4 in high-risk, medium-risk and low-risk scenarios; determine the user risk coefficient s through the following formula (1).

[0132] User risk coefficient s=5*c4+2*c3+0.5c2+0.1*c1(1)

[0133] Among them, coefficients 5, 2, 0.5, and 0.1 are the weights of each biometric identification attempt, which can be determined based on empirical values.

[0134] In step S620, a multi-image joint desensitization / anti-desensitization model is trained;

[0135] In the example embodiment, the multi-user, multi-image joint de-identification / de-identification model is trained based on the user risk coefficients generated in the previous step. This model is then used for subsequent deployment. The training of the adaptive multi-user, multi-image joint de-identification / de-identification model includes the following parts:

[0136] (1) Model structure: The model structure includes four modules. The first module is the multi-image feature extraction module, the second part is the feature adaptive fusion module, the third part is the image desensitization module, and the fourth part is the image desensitization model.

[0137] (2) Input and output: The input of the multi-image feature extraction module is multiple biometric images (e.g., N images) from multiple users, and the output is the image features of the corresponding multiple biometric images; the input of the feature adaptive fusion module is the image features of multiple biometric images and the corresponding user risk coefficients, and the output is the modulated multiple fused image features, which contain the correlation or dependency features between the image features of the biometric images of multiple users; the input of the image desensitization module is the modulated multiple fused image features, and the output is multiple corresponding desensitized images; the input of the image anti-desensitization model is multiple desensitized images, and the output is the corresponding anti-desensitized image.

[0138] (3) Loss Function: The loss function consists of four parts. The first part is the desensitization loss, which is used to make the desensitized image as inconsistent as possible with the original image. The second part is the reconstruction loss, which is the anti-desensitization loss, used to make the anti-desensitized image as consistent as possible with the original image. The third part is the desensitization loss ranking loss, which is used to make the positive ranking of the desensitization loss and the negative ranking of the user risk coefficient as consistent as possible. In other words, it makes the desensitization loss of the biometric image corresponding to the user with a higher user risk coefficient smaller, thus making the desensitization effect of the biometric image corresponding to the user with a higher user risk coefficient better. The fourth part is the multi-image dependency loss, which is used to make the anti-desensitization effect significantly worse when replacing one part of multiple desensitized images, thus increasing the attack cost for attackers.

[0139] (4) Training method: Based on the above model structure and loss function, the ADAM optimizer is used to optimize and train the model until the model converges.

[0140] In step S630, the user risk coefficient is periodically adjusted.

[0141] In the example embodiment, since users' biometric behavior changes over time, the risk of leakage must be reassessed periodically; when the user risk coefficient changes significantly, the user risk coefficient needs to be updated in a timely manner.

[0142] Furthermore, the user behavior change detection model, i.e., the risk coefficient adjustment sub-model, determines whether a risk coefficient update is needed. The training of the user behavior change detection model includes the following parts:

[0143] (1) Model structure: The model structure is a 3-layer MLP;

[0144] (2) Input and output: The model input is the biometric features of the current period and the previous period, such as the number of biometric identifications, location and scene level of the user used to calculate the user risk coefficient, and the output is the similarity between the biometric features of the current period and the biometric features of the previous period.

[0145] (3) Loss function: The loss function is the similarity loss;

[0146] (4) Training method: Based on the above model structure and loss function, the ADAM optimizer is used to optimize and train the model until the model converges;

[0147] (5) Daily user behavior change detection is performed for the above multiple users. For users whose behavior changes are obvious, such as users whose similarity is less than the predetermined threshold, the risk coefficient determination sub-model is used to calculate and update the risk coefficient.

[0148] In step S640, the model and method trained in the above steps are used for model deployment and application.

[0149] In the example embodiment, the trained model is deployed to a cloud server; during biometric image desensitization, N biometric images (from N users) are randomly selected, and the biometric images and the corresponding user risk coefficients are input into the image desensitization model to obtain desensitized images, which are then stored.

[0150] During the de-identification process, it is necessary to find N de-identified images corresponding to each de-identified image and perform the corresponding de-identification, which increases the attack cost for attackers (not only do they need to obtain a single de-identified sample, but they also need to obtain the corresponding associated de-identified samples).

[0151] according to Figure 6 The technical solution in the example embodiment uses multiple biometric images from multiple users for joint desensitization, making the biometric images of each user interconnected and dependent during the desensitization stage. Therefore, the desensitized images of these users are also needed for desensitization during the desensitization stage, thereby increasing the attack cost for attackers (from obtaining a single image sample to obtaining multiple image samples, increasing the difficulty of sample acquisition) and improving the security of privacy protection for biometric images.

[0152] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0153] Below, we will combine Figure 7 as well as Figure 1 The system architecture shown in this specification provides a detailed description of the image processing apparatus provided in the embodiments. It should be noted that... Figure 7 The image processing apparatus described herein is used to perform the functions described herein. Figures 2-6 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 2-6 The example shown.

[0154] Please see Figure 7 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this specification. Figure 7 As shown, the image processing apparatus 700 in this embodiment may include: a feature extraction module 710, a feature fusion module 720, and a desensitization processing module 730. Wherein:

[0155] Feature extraction module 710 is used to acquire biometric images of each user among multiple users and extract image features from the biometric images;

[0156] The feature fusion module 720 is used to perform feature fusion processing on the image features of the biometric images of each user to generate fused image features corresponding to the biometric images of each user. The fused image features include the correlation features between the image features of the biometric images of the multiple users.

[0157] The desensitization processing module 730 is used to generate desensitized images corresponding to the biometric images of each user based on the fused image features corresponding to the biometric images of each user.

[0158] In some example embodiments, based on the above scheme, the feature fusion module 720 includes:

[0159] A weighted processing unit is used to perform weighted fusion processing on the image features of the biometric images of each of the multiple users;

[0160] The fusion feature generation unit is used to generate fusion image features corresponding to the biometric images of each user based on the results of weighted fusion processing.

[0161] In some example embodiments, based on the above scheme, the weighting processing unit includes:

[0162] A risk coefficient determination unit is used to determine the user risk coefficient for each user, wherein the user risk coefficient represents the feature weight of the image features of the biometric image of each user.

[0163] The weighted fusion unit is used to perform weighted fusion processing on the image features of the biometric images corresponding to each user based on the user risk coefficient corresponding to each user.

[0164] In some example embodiments, based on the above scheme, the risk coefficient determination unit includes:

[0165] A biometric feature determination unit is used to determine the biometric features of each user among the plurality of users within a predetermined time period. The biometric features include biometric frequency features and scene frequency features of each biometric scenario.

[0166] The risk coefficient determination unit is used to determine the user risk coefficient for each user based on the biometric features.

[0167] In some example embodiments, the risk coefficient determination unit is configured as follows:

[0168] A weighted calculation is performed on the biometric frequency features of each user among the plurality of users and the scene frequency features of each biometric scene;

[0169] The user risk coefficient for each user is determined based on the weighted calculation results.

[0170] In some example embodiments, based on the above scheme, the risk coefficient determination unit is further configured as follows:

[0171] Periodically acquire the biometric features of each of the multiple users;

[0172] Determine the feature similarity between the user's biometric features in the current period and the biometric features in the previous period;

[0173] If the feature similarity is less than a predetermined threshold, the user risk coefficient corresponding to the user is adjusted based on the biometric features of the current period.

[0174] In some example embodiments, based on the above scheme, the desensitization processing module 730 is further configured as follows:

[0175] The fused image features corresponding to the biometric images of each user are input into the image desensitization model;

[0176] The image desensitization model generates desensitized images corresponding to the biometric images of each user.

[0177] In some example embodiments, the device 700 further includes:

[0178] The associated image acquisition module is used to acquire multiple associated desensitized images corresponding to the target desensitized image of the target user;

[0179] The anti-desensitization processing module is used to perform anti-desensitization processing on the target desensitized image based on the multiple associated desensitized images using an anti-desensitization model.

[0180] The above is an illustrative embodiment of an image processing apparatus according to this specification. It should be noted that the technical solution of this image processing apparatus and the technical solution of the image processing method described above belong to the same concept. Details not described in detail in the technical solution of the image processing apparatus can be found in the description of the technical solution of the image processing method described above.

[0181] Figure 8 This is a schematic diagram of the image model training device provided according to an embodiment of this specification. Figure 8 As shown, the image model includes an image desensitization sub-model. The image model training device 800 in this embodiment may include: a feature extraction module 810, a feature fusion module 820, an image desensitization module 830, a loss determination module 840, and a model training module 850. Wherein:

[0182] The feature extraction module 810 is used to acquire the biometric images of each user among multiple users and extract the image features of each user's biometric images;

[0183] The feature fusion module 820 is used to perform feature fusion processing on the image features of each user's biometric images to generate fused image features corresponding to each user's biometric images. The fused image features include correlation features between the image features of the biometric images of the multiple users.

[0184] The image desensitization module 830 is used to input the fused image features corresponding to each user into the image desensitization sub-model to obtain the desensitized image corresponding to the biometric image of each user.

[0185] The loss determination module 840 is used to determine the model loss of the image model based on the desensitized image and the biometric image, wherein the model loss includes the desensitization loss;

[0186] The model training module 850 is used to train the image model based on the model loss.

[0187] In some example embodiments, based on the above scheme, the image model further includes a feature fusion sub-model, the feature fusion module including:

[0188] The feature input unit is used to input the image features of each user's biometric image into the feature fusion sub-model;

[0189] The weighted processing unit is used to perform weighted fusion processing on the image features of the biometric images of each user through the feature fusion sub-model.

[0190] In some example embodiments, based on the above scheme, the feature input unit includes:

[0191] A risk coefficient determination unit is used to determine the user risk coefficient for each user, wherein the user risk coefficient represents the feature weight of the image features of the biometric image of each user.

[0192] The fusion input unit is used to input the image features of the biometric images corresponding to each user and the user risk coefficient into the feature fusion sub-model.

[0193] The weighting processing unit includes:

[0194] The weighted fusion unit is used to perform weighted fusion processing on the image features of the biometric images of each user based on the user risk coefficient corresponding to each user, through the feature fusion sub-model.

[0195] In some example embodiments, based on the above scheme, the image model further includes a risk coefficient determination sub-model, the risk coefficient determination unit comprising:

[0196] A biometric feature determination unit is configured to determine the biometric features of each user among the plurality of users within a predetermined time period, the biometric features including biometric frequency features and scene frequency features of each biometric scenario.

[0197] The coefficient determination unit is configured to determine the user risk coefficient for each user based on the biometric features and through the risk coefficient determination sub-model.

[0198] In some example embodiments, based on the above scheme, the image model further includes a risk coefficient adjustment sub-model, and the device 800 further includes:

[0199] The biometric determination module is configured to determine the user's first biometric feature for the current period and the second biometric feature for the previous period;

[0200] The feature similarity determination module is configured to input the first biometric feature and the second biometric feature into the risk coefficient adjustment sub-model to obtain the feature similarity between the first biometric feature and the second biometric feature;

[0201] The coefficient adjustment module is configured to adjust the user risk coefficient corresponding to the user based on the biometric features of the current period if the feature similarity is less than a predetermined threshold.

[0202] In some example embodiments, based on the above scheme, the model loss further includes a desensitization loss ranking loss, which is used to ensure that the forward ranking of the desensitization losses is consistent with the reverse ranking of the user risk coefficients. The device 800 includes:

[0203] The coefficient ranking determination module is configured to determine the risk coefficient ranking of the user risk coefficients of the multiple users based on the magnitude of the user risk coefficients.

[0204] The loss ranking determination module is configured to determine the desensitization loss ranking based on the magnitude of the desensitization loss;

[0205] The sorting loss determination module is configured to determine the desensitization loss sorting loss based on the desensitization loss sorting and the risk coefficient sorting.

[0206] The model training module 850 is configured as follows:

[0207] The image model is trained based on the desensitization loss and the desensitization loss ranking loss.

[0208] In some example embodiments, based on the above scheme, the image model further includes an image desensitization sub-model, the model loss of the image model includes desensitization loss, and the apparatus further includes:

[0209] The desensitization module is configured to input the desensitized images of the multiple users into the image desensitization sub-model to obtain the desensitized images corresponding to each user.

[0210] The desensitization loss determination module is configured to determine the desensitization loss based on the difference between the user's desensitized image and the biometric image;

[0211] The model training module 850 is also configured to:

[0212] The image model is trained based on the desensitization loss and the anti-desensitization loss.

[0213] In some example embodiments, based on the above scheme, the model loss further includes a multi-image dependency loss, which represents the model loss generated when at least one image in the de-identified images of the multiple users is replaced or missing. The model training module 850 is also configured to:

[0214] In the case of replacing or missing at least one image from the de-identified images of the multiple users, determine the corresponding multi-image dependency loss;

[0215] The image model is trained based on the desensitization loss, the anti-desensitization loss, and the multi-graph dependency loss.

[0216] The above is an illustrative scheme of an image model training device according to an embodiment of this specification. It should be noted that the technical solution of this image model training device and the technical solution of the image model training method described above belong to the same concept. Details not described in detail in the technical solution of the image model training device can be found in the description of the technical solution of the image model training method described above.

[0217] This specification also provides a computer storage medium that can store multiple program instructions adapted to be loaded and executed by a processor as described above. Figures 2-6 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figures 2-6 The specific details of the illustrated embodiments will not be elaborated here.

[0218] This specification also provides a computer program product that stores at least one instruction, which is loaded and executed by a processor as described above. Figures 2-6 The image processing method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 2-6 The specific details of the illustrated embodiments will not be elaborated here.

[0219] Please refer to Figure 9 This diagram illustrates the structure of an electronic device provided in an exemplary embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 910, a memory 920, an input device 930, an output device 940, and a bus 950. The processor 910, memory 920, input device 930, and output device 940 may be connected via the bus 950.

[0220] Processor 910 may include one or more processing cores. Processor 910 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data of electronic device 900 by running or executing instructions, programs, code sets, or instruction sets stored in memory 920, and by calling data stored in memory 920. Optionally, processor 910 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array. Processor 910 may integrate one or more of the following: central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU mainly handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem is used for wireless communication. It is understood that the modem may also not be integrated into processor 910 and may be implemented separately through a communication chip.

[0221] The memory 920 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 920 may include a non-transitory computer-readable storage medium. The memory 920 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be an Android system, including systems deeply developed based on the Android system, an iOS system, including systems deeply developed based on the iOS system, or other systems.

[0222] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0223] The input device 930 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 940 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 930 and the output device 940 are a touch display screen.

[0224] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, Wireless Fidelity (WiFi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0225] exist Figure 9 In the illustrated electronic device, the processor 910 can be used to invoke image processing applications or image model training applications stored in the memory 920. For example, the processor 910 can perform the following operations:

[0226] Acquire biometric images of each user from multiple users, and extract image features from the biometric images;

[0227] The image features of the biometric images of each user are subjected to feature fusion processing to generate fused image features corresponding to the biometric images of each user. The fused image features include the correlation features between the image features of the biometric images of the multiple users.

[0228] Based on the fused image features corresponding to the biometric images of each user, a desensitized image corresponding to the biometric image of each user is generated.

[0229] The above is an illustrative embodiment of an electronic device according to this specification. It should be noted that the technical solution of this electronic device belongs to the same concept as the technical solution of the image processing method or image model training method described above. Details not described in detail in the technical solution of the electronic device can be found in the description of the technical solution of the image processing method or image model training method described above.

[0230] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium for the computer program can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0231] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

[0232] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

Claims

1. An image processing method, comprising: Acquire biometric images of each user from multiple users, and extract image features from the biometric images; The image features of the biometric images of each user are fused to generate fused image features corresponding to the biometric images of each user. The fused image features include the correlation features between the image features of the biometric images of the multiple users. The fused image features of each user are obtained by fusing based on the user risk coefficients corresponding to each user. Based on the fused image features corresponding to the biometric images of each user, a desensitized image corresponding to the biometric image of each user is generated.

2. The method according to claim 1, wherein, The step of performing feature fusion processing on the image features of the biometric images of each user to generate fused image features corresponding to the biometric images of each user includes: The image features of the biometric images of each of the multiple users are weighted and fused. Based on the results of weighted fusion processing, fused image features corresponding to the biometric images of each user are generated.

3. The method according to claim 2, wherein, The weighted fusion processing of the image features of the biometric images of each user includes: Determine the user risk coefficient for each user, where the user risk coefficient represents the feature weight of the image features of the biometric image of each user. Based on the user risk coefficient corresponding to each user, the image features of the biometric images corresponding to each user are subjected to weighted fusion processing.

4. The method according to claim 3, wherein, Determining the user risk coefficient for each user includes: Determine the biometric characteristics of each user among the plurality of users within a predetermined time period, wherein the biometric characteristics include biometric frequency characteristics and scene frequency characteristics of each biometric scenario; Based on the biometric features, the user risk coefficient corresponding to each user is determined.

5. The method according to claim 4, wherein, The process of determining the user risk coefficient for each user based on the biometric features includes: A weighted calculation is performed on the biometric frequency features of each user among the plurality of users and the scene frequency features of each biometric scene; The user risk coefficient for each user is determined based on the weighted calculation results.

6. The method according to claim 3, wherein, Determining the user risk coefficient for each user includes: Periodically acquire the biometric features of each of the multiple users; Determine the feature similarity between the user's biometric features in the current period and the biometric features in the previous period; If the feature similarity is less than a predetermined threshold, the user risk coefficient corresponding to the user is adjusted based on the biometric features of the current period.

7. The method according to any one of claims 1 to 6, wherein, The step of generating desensitized images corresponding to the biometric images of each user based on the fused image features corresponding to the biometric images of each user includes: The fused image features corresponding to the biometric images of each user are input into the image desensitization model; The image desensitization model generates desensitized images corresponding to the biometric images of each user.

8. The method according to claim 7, wherein, The method further includes: Obtain multiple associated de-identified images corresponding to the target de-identified image of the target user; Based on the multiple associated desensitized images, the target desensitized image is subjected to desensitization processing using a desensitization model.

9. An image model training method, wherein, The image model includes an image desensitization sub-model, and the method includes: Acquire biometric images of each user from multiple users, and extract image features from each user's biometric images; The image features of each user's biometric image are fused to generate fused image features corresponding to each user's biometric image. The fused image features include the correlation features between the image features of the biometric images of the multiple users. The fused image features of each user are obtained by fusing based on the user risk coefficient corresponding to each user. The fused image features corresponding to each user are input into the image desensitization sub-model to obtain the desensitized image corresponding to the biometric image of each user. Based on the desensitized image and the biometric image, the model loss of the image model is determined, and the model loss includes the desensitization loss; The image model is trained based on the model loss.

10. The method according to claim 9, wherein, The image model further includes a feature fusion sub-model, wherein the feature fusion processing of the image features of each user's biometric image includes: The image features of each user's biometric image are input into the feature fusion sub-model; The feature fusion sub-model performs weighted fusion processing on the image features of the biometric images of each user.

11. The method according to claim 10, wherein, The step of inputting the image features of each user's biometric image into the feature fusion sub-model includes: Determine the user risk coefficient for each user, where the user risk coefficient represents the feature weight of the image features of the biometric image of each user. The image features of the biometric images corresponding to each user and the user risk coefficient are input into the feature fusion sub-model. The weighted fusion processing of image features of the biometric images of each user through the feature fusion sub-model includes: Based on the user risk coefficient corresponding to each user, the image features of the biometric images of each user are weighted and fused through the feature fusion sub-model.

12. The method according to claim 11, wherein, The image model also includes a risk coefficient determination sub-model, wherein determining the user risk coefficient for each user includes: Determine the biometric characteristics of each user among the plurality of users within a predetermined time period, wherein the biometric characteristics include biometric frequency characteristics and scene frequency characteristics of each biometric scenario; Based on the biometric features, the risk coefficient for each user is determined through the risk coefficient determination sub-model.

13. The method according to claim 12, wherein, The image model further includes a risk coefficient adjustment sub-model, and the method further includes: Determine the user's first biometric feature for the current period and the second biometric feature for the previous period; The first biometric feature and the second biometric feature are input into the risk coefficient adjustment sub-model to obtain the feature similarity between the first biometric feature and the second biometric feature; If the feature similarity is less than a predetermined threshold, the user risk coefficient corresponding to the user is adjusted based on the biometric features of the current period.

14. The method according to claim 11, wherein, The model loss also includes a desensitization loss ranking loss, which is used to ensure that the forward order of the desensitization losses is consistent with the reverse order of the user risk coefficients. The method further includes: The risk coefficient ranking of the multiple users is determined based on the magnitude of the user risk coefficient. The desensitization loss ranking is determined based on the magnitude of the desensitization loss; The desensitization loss ranking loss is determined based on the desensitization loss ranking and the risk coefficient ranking. Training the image model based on the model loss includes: The image model is trained based on the desensitization loss and the desensitization loss ranking loss.

15. The method according to claim 9, wherein, The image model further includes an image desensitization sub-model, and the model loss of the image model includes a desensitization loss. The method further includes: The de-identified images of the multiple users are input into the image de-identification sub-model to obtain the de-identified image corresponding to each user. The desensitization loss is determined based on the difference between the user's desensitized image and the biometric image; Training the image model based on the model loss includes: The image model is trained based on the desensitization loss and the anti-desensitization loss.

16. The method according to claim 15, wherein, The model loss also includes a multi-image dependency loss, which represents the model loss in the event of replacing or missing at least one image from the de-identified images of the multiple users. Training the image model based on the model loss includes: In the case of replacing or missing at least one image from the de-identified images of the multiple users, determine the corresponding multi-image dependency loss; The image model is trained based on the desensitization loss, the anti-desensitization loss, and the multi-graph dependency loss.

17. An image processing apparatus, comprising: The feature extraction module is used to acquire biometric images of each user among multiple users and extract image features from the biometric images; The feature fusion module is used to perform feature fusion processing on the image features of the biometric images of each user to generate fused image features corresponding to the biometric images of each user. The fused image features include the correlation features between the image features of the biometric images of the multiple users. The fused image features of each user are obtained by fusing based on the user risk coefficients corresponding to each user. The desensitization processing module is used to generate desensitized images corresponding to the biometric images of each user based on the fused image features corresponding to the biometric images of each user.

18. An image model training device, wherein, The image model includes an image desensitization sub-model, and the device includes: The feature extraction module is used to acquire biometric images of each user among multiple users and extract the image features of each user's biometric images; The feature fusion module is used to perform feature fusion processing on the image features of the biometric images of each user to generate fused image features corresponding to the biometric images of each user. The fused image features include the correlation features between the image features of the biometric images of the multiple users. The fused image features of each user are obtained by fusing based on the user risk coefficients corresponding to each user. The image desensitization module is used to input the fused image features corresponding to each user into the image desensitization sub-model to obtain the desensitized image corresponding to the biometric image of each user. The loss determination module is used to determine the model loss of the image model based on the desensitized image and the biometric image, wherein the model loss includes the desensitization loss; The model training module is used to train the image model based on the model loss.

19. A computer storage medium storing a plurality of instructions adapted for loading by a processor and performing the steps of the method as claimed in any one of claims 1 to 16.

20. An electronic device, comprising: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as claimed in any one of claims 1 to 16.

21. A computer program product comprising instructions that, when run on a computer or processor, causes the computer or processor to perform the steps of the method as claimed in any one of claims 1-16.

Images