Social trust-driven face information protection method

Abstract

The invention relates to a social trust-driven face information protection method, which belongs to the technical field of information security, and comprises the following steps: S1, a system initializes three entities, including a data publisher, a trusted third party and a data visitor; s2, the trusted third party calculates a real-time social trust value between the data publisher and the data visitor by using a graph attention mechanism; s3, a privacy level is given to the data visitor according to the social trust value of the data visitor, a differential privacy mechanism is used for generating a protection identity vector, and the privacy level is used for controlling a privacy budget; and S4, inputting the protection identity vector into the conditional identity anonymous generative adversarial network to obtain a face information protection image, and sending the face information protection image to a corresponding data visitor.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology and relates to a facial information protection method driven by social trust. Background Art

[0002] With the rapid development of Internet technology, more and more multimedia data (images and videos) are transmitted through the Internet and shared on social platforms. However, when sharing image data, social network users often ignore the facial information contained in the massive data, which leads to security risks. Nowadays, facial biometrics technology based on deep learning algorithms can easily identify and steal facial feature information in pictures, transform them into 3D facial masks, and conduct deception attacks on facial recognition systems in life to obtain unauthorized access or pass identity authentication. Today, soft biometrics such as face have become a new form of password, which seriously threatens the personal information security of users.

[0003] At present, relevant research on the security of face information on social networks has made certain progress. Existing studies often adopt image editing, image encryption, and face de-identification technology based on deep learning to achieve face information protection. More and more researchers hope to ensure the security of face information under the premise that the de-identified image is real and natural. For example, Li et al. proposed a SF-GAN method based on generative adversarial network in "SF-GAN: face de-identification method without losing facial attribute information" [IEEE Signal Processing Letters, 2021, 28: 1345-1349.], which uses shallow face attribute information and deep face attribute information to process non-important information such as facial expressions and backgrounds and important information related to identity attributes differently. On the basis of generating high-fidelity de-identified images, the protection ability of face information is further improved. The face de-identification effect achieved by the generative adversarial network is encouraging, but simply replacing or modifying all the face information in the image may reduce the utility of the image, thereby affecting the user's willingness to share the image. Users often hope to reduce information security risks while retaining the original image effect as much as possible, so that trusted users can see more restored images. At the same time, the image sharing scenario of online social networks is relatively complex. The social attributes, social relationships, and social trust between users are in a state of constant change, which makes it cumbersome for users to manually set privacy policies each time. Some users may not be willing to invest extra time and energy to screen who can view the images they share. In order to reduce the time cost of manually setting privacy policies and improve the efficiency of user image sharing, many works are currently studying the automatic setting of privacy policies. For example, Jiao et al. proposed an IEye model for automatic detection of image privacy based on common sense and personal knowledge in "Ieye: Personalized image privacy detection" [2020 6th International Conference on Big Data Computing and Communications (BIGCOM). IEEE, 2020: 91-95]. Specifically, for each user's image, a multi-layer semantic graph is constructed as the feature representation of its image to describe its personalized privacy policy and minimize the image utility loss, but there are certain limitations in efficiency.

[0004] Although existing face information protection methods have been widely studied and applied, the following major challenges still exist:

[0005] (1) Adaptive privacy protection strategy: It takes a lot of time for users to manually set privacy protection strategies, and the existing adaptive privacy strategies cannot take into account the frequently changing factors in social networks and are not suitable for dynamic social network scenarios.

[0006] (2) Fine-grained face information protection: Most existing face information protection methods replace or modify all faces in the original image to achieve the purpose of face information protection. They do not take into account the user's trust-based information sharing needs, resulting in a certain degree of image utility loss and the inability to achieve fine-grained image sharing. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a social trust-driven facial information protection method, combining social trust evaluation and generative adversarial network technology, to provide a fine-grained facial information protection solution based on the user's social trust value, and solve the problems faced by the prior art. With the help of the powerful computing power of a trusted third party, dynamic data sharing between users is achieved, while balancing the information security and image utility of shared images. Specifically, the method achieves fine-grained image access sharing by setting different privacy levels for visitors, thereby reducing the threat caused by facial information leakage. First, a social network trust graph is constructed, and a graph attention mechanism is introduced to calculate the real-time social trust between users as the basis for dividing the privacy level of visitors. In addition, in order to take into account the security protection of user facial information and the high fidelity of the image, the present invention designs a face protection algorithm based on a generative adversarial model, which adjusts the facial features of the face part of the original image to different degrees according to the privacy level of the visitor to achieve different degrees of facial information protection.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A social trust-driven face information protection method comprises the following steps:

[0010] S1: The system initializes three entities, including the data publisher, the trusted third party, and the data accessor;

[0011] S2: The trusted third party uses the graph attention mechanism to calculate the real-time social trust value between the data publisher and the data accessor;

[0012] S3: Assign a privacy level to the data accessor based on his / her social trust value, and use the differential privacy mechanism to generate a protected identity vector, where the privacy level is used to control the size of the privacy budget;

[0013] S4: Input the protected identity vector into the conditional identity anonymous generative adversarial network to obtain the face information protection image, and send it to the corresponding data accessor.

[0014] Further, the system initialization in step S1 includes:

[0015] Initialize three entities, including data publisher, trusted third party, and data accessor;

[0016] The data publisher is a user who generates images and shares them on social networks;

[0017] The trusted third party is the transfer party of social network image upload, which builds a social trust graph based on the basic attributes of data publishers and data accessors, and is responsible for the social trust evaluation of users and image processing;

[0018] The data accessor is used to send a data request to a trusted third party to obtain the data published by the data publisher;

[0019] Initialize the user set of online social network in the social trust graph Set the set of all user social interactions in the period [0, T] to be in, represents the set of neighboring nodes that have interacted with node i within time [0, T]; social trust is modeled as a series of interaction events with timestamps in the social trust graph G = {e(t1), e(t2), ...}, where e(t m ) indicates t m The social interaction between any pair of user nodes at time t; user nodes i and j at time t m The social trust with a value of c is represented by a directed edge

[0020] Further, step S2 specifically includes the following steps:

[0021] S21: The trusted third party generates a social trust graph G(T) at a given time T, where the nodes in the graph are social network users and the edges are the social trust values ​​c connecting pairs of users;

[0022] S22: Given a pair of social user nodes i, j, first extract the user's basic attributes s i , user i and neighboring nodes k∈N i (T) Historical Interaction and the associated timestamp Ψ t ,according to Generate joint embedding vector n ik , to quantify the message signal from user i to neighboring user k;

[0023] S23: Weighted calculation and aggregation of neighboring users k∈N i The joint embedding vector n of (T) ikIn this process, the graph attention mechanism is executed to assign higher weights to the latest interactions. The weight assigned by the user to each neighboring user k is expressed as a ik , and finally obtain the first-order feature representation of user i

[0024] S25: By stacking L trust propagation layers, aggregate the influence of multi-hop user nodes on user i to obtain high-order feature representation

[0025] S26: Calculate the high-order feature representation of the user node pair i, j in step S25 and It is passed into the MLP and processed through a softmax layer to output the predicted probability h of social trust between any user pair i and j. ij ; Finally, through c = argmax (h ij ) Calculate the social trust c of any pair of users.

[0026] Further, in step S22, the mapping function is used Get timestamp t ; The historical interaction trust value between user i and neighboring nodes Convert to dense vector embedding; then embed the user's attributes into vector s i , timestamp embedding vector Ψ t and the interaction embedding vector according to Generate a joint embedding vector as input to the trust propagation layer model.

[0027] Further, in step S23, the graph attention mechanism is used to obtain the interaction weight a assigned by user i to user k. ik ←attn(n ik W ik ,p i W i )=n ik W ik ·(p i W i ) T ,in, and is a trainable matrix, d s ,d t ,d c is the embedding vector dimension, and then use Get the normalized weight; calculate and aggregate the joint embedding vector n of adjacent users by weighted calculation ik To obtain the first-order feature representation of user i Among them, W, b are the parameters that need to be trained, k∈N i(T) is the set of neighboring nodes of user i.

[0028] Further, in step S26, the high-order feature representation of user i and j is and Passed to MLP, output social trust prediction probability h ij :

[0029]

[0030] Among them, W f , b f is the trainable weight matrix defined in the multilayer perceptron;

[0031] The objective function of the entire model is defined as the cross entropy loss between the predicted value and the true value of the confidence in the graph G:

[0032]

[0033] in Represents all trainable model parameters.

[0034] Further, step S3 specifically includes the following steps:

[0035] S31: assigning a trust level d to the accessing user according to the social trust value c calculated in step S2;

[0036] S32: Extract features from the original image and integrate to obtain the identity vector I id ;

[0037] S33: Use differential privacy mechanism based on visitor’s trust level d Get the protected identity vector I i ' d ,in, Indicates that the mean is 0 and the scale is Laplace distributed random noise, Δ is a parameter used to control the strength of privacy protection in differential privacy, which is related to the sensitivity of the data; d is the visitor’s trust level, which determines I id The smaller the d value, the greater the added noise. i ' d with I id The larger the gap, the smaller the noise. i ' d with I id The smaller the gap.

[0038] Further, step S4 specifically includes the following steps:

[0039] The perturbed identity vector obtained in step S33, the non-face region image of the original image, and the target identity image are used as inputs of the generator in the generative adversarial network;

[0040] The generative adversarial network uses an encoder-decoder architecture, where the encoder represents the input combination in step S41 as a low-dimensional vector, and the decoder then represents the vector combination, and generates a face information protection image by upsampling;

[0041] The loss function is set as:

[0042] L total =L G +L D +L Rec

[0043] Where L G , L D is the loss function of the generator and the discriminator, L Rec is the image reconstruction loss;

[0044]

[0045] Where a represents the label of generated data, b represents the label of real data; D(x) is the output of discriminator D for real data x; (D(x)-b) 2 represents the square error between the discriminator's output of the real data and the real label; x~p data (x) represents the distribution of real data p data The data sampled from the generator G; D(G(z)) represents the output of the discriminator D on the data G(z) generated by the generator G; (D(G(z))-a) 2 The squared error between the discriminator's output of the generated data and the generated label; represents the squared error between the output of the discriminator and the generated label from the prior distribution z~p z The noise z sampled in (z);

[0046]

[0047] L Rec =||x raw -x adv ||.

[0048] The beneficial effect of the present invention is that the present invention relies on the trust relationship between social network users to effectively deal with potential facial information security risks in social network image sharing, while ensuring the effectiveness of shared data.

[0049] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0051] Figure 1 This is a system architecture diagram of the present invention;

[0052] Figure 2 The present invention is described in the working flow chart. DETAILED DESCRIPTION

[0053] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0054] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0055] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0056] The present invention provides Figure 1-2, a social trust-driven face information protection method, which combines real-time social trust evaluation and generative adversarial network technology, aims to address the challenges faced by existing technologies and provide a flexible solution to the face information security issues that may exist in image sharing in social networks. To achieve the above purpose, the present invention provides the following technical solutions:

[0057] S1 System Initialization: The system initializes three entities, namely data publishers, trusted third parties, and data accessors. Data publishers are users who generate images and share them on social networks. Before sharing data, they upload images to trusted third parties for processing. Trusted third parties extract social interaction information between data publishers and accessors to construct a social network graph G. In graph G, the user set of the online social network is initialized. Set the set of all user social interactions in the period [0, T] to be in, represents the set of neighboring nodes that have interacted with node i within time [0, T]. Furthermore, social trust is modeled as a series of interaction events with timestamps in the social graph G = {e(t1), e(t2), ...}, where e(t m ) indicates t m The social interaction between any pair of user nodes at time t; user nodes i and j at time t m The social trust with a value of c is represented by a directed edge

[0058] S2 Dynamic Trust Evaluation of Social Network Users: This part mainly consists of three modules, namely, embedding layer, trust propagation layer, and trust prediction layer. In the embedding layer, firstly, the basic information and social attributes of user i are embedded into the vector s using attributes. i Then, use the mapping function Get the timestamp embedding vector Ψ t Finally, the historical interaction trust value between user i and its neighboring nodes is Convert to dense vector embedding. Further, embed the user's attributes into vector s i , timestamp embedding vector Ψ t and the interaction embedding vector according to Generate a joint embedding vector as input to the trust propagation layer model.

[0059] In the single-layer trust propagation layer, the graph attention mechanism is used to obtain the interaction weight a assigned by user i to user k ik ←attn(n ik W ik ,p i W i )=n ik Wik ·(p i W i ) T .in, and is a trainable matrix, d s ,d t ,d c is the embedding vector dimension. Then use Get the normalized weight By weighted calculation and aggregation of the joint embedding vector n of adjacent users ik To obtain the first-order feature representation of user i Among them, W, b are the parameters that need to be trained, k∈N i (T) is the set of neighboring nodes of user i.

[0060] By stacking L trust propagation layers, the influence of multi-hop user nodes on user i is aggregated to obtain high-order feature representation

[0061]

[0062] in, is a trainable parameter. The high-order features of user i and j are represented by and Passed to MLP, output social trust prediction probability h ij :

[0063]

[0064] Among them, W f , b f is the trainable weight matrix defined in the multilayer perceptron. Finally, the social trust value of user i and j is calculated as c = argmax i (h ij ).

[0065] The objective function of the entire model is defined as the cross entropy loss between the predicted value and the true value of the confidence in the graph G:

[0066]

[0067] in Represents all trainable model parameters.

[0068] S3: Trust value driven facial feature point perturbation:

[0069] The trusted third party uses the calculated social trust c in step S3 to determine the visitor's trust level d through a simple linear mapping function f(c)→d. This linear mapping function f() can be a function that maps the trust value to a predetermined level range, for example, mapping the trust value to between 0 and 1, and then dividing different trust levels according to this range.

[0070] Next, the deep learning model is used to extract features from the original image and obtain the identity vector I of the face information. id Then, according to the visitor's trust level d, the differential privacy mechanism is used to calculate the protected identity vector I i ' d ,Right now in, Indicates that the mean is 0 and the scale is Here Δ is a parameter used to control the strength of privacy protection in differential privacy, which is related to the sensitivity of the data; and d is the visitor’s trust level, which determines I id The smaller the d value, the greater the added noise. i ' d with I id The larger the gap, the smaller the noise. i ' d with I id The smaller the gap.

[0071] S4: Face de-identification technology based on conditional identity anonymous generative adversarial network:

[0072] S41: Generator input: The perturbed identity vector obtained in step S33, the non-face region image of the original image, and the target identity image are used as generator input;

[0073] (1) Perturbed identity vector: Use the perturbed identity vector obtained by S33 as the generator input;

[0074] (2) Non-facial region image: The goal of this method is to protect information only in the facial region, without changing the non-facial region of the original image. Therefore, this method extracts the non-facial region from the original image as the input of the generator to ensure that the non-facial region of the generated face information protection image is as close to the original image as possible.

[0075] (3) Target identity: To prevent the generator from overfitting, resulting in the generated image and the original image having the same identity information, this method introduces a set of target images with different identities to guide the image generation of the generator. The target identity information is represented as a one-hot vector through MLP, which is used as the input of the transposed convolutional neural network and input into the generator.

[0076] S42: Generative Adversarial Network Module: The generative adversarial network uses an encoder-decoder architecture. The encoder represents the input combination in step S41 as a low-dimensional vector, and the decoder then represents the vector combination and generates a de-identified image through upsampling. The target loss of the discriminator is defined as follows:

[0077]

[0078] Where a represents the label of generated data, b represents the label of real data; D(x) is the output of discriminator D for real data x; (D(x)-b) 2 represents the square error between the discriminator's output of the real data and the real label; x~p data (x) represents the distribution of real data p data The data sampled from the generator G; D(G(z)) represents the output of the discriminator D on the data G(z) generated by the generator G; (D(G(z))-a) 2 The squared error between the discriminator's output of the generated data and the generated label; represents the squared error between the output of the discriminator and the generated label from the prior distribution z~p z The noise z sampled in (z). The loss of the generator is defined as follows:

[0079]

[0080] In order to ensure the similarity between the generated image and the original image, the mean absolute error between the generated image and the original image is used to construct the reconstruction loss L Rec , used to preserve more details of the original image:

[0081] L Rec =‖x raw -x adv ‖

[0082] The model loss function is set to the least squares loss and the reconstruction loss L Rec The total loss function can be expressed as:

[0083] L total =L G +L D +L Rec

[0084] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0085] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0086] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0087] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0088] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0089] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0090] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0091] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0092] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0093] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0094] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A social trust-driven facial information protection method, characterized by: The following steps are involved: S1: The system initializes three entities, including the data publisher, the trusted third party, and the data accessor; S2: The trusted third party uses the graph attention mechanism to calculate the real-time social trust value between the data publisher and the data accessor; S3: Assign a privacy level to the data accessor based on his / her social trust value, and use the differential privacy mechanism to generate a protected identity vector, where the privacy level is used to control the size of the privacy budget; S4: Input the protected identity vector into the conditional identity anonymous generative adversarial network to obtain the face information protection image, and send it to the corresponding data accessor.

2. The social trust-driven facial information protection method according to claim 1, characterized in that: The system initialization in step S1 includes: Initialize three entities, including data publisher, trusted third party, and data accessor; The data publisher is a user who generates images and shares them on social networks; The trusted third party is the transfer party of social network image upload, which builds a social trust graph based on the basic attributes of data publishers and data accessors, and is responsible for the social trust evaluation of users and image processing; The data accessor is used to send a data request to a trusted third party to obtain the data published by the data publisher; Initialize the user set of online social network in the social trust graph The set of all user social interactions in the period [0, T] is in, represents the set of neighboring nodes that have interacted with node i within time [0, T]; social trust is modeled as a series of interaction events with timestamps in the social trust graph G = {e(t1), e(t2), ...}, where e(t m ) indicates t m The social interaction between any pair of user nodes at time t; user nodes i and j at time t m The social trust formed with a value of c is represented by a directed edge e c ij (t m ).

3. The social trust-driven facial information protection method according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21: The trusted third party generates a social trust graph G(T) at a given time T, where the nodes in the graph are social network users and the edges are the social trust values ​​c connecting pairs of users; S22: Given a pair of social user nodes i, j, first extract the user's basic attributes s i , user i and neighboring nodes k∈N i (T) Historical Interaction and the associated timestamp Ψ t ,according to Generate joint embedding vector n ik , to quantify the message signal from user i to neighboring user k; S23: Weighted calculation and aggregation of neighboring users k∈N i The joint embedding vector n of (T) ik In this process, the graph attention mechanism is executed to assign higher weights to the latest interactions. Let the weight assigned by the user to each neighboring user k be expressed as a ik , and finally obtain the first-order feature representation of user i S25: By stacking L trust propagation layers, aggregate the influence of multi-hop user nodes on user i to obtain high-order feature representation S26: Calculate the high-order feature representation of the user node pair i, j in step S25 and It is passed into the MLP and processed through a softmax layer to output the predicted probability h of social trust between any user pair i and j. ij ; Finally, through c = argmax (h ij ) Calculate the social trust c of any pair of users.

4. The social trust-driven facial information protection method according to claim 3, characterized in that: In step S22, the mapping function is used Get timestamp t ; The historical interaction trust value between user i and neighboring nodes Convert to dense vector embedding; then embed the user's attributes into vector s i , timestamp embedding vector Ψ t and the interaction embedding vector according to Generate a joint embedding vector as the input of the trust propagation layer model.

5. The social trust-driven facial information protection method according to claim 3, characterized in that: In step S23, the graph attention mechanism is used to obtain the interaction weight a assigned by user i to user k ik ←attn(n ik W ik ,p i W i )=n ik W ik ·(p i W i ) T ,in, and is a trainable matrix, d s d t d c is the embedding vector dimension, and then use Get the normalized weight; calculate and aggregate the joint embedding vector n of adjacent users by weighted calculation ik To obtain the first-order feature representation of user i Among them, W, b are the parameters that need to be trained, k∈N i (T) is the set of neighboring nodes of user i.

6. The social trust-driven facial information protection method according to claim 1, characterized in that: In step S26, the high-level feature representation of user i and j is and Passed to MLP, output social trust prediction probability h ij : Among them, W f 、b f is the trainable weight matrix defined in the multilayer perceptron; The objective function of the entire model is defined as the cross entropy loss between the predicted value and the true value of the confidence in the graph G: in Represents all trainable model parameters.

7. The social trust-driven facial information protection method according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31: assigning a trust level d to the accessing user according to the social trust value c calculated in step S2; S32: Extract features from the original image and integrate to obtain the identity vector I id ; S33: Use differential privacy mechanism based on visitor’s trust level d Get the protected identity vector I i ' d ,in, Indicates that the mean is 0 and the scale is Laplace distributed random noise, Δ is a parameter used to control the strength of privacy protection in differential privacy, which is related to the sensitivity of the data; d is the visitor’s trust level, which determines I id The smaller the d value, the greater the added noise. i ' d with I id The larger the gap, the smaller the noise. i ' d with I id The smaller the gap.

8. The social trust-driven facial information protection method according to claim 7, characterized in that: Step S4 specifically includes the following steps: The perturbed identity vector obtained in step S33, the non-face region image of the original image, and the target identity image are used as inputs of the generator in the generative adversarial network; The generative adversarial network uses an encoder-decoder architecture, where the encoder represents the input combination in step S41 as a low-dimensional vector, and the decoder then represents the vector combination, and generates a face information protection image by upsampling; The loss function is set as: L total =L G +L D +L Rec Where L G , L D is the loss function of the generator and the discriminator, L Rec is the image reconstruction loss; Where a represents the label of generated data, b represents the label of real data; D(x) is the output of discriminator D for real data x; (D(x)-b) 2 represents the square error between the discriminator's output of the real data and the real label; x~p data (x) represents the distribution of real data p data The data sampled from the generator G; D(G(z)) represents the output of the discriminator D on the data G(z) generated by the generator G; (D(G(z))-a) 2 The squared error between the discriminator's output of the generated data and the generated label; represents the squared error between the output of the discriminator and the generated label from the prior distribution z~p z The noise z sampled in (z); L Rec =‖x raw -x adv ‖。

Images