Face recognition method, electronic device and storage medium based on privacy protection

Abstract

The embodiments of the present application relate to the field of face recognition processing and disclose a face recognition method, electronic device, and storage medium based on privacy protection. The method includes: obtaining a first image matrix of a face image set to be recognized, a second feature matrix of a target face image set, and a projection matrix; generating a first vector and a second vector based on the number of rows of the first image matrix; using the first vector and the second vector to encrypt the projection matrix and the second feature matrix, respectively, to obtain an encrypted projection matrix and an encrypted second feature matrix; using the first vector and the second vector to encrypt the first image matrix to obtain an encrypted first image matrix; sending the encrypted projection matrix and the encrypted second feature matrix to each edge server, and sending multiple encrypted first sub-image matrices obtained by evenly splitting the encrypted first image matrix to the edge server; and receiving the face recognition results sent by the edge server. The present application improves the efficiency of face recognition and also avoids the leakage of face data.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of face recognition processing, and in particular to a face recognition method, electronic device, and storage medium based on privacy protection. Background Art

[0002] In today's information age, facial recognition technology is widely used in a variety of fields, including smart travel, identity recognition, access control systems, and surveillance systems. The privacy of facial data and the efficiency of facial recognition are key issues in facial recognition.

[0003] In the current IoT landscape, many IoT applications require high real-time facial recognition performance. For example, in smart security systems, monitoring dense crowds requires rapid identification and targeting of target subjects. Specifically, online facial recognition involves processing the target subject's facial data, acquiring a target facial database, and then using smart monitoring equipment to collect facial images of the subject in public places. These images are then matched against the target facial database. Smart security systems place high demands on facial recognition response, and facial images are typically stored in matrix form. The facial recognition process involves complex matrix operations, which place a computational burden on resource-constrained user servers.

[0004] Although traditional face recognition methods based on remote cloud server assistance can solve the computational burden brought by complex matrix operations to a certain extent, they are difficult to meet the real-time response requirements of IoT applications and cannot completely avoid the risk of facial feature information leakage in the cloud. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a face recognition method, electronic device and storage medium based on privacy protection, which encrypts the projection matrix, the first image matrix and the second feature matrix by only storing the first vector and the second vector on the user server, and evenly splits the encrypted first image matrix into multiple encrypted first sub-image matrices and sends them to the edge server, so that multiple edge servers simultaneously calculate different parts of the encrypted first image matrix in parallel, effectively improving the efficiency of face recognition and avoiding the leakage of face data information by the edge server.

[0006] To solve the above technical problems, an embodiment of the present application provides a face recognition method based on privacy protection, which is applied to a user server, including: obtaining a first image matrix corresponding to a face image set to be identified, a second feature matrix corresponding to a target face image set, and a projection matrix for converting the second image matrix of the target face image set into the second feature matrix; generating an n-dimensional non-zero first vector and a second vector based on the number of rows of the first image matrix; wherein the L2 norm of the first vector is equal to 1; using the first vector and the second vector to encrypt the projection matrix and the second feature matrix respectively to obtain an encrypted projection matrix and an encrypted second feature matrix; using the first vector and the second vector to encrypt the first image matrix to obtain an encrypted first image matrix; sending the encrypted projection matrix and the encrypted second feature matrix to each edge server, and sending multiple encrypted first sub-image matrices obtained by evenly splitting the encrypted first image matrix to each edge server, so that each edge server calculates the distance between the encrypted first sub-feature matrix and the encrypted second feature matrix based on the encrypted projection matrix and the encrypted first sub-image matrix to obtain a face recognition result of the face image to be identified; and receiving the face recognition result sent by the edge server.

[0007] An embodiment of the present application also provides a face recognition method based on privacy protection, which is applied to an edge server, including: for each of the edge servers, receiving an encrypted projection matrix, an encrypted first sub-image matrix and the encrypted second feature matrix sent by a user server; obtaining an encrypted first sub-feature matrix based on the encrypted projection matrix and the encrypted first sub-image matrix; calculating the distance between each encrypted first eigenvector in the encrypted first sub-feature matrix and each encrypted second eigenvector in the encrypted second feature matrix; when the distance is less than a preset threshold, determining that the face image to be identified corresponding to the encrypted first eigenvector corresponding to the distance matches the target face image corresponding to the encrypted second eigenvector corresponding to the distance, obtaining a face recognition result, and sending the face recognition result to the user server.

[0008] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so as to enable the at least one processor to execute the privacy-protection-based face recognition method mentioned in the above embodiment.

[0009] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the face recognition method based on privacy protection mentioned in the above embodiment.

[0010] The privacy-preserving face recognition method provided by the embodiment of the present application generates an n-dimensional non-zero first vector and second vector based on the number of rows of the first image matrix, then uses the first vector to encrypt the projection matrix to obtain an encrypted projection matrix, uses the second vector to encrypt the second feature matrix to obtain an encrypted second feature matrix, and uses the first vector and the second vector to jointly encrypt the first image matrix to obtain an encrypted first image matrix. In other words, the first vector and the second vector are equivalent to encryption keys, and the encryption keys are only stored in the user server. The encrypted projection matrix and the encrypted second feature matrix are sent to each edge server, and the encrypted first image matrix is ​​evenly split into multiple encrypted first sub-image matrices and sent to the edge servers. In this way, each edge server obtains a portion of the encrypted first image matrix (i.e., the encrypted first sub-image matrix). Multiple edge servers calculate the encrypted projection matrix, the encrypted first sub-image matrix, and the encrypted second feature matrix in parallel, effectively improving the efficiency of face recognition. Since the first vector and the second vector are only stored in the user server, the edge server only obtains the encrypted data, which can effectively avoid the leakage of face data information by the edge server.

[0011] In addition, the privacy-preserving face recognition method provided in an embodiment of the present application obtains a first image matrix corresponding to a set of facial images to be recognized, including: for q facial images to be recognized in the set, concatenating the pixel values ​​of each facial image to be recognized by column, end to end, to form an n-dimensional column vector; and concatenating the q n-dimensional column vectors to obtain an n×q-dimensional first image matrix corresponding to the set of facial images to be recognized. The present application represents each facial image in the set of facial images to be recognized as a column vector and the entire set of facial images as the first image matrix, facilitating subsequent rapid calculations.

[0012] In addition, the face recognition method based on privacy protection provided by the embodiment of the present application, wherein the first vector and the second vector are used to encrypt the projection matrix and the second feature matrix respectively to obtain an encrypted projection matrix and an encrypted second feature matrix, including: constructing a Householder matrix based on the first vector, and encrypting the projection matrix using the Householder matrix to obtain an encrypted projection matrix; constructing an n×m-dimensional matrix based on the second vector and the number of columns of the second feature matrix, and encrypting the second feature matrix using the n×m-dimensional matrix and the projection matrix to obtain an encrypted second feature matrix. The present application uses the Householder matrix to encrypt the projection matrix. Since the Householder matrix is ​​a dense matrix, it is more secure. The second feature matrix is ​​encrypted using an n×m-dimensional matrix, where n is the number of rows of the first image matrix and m is the number of columns of the second feature matrix. This is conducive to achieving the face recognition result calculated based on the ciphertext data (encrypted first sub-feature matrix and encrypted second feature matrix), that is, the face recognition result calculated based on the plaintext data (first sub-feature matrix and second feature matrix).

[0013] In addition, the privacy-preserving face recognition method provided in the embodiment of the present application, wherein the first image matrix is ​​encrypted using the first vector and the second vector to obtain an encrypted first image matrix, includes: constructing a Householder matrix based on the first vector, constructing an n×q-dimensional matrix based on the second vector and the number of columns of the first image matrix; and encrypting the first image matrix using the Householder matrix and the n×q-dimensional matrix to obtain an encrypted first image matrix. The present application uses an n×q-dimensional matrix with the same number of rows and columns as the first image matrix and an orthogonal Householder matrix to encrypt the first image matrix, which is conducive to achieving a face recognition result calculated based on ciphertext data (encrypted first sub-feature matrix and encrypted second feature matrix), that is, a face recognition result calculated based on plaintext data (first sub-feature matrix and second feature matrix). BRIEF DESCRIPTION OF THE DRAWINGS

[0014] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0015] Figure 1 This is the process of the face recognition method based on privacy protection provided by the embodiment of this application Figure 1 ;

[0016] Figure 2 This is the process of the face recognition method based on privacy protection provided by the embodiment of this application Figure 2 ;

[0017] Figure 3 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] To make the purpose, technical solutions, and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that many technical details are provided in each embodiment of the present application to help readers better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0019] The following examples illustrate the implementation details of the privacy-preserving face recognition method of this embodiment. The following content is only provided for ease of understanding and is not required for the implementation of this solution.

[0020] The embodiment of the present application relates to a face recognition method based on privacy protection, which is applied to a user server, such as Figure 1 As shown, including:

[0021] Step 101: Obtain a first image matrix corresponding to a face image set to be recognized, a second feature matrix corresponding to a target face image set, and a projection matrix for converting the second image matrix of the target face image set into a second feature matrix.

[0022] In this embodiment, the first image matrix B corresponding to the face image set to be identified and the second image matrix A corresponding to the target face image set can be the original image matrix that has not been preprocessed, or the image matrix that has been preprocessed (such as centering or filtering). The projection matrix W is used to convert the second image matrix A of the target face image set into the second feature matrix Y. That is, the projection matrix represents the conversion relationship from the image space to the feature space, which is expressed as Y=W T A.

[0023] Among them, the target face image set has been acquired and stored in the face database during the training phase. Similarly, the projection matrix, the second image matrix corresponding to the target face image set, and the second feature matrix have all been acquired during the training phase (offline state). This application focuses on the recognition phase after the training phase (online state).

[0024] In one embodiment, obtaining a first image matrix corresponding to the set of facial images to be identified includes: for q facial images to be identified in the set of facial images to be identified, connecting the pixel values ​​of each facial image to be identified column by column to form an n-dimensional column vector; and splicing the q n-dimensional column vectors to obtain an n×q-dimensional first image matrix B corresponding to the set of facial images to be identified.

[0025] Specifically, each column vector in the first image matrix B represents a facial image to be recognized. If the size of the facial image to be recognized is a×b, then the number of rows n in the first image matrix equals the size of the facial image to be recognized (a×b), and the number of columns in the first image matrix B equals the number of facial images to be recognized. Similarly, in the n×m-dimensional second image matrix A corresponding to the target facial image set, where n equals the size of the target facial image (a×b) and m equals the number of target facial images, each column vector in the second image matrix B represents a target facial image.

[0026] Step 102 : Generate an n-dimensional non-zero first vector and a second vector based on the number of rows of the first image matrix; wherein the L2 norm of the first vector is equal to 1.

[0027] In this embodiment, assuming that the number of rows of the first image matrix is ​​n, the first vector V and the second vector C are both n-dimensional non-zero column vectors, and the first vector is expressed as V = [v1, v2, ..., v n ] T , the second vector is represented as C = [c1, c2, ..., c n ] T , the first vector satisfies ||V||2 = 1. The two n-dimensional random vectors V and C constitute the encryption key, which is secretly held by the user server.

[0028] Step 103: Use the first vector and the second vector to encrypt the projection matrix and the second characteristic matrix respectively to obtain an encrypted projection matrix and an encrypted second characteristic matrix; and use the first vector and the second vector to encrypt the first image matrix to obtain an encrypted first image matrix.

[0029] In this embodiment, the projection matrix W is encrypted using the first vector V to obtain the encrypted projection matrix W′. The second characteristic matrix Y is encrypted using the second vector C to obtain the encrypted second characteristic matrix Y′. The first vector V and the second vector C are used together to encrypt the first image matrix B to obtain the encrypted first image matrix B′.

[0030] In one embodiment, a projection matrix and a second characteristic matrix are encrypted using a first vector and a second vector, respectively, to obtain an encrypted projection matrix and an encrypted second characteristic matrix, including: constructing a Householder matrix H based on the first vector V, and encrypting the projection matrix W using the Householder matrix H to obtain an encrypted projection matrix W′; constructing an n×m-dimensional matrix C′ based on the second vector C and the number of columns of the second characteristic matrix, and encrypting the second characteristic matrix Y using the n×m-dimensional matrix C′ and the projection matrix W to obtain an encrypted second characteristic matrix Y′.

[0031] Specifically, the Householder matrix H = I-2VV is constructed based on the n-dimensional first vector VT , the Householder matrix H is a symmetric matrix, then the dimension of the Householder matrix H is n×n, and the dimension of the projection matrix W is n×k, where k<n. The Householder matrix H is used to encrypt the projection matrix W as W′=HW to obtain the encrypted projection matrix W′.

[0032] Since the k×n-dimensional transposed projection matrix W T Multiplying the second image matrix A by the n×m matrix yields the second feature matrix Y. The dimension of the second feature matrix is ​​k×m. It is understood that each column vector in the second feature matrix Y is a feature vector corresponding to the target face image, and m is the number of target face images.

[0033] Based on the second vector C and the number of columns m of the second characteristic matrix Y, an n×m dimensional matrix C′=(C1, C2, …, C m ), that is, the n×m dimensional matrix C′ has m n-dimensional second vectors C. The second characteristic matrix Y is encrypted using the n×m dimensional matrix C′ and the projection matrix W: Y′=Y+W T C′, and obtain the encrypted second characteristic matrix Y′.

[0034] In one embodiment, a Householder matrix H is constructed based on the first vector V, and an n×q-dimensional matrix C″ is constructed based on the second vector C and the number of columns q of the first image matrix B; the first image matrix B is encrypted using the Householder matrix H and the n×q-dimensional matrix C″ to obtain an encrypted first image matrix B′.

[0035] Specifically, the Householder matrix H = I-2VV is constructed based on the n-dimensional first vector V T , the Householder matrix H is a symmetric matrix, and the dimension of the Householder matrix H is n×n. Based on the second vector C and the number of columns of the n×q-dimensional first image matrix, an n×q-dimensional matrix C″=(C1,C2,…,C q ), that is, the n×q-dimensional matrix C″ has q n-dimensional second vectors C. The Householder matrix H and the n×q-dimensional matrix C″ are used to encrypt the first image matrix B′=H(B+C″) to obtain the encrypted first image matrix B′,

[0036] In the process of encrypting the projection matrix H and the first image matrix B, this application uses the Householder matrix. The Householder matrix has symmetry, orthogonality and non-singularity, and it is a dense matrix. The encryption process is more secure and can reduce the complexity of the encryption calculation from n³ to n², thereby improving computing efficiency.

[0037] In step 104, the encrypted projection matrix and the encrypted second feature matrix are sent to each edge server, and multiple encrypted first sub-image matrices obtained by evenly splitting the encrypted first image matrix are sent to each edge server, so that each edge server calculates the distance between the encrypted first sub-feature matrix and the encrypted second feature matrix based on the encrypted projection matrix and the encrypted first sub-image matrix to obtain the face recognition result of the face image to be recognized.

[0038] In this embodiment, the encrypted first image matrix B′ is split into multiple encrypted first sub-image matrices by column according to the number of edge servers. For example, if there are five edge servers S (S1, S2, S3, S4, S5), the dimension of the encrypted first image matrix B′ is 9×10, and each column vector represents an encrypted face image to be recognized. The encrypted first image matrix is ​​evenly split into five 9×2 encrypted first sub-image matrices by column. The first encrypted first sub-image matrix (The first and second columns are the first and second columns of the encrypted first image matrix B′) are sent to the edge server S1, and the second encrypted first sub-image matrix (The first and second columns are the third and fourth columns of the encrypted first image matrix B′) are sent to the edge server S2, and the third encrypted first sub-image matrix The first and second columns (the fifth and sixth columns of the encrypted first image matrix B′) are sent to edge server S3. Similarly, the remaining encrypted first sub-image matrices are sent to multiple edge servers in the same manner. In this way, all five edge servers participate in the face recognition process for the ten facial images to be identified. These five edge servers perform parallel computations while independently executing their respective computational tasks, fully utilizing the computational efficiency of the edge servers, improving face recognition efficiency, reducing latency, and accelerating face recognition response speed.

[0039] That is, the present application splits the encrypted first image matrix B′ of dimension n×q into p blocks by columns to obtain the encrypted first sub-image matrix Right now Each encrypted first sub-image matrix The dimension is n×q j (q j =q / p), encrypt the first sub-image matrix Send to the corresponding j-th edge server S among the p edge servers j .

[0040] Furthermore, each edge server receives the encrypted projection matrix W′, the encrypted second feature matrix Y′ (target face image) and the encrypted first sub-image matrix Then, calculate the encrypted first sub-feature matrix That is, the encrypted image space of the face to be identified is converted into an encrypted feature space through the encrypted projection matrix. It can be understood that multiple encrypted first sub-feature matrices are converted into encrypted feature space. The encrypted first feature matrix Z′ corresponding to the face image set to be identified can be obtained by splicing and combining. Each column vector in the encrypted first sub-feature matrix represents an encrypted first feature vector of a face image to be identified.

[0041] Edge server for encryption of the first sub-feature matrix Each column vector in the encrypted first eigenvector Z i ′(1≤i≤q j ), calculate its value and each column vector in the encrypted second characteristic matrix Y′, that is, the encrypted second characteristic vector Y t ′’s Euclidean distance:

[0042] d i，t =||Z i ′-Y t ′||2(1≤t≤m)

[0043] If d i，t is less than the threshold δ, then the i-th first eigenvector Z i ′ corresponds to the i-th face image to be recognized and the t-th target face image corresponding to the t-th second eigenvector, then add the subscript pair {i, t} to the recognition result set Res j The edge server sends the recognition results to the user server.

[0044] Step 105: Receive the face recognition result sent by the edge server.

[0045] In this embodiment, the user server receives the face recognition result calculated based on the ciphertext data sent by the edge server, and can directly determine the face recognition result of the plaintext data without involving the decryption algorithm. i ′ and the encrypted second eigenvector Y t ′ is the first eigenvector Z of the plaintext data i With the second eigenvector Y t The Euclidean distance.

[0046] This conclusion can be drawn through the following reasoning: p edge servers cooperate to calculate Z′=(W′) T B′. According to the data encryption process, W′=HW and B′=H(B+C″). Combined, we can get: Z′=(HW) T H(B+C″)=W T H TH(B+C″).

[0047] According to the orthogonality of the Householder matrix H T =H -1 , that is, H T H = I, so Z′ = W T (B+C″).

[0048] Each column vector Z in Z′ i ′=W T (B i +C). The Euclidean distance d calculated by the edge server i，t =||Z i ′-Y t ′||2. According to Y t ′=Y t +W T C, we can get:

[0049] d i，t =||Z i ′-Y t ′||2=||W T (B i +c)-(Y t +W T C)||2

[0050] =||W T B i -Y t ||2=||Z i -Y t ||2

[0051] That is, encrypt the first eigenvector Z i ′ and the encrypted second eigenvector Y t ′ is equal to the first eigenvector Z i With the second eigenvector Y t The user server in this application encrypts facial data and sends it to multiple edge servers for calculation. The encryption key is only stored on the user server. The edge server directly calculates the facial recognition process based on the encrypted facial data and returns the result to the user server. The user server directly receives the facial recognition result, which not only ensures the privacy of facial data but also realizes real-time response of facial recognition.

[0052] Furthermore, after step 104, the method further includes: obtaining a target face image that matches each face image to be recognized in the face image set according to the face recognition result; obtaining a first feature matrix Z of the face image set according to the first image matrix B and the projection matrix W of the face image set; and obtaining a first feature vector Z corresponding to each face image to be recognized based on the first feature matrix Z.i The second eigenvector Y corresponding to the matched target face image in the second eigenmatrix Y i The distance between them is used to verify whether the face recognition result is accurate.

[0053] Specifically, after receiving the face recognition result, the user server obtains the index pair {i, t} that matches the face image to be recognized and the target face image, and calculates the first eigenvector Z i and the second eigenvector Y t ) Verify the face recognition results. The verification process is: i′，t =||W T B i′ -Y t ||2(i′=q1(j-1)+i). If d i′，t <δ, then {i, t} passes the verification and is indeed the index of the face data that matches successfully; otherwise, an error is returned to the jth edge server. i represents the serial number of the encrypted first feature vector in the encrypted first sub-feature matrix, j represents the serial number of the edge server, and q1 represents the first encrypted first sub-image matrix The number of columns, i′ represents the sequence number of the column vector (face image to be recognized) in the first image matrix.

[0054] That is, the formula i′=q1(j-1)+i is used to set the j-th edge server based on the j-th encrypted first sub-image matrix The calculated serial number i of the face image to be recognized is converted into the serial number i′ of the corresponding face image to be recognized in the first image matrix B.

[0055] The privacy-preserving face recognition method provided by the embodiment of the present application generates an n-dimensional non-zero first vector and second vector based on the number of rows of the first image matrix, then uses the first vector to encrypt the projection matrix to obtain an encrypted projection matrix, uses the second vector to encrypt the second feature matrix to obtain an encrypted second feature matrix, and uses the first vector and the second vector to jointly encrypt the first image matrix to obtain an encrypted first image matrix. In other words, the first vector and the second vector are equivalent to encryption keys, and the encryption keys are only stored in the user server. The encrypted projection matrix and the encrypted second feature matrix are sent to each edge server, and the encrypted first image matrix is ​​evenly split into multiple encrypted first sub-image matrices and sent to the edge servers. In this way, each edge server obtains a portion of the encrypted first image matrix (i.e., the encrypted first sub-image matrix). Multiple edge servers calculate the encrypted projection matrix, the encrypted first sub-image matrix, and the encrypted second feature matrix in parallel, effectively improving the efficiency of face recognition. Since the first vector and the second vector are only stored in the user server, the edge server only obtains the encrypted data, which can effectively avoid the leakage of face data information by the edge server.

[0056] The embodiment of this application relates to a face recognition method based on privacy protection, which is applied to an edge server, such as Figure 2 As shown, including:

[0057] Step 201: For each edge server, receive the encrypted projection matrix, the encrypted first sub-image matrix, and the encrypted second feature matrix sent by the user server.

[0058] Specifically, multiple edge servers receive the encrypted projection matrix, encrypted first sub-image matrix, and encrypted second feature matrix sent by the user server, and independently perform their respective computing tasks while performing parallel calculations. This fully utilizes the computing efficiency of the edge servers, improves the efficiency of face recognition, reduces latency, and speeds up the response speed of face recognition.

[0059] Step 202: Obtain an encrypted first sub-feature matrix according to the encrypted projection matrix and the encrypted first sub-image matrix.

[0060] Specifically, each edge server receives the encrypted projection matrix W′, the encrypted second feature matrix Y′ (target face image), and the encrypted first sub-image matrix Then, calculate the encrypted first sub-feature matrix That is, the encrypted image space of the face to be identified is converted into an encrypted feature space through the encrypted projection matrix. It can be understood that multiple encrypted first sub-feature matrices are converted into encrypted feature space. The encrypted first feature matrix z′ corresponding to the face image set to be identified can be obtained by splicing and combining. Each column vector in the encrypted first sub-feature matrix represents an encrypted first feature vector of a face image to be identified.

[0061] Step 203 : Calculate the distance between each encrypted first eigenvector in the encrypted first sub-eigenmatrix and each encrypted second eigenvector in the encrypted second sub-eigenmatrix.

[0062] Specifically, the edge server encrypts the first sub-feature matrix Each column vector in the encrypted first eigenvector Z i ′(1≤i≤q j ), calculate its value and each column vector in the encrypted second characteristic matrix Y′, that is, the encrypted second characteristic vector Y t ′’s Euclidean distance:

[0063] d i，t =||Z i ′-Y t ′||2(1≤t≤m)

[0064] If d i，t is less than the threshold δ, then the i-th first eigenvector Zi ′ corresponds to the i-th face image to be recognized and the t-th target face image corresponding to the t-th second eigenvector, then add the subscript pair {i, t} to the recognition result set Res j The edge server sends the recognition results to the user server.

[0065] Step 204: When the distance is less than a preset threshold, determine that the face image to be identified corresponding to the encrypted first feature vector corresponding to the distance matches the target face image corresponding to the encrypted second feature vector corresponding to the distance, obtain a face recognition result, and send the face recognition result to the user server.

[0066] The privacy-preserving face recognition method provided by the embodiments of this application leverages the computing resources of edge servers, enabling resource-constrained users to complete large-scale face recognition tasks. Parallel computing across multiple edge servers improves face recognition efficiency, reduces latency, and speeds up response times.

[0067] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0068] The embodiments of the present application relate to an electronic device, such as Figure 3 As shown, including:

[0069] At least one processor 301; and a memory 302 communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 to enable the at least one processor 301 to perform the model training mentioned in the above embodiment, or to perform the kinship identification method mentioned in the above embodiment.

[0070] The electronic device includes: one or more processors 301 and a memory 302, Figure 3 A processor 301 is used as an example. The processor 301 and the memory 302 may be connected via a bus or other means. Figure 3The example of a bus connection is used. Memory 302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. For example, the algorithms corresponding to the various processing strategies within the policy space in the embodiments of the present application are stored in memory 302. Processor 301 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 302, thereby implementing the above-mentioned face recognition method or kinship recognition method based on privacy protection.

[0071] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 302 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 302 may optionally include a memory remotely located relative to the processor 301, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0072] One or more modules are stored in the memory 302 and, when executed by one or more processors 301 , perform the face recognition method based on privacy protection in any of the above embodiments.

[0073] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0074] The embodiments of the present application relate to a computer-readable storage medium storing a computer program, which implements the above method embodiments when executed by a processor.

[0075] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0076] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A face recognition method based on privacy protection, characterized in that: Applied to a user server, the method includes: Obtaining a first image matrix corresponding to the face image set to be recognized, a second feature matrix corresponding to the target face image set, and a projection matrix for converting the second image matrix of the target face image set into the second feature matrix; generating an n-dimensional non-zero first vector and a second vector based on the number of rows of the first image matrix; wherein the L2 norm of the first vector is equal to 1; Encrypting the projection matrix and the second characteristic matrix using the first vector and the second vector, respectively, to obtain an encrypted projection matrix and an encrypted second characteristic matrix; encrypting the first image matrix using the first vector and the second vector, to obtain an encrypted first image matrix; Sending the encrypted projection matrix and the encrypted second feature matrix to each edge server, and sending a plurality of encrypted first sub-image matrices obtained by evenly splitting the encrypted first image matrix to each edge server, so that each edge server calculates the distance between the encrypted first sub-feature matrix and the encrypted second feature matrix based on the encrypted projection matrix and the encrypted first sub-image matrix to obtain a face recognition result for the face image to be recognized; Receive the face recognition result sent by the edge server.

2. The face recognition method based on privacy protection according to claim 1, characterized in that: Obtaining the first image matrix corresponding to the set of face images to be recognized, including: For the q facial images to be identified in the facial image set, the pixel values ​​of each facial image to be identified are connected head to tail in columns to form an n-dimensional column vector; The q n-dimensional column vectors are concatenated to obtain a first image matrix of n×q dimensions corresponding to the set of face images to be recognized.

3. The face recognition method based on privacy protection according to claim 1, characterized in that: The using the first vector and the second vector to encrypt the projection matrix and the second characteristic matrix respectively to obtain an encrypted projection matrix and an encrypted second characteristic matrix includes: Constructing a Householder matrix based on the first vector, and encrypting the projection matrix using the Householder matrix to obtain an encrypted projection matrix; An n×m dimensional matrix is ​​constructed based on the second vector and the number of columns of the second characteristic matrix, and the second characteristic matrix is ​​encrypted using the n×m dimensional matrix and the projection matrix to obtain an encrypted second characteristic matrix.

4. The face recognition method based on privacy protection according to claim 1, characterized in that: The step of encrypting the first image matrix by using the first vector and the second vector to obtain an encrypted first image matrix includes: constructing a Householder matrix based on the first vector, and constructing an n×q dimensional matrix based on the second vector and the number of columns of the first image matrix; The first image matrix is ​​encrypted using the Householder matrix and the n×q dimensional matrix to obtain an encrypted first image matrix.

5. The face recognition method based on privacy protection according to claim 3, characterized in that: The projection matrix is ​​encrypted using the Householder matrix using the following formula to obtain an encrypted projection matrix: W′=HW H=I-2VV T Where W′ is the encrypted projection matrix, H is the Householder matrix, W is the projection matrix, I is the identity matrix, V is the first n-dimensional vector, V T is the transposed vector of the first vector; The second characteristic matrix is ​​encrypted using the n×m dimensional matrix and the projection matrix using the following formula to obtain an encrypted second characteristic matrix: Y′=Y+W T C′ C′=(C1,C2,...,C m ),C i =C(1≤i≤m) Where Y′ is the encrypted second characteristic matrix, Y is the second characteristic matrix, W is the projection matrix, I is the identity matrix, C is the n-dimensional second vector, and C′ is an n×m-dimensional matrix.

6. The face recognition method based on privacy protection according to claim 4, characterized in that: The Householder matrix and the n×q-dimensional matrix are used to encrypt the first image matrix using the following formula to obtain an encrypted first image matrix: B′=H(B+C″) H=I-2VV T C″=(C1,C2,…,C q ),C i =C(1≤i≤q) Where B′ is the encrypted first image matrix, B is the first image matrix, H is the Householder matrix, I is the identity matrix, V is the n-dimensional first vector, C is the n-dimensional second vector, C″ is the n×q-dimensional matrix, V T is the transposed vector of the first vector.

7. The face recognition method based on privacy protection according to any one of claims 1 to 6, characterized in that: After receiving the face recognition result sent by the edge server, the method further includes: Acquire a target face image that matches each face image to be recognized in the face image set according to the face recognition result; Obtaining a first feature matrix of the face image set to be identified based on the first image matrix of the face image set to be identified and the projection matrix; Verify whether the face recognition result is accurate based on the distance between the first eigenvector corresponding to each to-be-recognized face image in the first eigenmatrix and the second eigenvector corresponding to the matched target face image in the second eigenmatrix.

8. A face recognition method based on privacy protection, characterized in that: Applied to an edge server, the method includes: For each of the edge servers, receiving the encrypted projection matrix, the encrypted first sub-image matrix, and the encrypted second feature matrix sent by the user server; Obtaining an encrypted first sub-feature matrix according to the encrypted projection matrix and the encrypted first sub-image matrix; Calculating the distance between each encrypted first eigenvector in the encrypted first sub-eigenmatrix and each encrypted second eigenvector in the encrypted second sub-eigenmatrix; When the distance is less than a preset threshold, it is determined that the face image to be identified corresponding to the encrypted first feature vector corresponding to the distance matches the target face image corresponding to the encrypted second feature vector corresponding to the distance, a face recognition result is obtained, and the face recognition result is sent to the user server.

9. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the privacy-protection-based face recognition method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the face recognition method based on privacy protection according to any one of claims 1 to 8 is implemented.

Images