Biometric feature extraction method and device for multi-party secure computing system

Abstract

The present invention relates to a method for extracting biometric features for a multi-party secure computing system, the method comprising: obtaining a biometric preprocessing slice; and extracting features from the preprocessing slice using a neural network, wherein in the first N layers of convolution and pooling layers, data calculations are performed on the preprocessing slices respectively by multi-party secure computing to obtain intermediate data; in the convolution and pooling layers after the Nth layer, the intermediate data is aggregated to a single-party server platform for subsequent calculations, and feature extraction is completed after inputting into a fully connected layer. The present invention also relates to a biometric feature extraction device, a computer storage medium, and a computer program product for a multi-party secure computing system.

Description

Technical Field

[0001] The present invention relates to the field of biometric feature extraction, and more specifically, to a biometric feature extraction method and device for a multi-party secure computing system, a computer storage medium, and a computer program product. Background Art

[0002] In recent years, biometrics (such as face and fingerprint) have gradually become the mainstream identification method due to their close correlation with individual identity, and have been widely used in important financial scenarios such as withdrawal, payment, and mobile banking login. At present, most biometric extraction systems directly collect and store the plaintext original images of biometrics (such as the original images of face and fingerprint). This solution can achieve good biometric matching, but it also has huge security risks and privacy compliance issues. Because once the biometric data is leaked, it cannot be changed or destroyed, and biometrics are related to user privacy, directly collecting and storing the plaintext original images will have compliance risks.

[0003] Multi-party secure computing, also known as secure multi-party computing or MPC, SMPC (Secure Multi-Party Computation), is a good technical solution. Biometric extraction based on MPC technology can achieve secure sharing of biometric information and reduce the risk of personal and property information caused by biometric information leakage. However, in the biometric extraction scenario based on multi-party secure computing, the computational complexity and communication overhead of multi-party MPC platform interaction are huge, which does not meet the actual application requirements of production scenarios. Summary of the invention

[0004] According to one aspect of the present invention, a biometric feature extraction method for a multi-party secure computing system is provided, the method comprising: obtaining a biometric feature preprocessing slice; and performing feature extraction on the preprocessing slice using a neural network, wherein in the first N convolution and pooling layers in the neural network, a multi-party server platform in the multi-party secure computing system performs data calculations on the preprocessing slice respectively through multi-party secure computing to obtain intermediate data, the intermediate data being the processing results of the first N convolution and pooling layers, the processing results cannot be reversibly restored to the preprocessing slice, and N is a natural number; in the convolution and pooling layers after the Nth layer of the neural network, the intermediate data is aggregated to a single-party server platform in the multi-party secure computing system for subsequent calculations, and feature extraction is completed after inputting into a fully connected layer.

[0005] As a supplement or replacement for the above solution, in the above method, completing feature extraction after inputting the fully connected layer includes: in the fully connected layer, through multi-party secure computing, each server platform obtains a slice of the biometric feature value respectively.

[0006] As a supplement or alternative to the above-mentioned scheme, in the above-mentioned method, the sharding of the biometric value is realized by constructing a computing operator corresponding to the party server platforms in the average pooling layer and the fully connected layer, wherein the sharding of the biometric value includes a first feature vector and a second feature vector, wherein the first feature vector is stored on a first-party server platform of the party server platforms, and the second feature vector is stored on a second-party server platform of the party server platforms.

[0007] As a supplement or replacement for the above solution, the above method may further include: when the client registers, receiving the first biometric original image slice and the feature value slice from the client and saving them.

[0008] As a supplement or alternative to the above scheme, in the above method, obtaining the biometric preprocessing slice includes: receiving a second biometric original image slice from the client; and preprocessing the second biometric original image slice to obtain the preprocessing slice.

[0009] As a supplement or replacement for the above-mentioned scheme, the above-mentioned method may also include: based on the feature value slices or the first biometric original image slices sent by the client, through the slices of the biometric value in the multi-party secure computing system, according to the operation rules and formula decomposition of the multi-party secure computing, realize the similarity comparison calculation of the biometrics.

[0010] As a supplement or replacement for the above solution, the above method may further include: when the comparison algorithm of the multi-party secure computing system is updated, calculating a new feature value based on the saved first biometric original image fragment.

[0011] As a supplement or replacement for the above scheme, in the above method, calculating the new feature value based on the saved first biometric original image fragment includes: after preprocessing the first biometric original image fragment, re-extracting the feature using the neural network.

[0012] As a supplement or replacement for the above solution, in the above method, the multi-party secure computing system is a multi-party MPC face platform.

[0013] As a supplement or alternative to the above-mentioned scheme, in the above-mentioned method, the second biometric original image slice is a facial image, and the preprocessing of the second biometric original image slice includes: based on the secret sharing method, constructing MPC multiplication and MPC addition to realize the operation of the operator / function, thereby realizing the preprocessing of the facial image.

[0014] According to another aspect of the present invention, a biometric feature extraction device for a multi-party secure computing system is provided, the device comprising: an acquisition device for acquiring a biometric feature pre-processing slice; and a feature extraction device for extracting features from the pre-processing slice using a neural network, wherein the feature extraction device is configured to obtain intermediate data by performing data calculations on the pre-processing slices in the first N convolution and pooling layers in the neural network by the multi-party server platforms in the multi-party secure computing system through multi-party secure computing, wherein the intermediate data is the processing result of the first N convolution and pooling layers, and the processing result cannot be reversibly restored to the pre-processing slice, where N is a natural number; the feature extraction device is also configured to collect the intermediate data to the single-party server platform in the multi-party secure computing system for subsequent calculations in the convolution and pooling layers after the Nth layer of the neural network, and complete the feature extraction after inputting the fully connected layer.

[0015] As a supplement or replacement for the above solution, in the above device, the feature extraction device is configured to enable each party's server platform to obtain shards of the biometric feature value respectively through multi-party secure computing in the fully connected layer.

[0016] As a supplement or alternative to the above-mentioned scheme, in the above-mentioned device, the feature extraction device is configured to realize the sharding of the biometric value by constructing a calculation operator corresponding to the party server platform in the average pooling layer and the fully connected layer, wherein the sharding of the biometric value includes a first feature vector and a second feature vector, wherein the first feature vector is stored on a first-party server platform of the party server platforms, and the second feature vector is stored on a second-party server platform of the party server platforms.

[0017] As a supplement or replacement for the above solution, the above device also includes: a receiving device, which is used to receive the first biometric original image slice and the feature value slice from the client when the client registers, and save them.

[0018] As a supplement or alternative to the above scheme, in the above device, the acquisition device includes: a receiving unit, used to receive a second biometric original image slice from a client; and a preprocessing unit, used to preprocess the second biometric original image slice to obtain the preprocessed slice.

[0019] As a supplement or replacement for the above-mentioned scheme, the above-mentioned device may also include: a comparison device, which is used to realize biometric similarity comparison calculation based on the feature value slices or the first biometric original image slices sent by the client, through the slices of the biometric value in the multi-party secure computing system, and according to the operation rules and formula decomposition of the multi-party secure computing.

[0020] As a supplement or replacement for the above solution, the above device may also include: a computing device for calculating a new feature value based on the saved first biometric original image fragment when the comparison algorithm of the multi-party secure computing system is updated.

[0021] As a supplement or alternative to the above solution, in the above device, the computing device is configured to re-extract features using the neural network after preprocessing the first biometric original image fragment.

[0022] As a supplement or replacement for the above solution, in the above device, the multi-party secure computing system is a multi-party MPC face platform.

[0023] As a supplement or alternative to the above-mentioned scheme, in the above-mentioned device, the second biometric original image is segmented into facial images, and the preprocessing unit is configured to implement the operation of the operator / function based on secret sharing by constructing MPC multiplication and MPC addition to achieve preprocessing of the facial image.

[0024] According to yet another aspect of the present invention, a computer storage medium is provided, wherein the medium includes instructions, and the instructions execute the above method when executed.

[0025] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, and when the computer program is executed by a processor, the method as described above is implemented.

[0026] In the biometric feature extraction scheme of one or more embodiments of the present invention, in feature extraction, the multi-party server platform in the multi-party secure computing system performs data calculations on the pre-processed slices in the first N convolution and pooling layers by means of multi-party secure computing to obtain intermediate data, where N is a natural number; and in the convolution and pooling layers after the Nth layer of the neural network, the intermediate data is collected to the single-party server platform in the multi-party secure computing system for subsequent calculations, thereby reducing the computational complexity and communication overhead of the backend and improving performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other objects and advantages of the present invention will become more fully apparent from the following detailed description taken in conjunction with the accompanying drawings, wherein the same or similar elements are denoted by the same reference numerals.

[0028] Figure 1 A schematic flow chart of a biometric feature extraction method for a multi-party secure computing system according to an embodiment of the present invention is shown;

[0029] Figure 2 A schematic diagram of the structure of a biometric feature extraction device for a multi-party secure computing system according to an embodiment of the present invention is shown;

[0030] Figure 3 An example of a calculation process of MPC multiplication based on secret sharing is shown; and

[0031] Figure 4 A diagram showing the architecture of a biometric feature recognition platform based on MPC technology according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] Hereinafter, a biometric feature extraction scheme for a multi-party secure computing system according to various exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0033] Figure 1 FIG. 1 is a flow chart of a biometric feature extraction method 1000 for a multi-party secure computing system according to an embodiment of the present invention. Figure 1 As shown, the biometric feature extraction method 1000 for a multi-party secure computing system includes the following steps:

[0034] In step S110, a biometric pre-processing slice is obtained; and

[0035] In step S120, feature extraction is performed on the preprocessed slice using a neural network, wherein in the first N convolution and pooling layers in the neural network, the multi-party server platform in the multi-party secure computing system performs data calculations on the preprocessed slices respectively through multi-party secure computing to obtain intermediate data, and the intermediate data is the processing result of the first N convolution and pooling layers, and the processing result cannot be reversibly restored to the preprocessed slice, where N is a natural number; in the convolution and pooling layers after the Nth layer of the neural network, the intermediate data is collected to the single-party server platform in the multi-party secure computing system for subsequent calculations, and feature extraction is completed after inputting the fully connected layer.

[0036] In the context of the present invention, the so-called "multi-party secure computing" is to solve the problem of collaborative computing to protect privacy between a group of mutually distrustful participants. It must ensure the independence of input, the correctness of calculation, decentralization and other characteristics, while not leaking the input values ​​to other members participating in the calculation. Each multi-party secure computing participating node has the same status and can initiate collaborative computing tasks or choose to participate in computing tasks initiated by other parties. Routing addressing and computing logic transmission are controlled by the hub node, which searches for relevant data and transmits computing logic at the same time. Each multi-party secure computing node completes data extraction and calculation in the local database according to the computing logic, and routes the output calculation results to the designated node, so that the multi-party nodes complete the collaborative computing task and output unique results. During the whole process, all the data of all parties are local. While ensuring data privacy, the calculation results are fed back to the entire computing task system, so that all parties get correct data feedback.

[0037] In multi-party secure computation based on secret sharing, data input and computational intermediate values ​​exist in the form of "shards". For example, private data can be cut into two or more parts, and then the random shards can be distributed to the computation participants. This process protects data privacy and allows multiple parties to jointly compute the data. After that, the homomorphic computing properties between shards can be used to compute on the shards and reconstruct the computation results of the private data.

[0038] In the context of the present invention, the term "client" is also referred to as a user terminal, and refers to a device or apparatus that provides local services to a client. In one or more embodiments, the client has some basic functions, including a collection function for collecting biometric information of the user, and a data processing function for extracting feature values ​​from the collected biometric information using an extraction algorithm pre-stored in the client. The client can be a user's smart device, including but not limited to the user's mobile phone, laptop computer, and head-mounted device.

[0039] The term "server platform" is also referred to as a backend / cloud server, and refers to a device or apparatus that provides remote services to a client, corresponding to a "client". In one or more embodiments, the server platform may be an MPC face platform.

[0040] In the context of the present invention, the term "biometric" or "biometric information" refers to any information inherent to the human body that can be used for personal identification, including but not limited to physiological characteristics (e.g., fingerprints, irises, physiognomy, DNA, etc.) and behavioral characteristics (gait, typing habits, etc.). In addition, in the context of the present invention, "biometric pre-processed slices" are obtained by pre-processing the "slices" of biometric information.

[0041] The term "feature extraction" refers to extracting or calculating a feature vector of a specific dimension from biometric information through a specific algorithm (such as an extraction algorithm), so that the distance between two feature vectors of different biometric information (such as different faces) is as large as possible.

[0042] In the context of the present invention, "intermediate data" refers to the processing results of the first N layers of convolution and pooling layers of a neural network, which cannot be reversed to the input of the neural network (i.e., preprocessing slices), which can effectively ensure the security of the data. N can be a natural number. For example, when N=1, "intermediate data" refers to the output obtained by processing the convolution layer and pooling layer of the first layer of the neural network. N can be selected according to actual needs, but no matter how it is selected, it is necessary to ensure that the processing results of the first N layers of convolution and pooling layers (i.e., "intermediate data") cannot be reversed to preprocessing slices.

[0043] In one embodiment, the neural network includes multiple convolutional layers, pooling layers, and fully connected layers, and in the calculation process of the first N convolutional layers and pooling layers, N is a natural number, and the multi-party server platform in the multi-party secure computing system performs data calculation on the pre-processing slices respectively by means of multi-party secure computing; in the calculation process of the subsequent convolutional layers and pooling layers, the intermediate data of the neural network is collected to the unilateral server platform in the multi-party secure computing system for subsequent calculations, and the intermediate data is the processing results of the first N convolutional layers and pooling layers, which are subject to irreversible recovery.

[0044] In a neural network, a convolutional layer consists of a group of convolutional units (also called "convolution kernels"). The convolution kernels are generally considered as filters. A specific feature is extracted through the filters to obtain multiple feature maps (usually deep feature maps are obtained through multiple layers of convolution). The pooling layer reduces the dimension by selecting a value to represent all pixel values ​​within a range, reducing the image size (feature compression) while retaining the features, thereby simplifying the computational complexity of the network. The fully connected layer will generally convert the two-dimensional feature map output by the convolution into a one-dimensional vector to identify the features of the image.

[0045] For example, in an embodiment where the multi-party secure computing system is a multi-party MPC face platform (note: the term "multi-party" means "two or more parties"), the multi-party MPC face platform pre-processes the slices based on the input face image, and in the convolution, pooling and other calculation processes of the first layer of the deep neural network (note: considering that the reverse restoration of the face image is not limited to the first layer, such as the first two layers / first three layers, etc.), the multi-party MPC face platform completes the data processing and calculation of the slices; in the subsequent convolution, pooling and other calculation processes, the intermediate data of the neural network is collected to one party to complete the face feature extraction process. In this way, in this embodiment, in the feature extraction link, the multi-party secure computing of the image slices is only performed in the first layer, and in the subsequent processing, the processing for one original image is formed based on the multi-party MPC platform nodes (only processed by a single party), avoiding the communication overhead and calculation merging overhead of the multi-party MPC platform for multi-party secure computing based on the image slices, which greatly reduces the processing complexity and processing efficiency of the MPC platform.

[0046] In a further embodiment, after inputting the fully connected layer, feature extraction is completed, including: in the fully connected layer, by means of multi-party secure computing, each server platform obtains the sharding of the biometric value respectively. In one embodiment, the sharding of the biometric value is realized by constructing a computing operator corresponding to the server platform of each party in the average pooling layer and the fully connected layer, wherein the sharding of the biometric value includes a first feature vector and a second feature vector, wherein the first feature vector is stored in the first server platform of the server platform of each party, and the second feature vector is stored in the second server platform of the server platform of each party. For example, in an embodiment where the multi-party secure computing system is a multi-party MPC face platform, the sharding of the face feature vector is realized based on the multi-party MPC face platform in the fully connected layer, and the face feature value sharding information of the user is saved by multiple parties (e.g., two or more parties). This enables the biometric feature extraction system based on the multi-party secure computing technology to protect biometric privacy information while performing distributed computing of image features based on multiple parties.

[0047] although Figure 1 Not shown in the figure, in one embodiment, the method 1000 may further include: when the client registers, receiving the first biometric original image slice and the feature value slice from the client, and saving them.

[0048] In one embodiment, step S110 includes: receiving a second biometric original image slice from a client; and preprocessing the second biometric original image slice to obtain the preprocessed slice. That is, in this embodiment, the client only needs to provide the biometric original image slice without preprocessing the original image slice, which reduces the processing requirements of the client. In another embodiment, when the client has relevant processing capabilities, the preprocessed biometric original image slice can also be directly received from the client.

[0049] In one embodiment, the method 1000 may further include: based on the feature value slices or the first biometric original image slices sent by the client, the biometric value slices in the multi-party secure computing system are decomposed according to the operation rules and formulas of the multi-party secure computing to realize the similarity comparison calculation of the biometrics. For example, in the face comparison stage, based on the feature value slices / face image slices sent by the client, based on the feature value slices stored in the back-end multi-party MPC face feature comparison platform, the face similarity comparison calculation is realized based on the MPC operation rules and formula decomposition.

[0050] In one embodiment, the method 1000 may further include: when the comparison algorithm of the multi-party secure computing system is updated, calculating the new feature value based on the saved first biometric original image slice. In one embodiment, calculating the new feature value based on the saved first biometric original image slice includes: re-extracting features using the neural network after preprocessing the first biometric original image slice.

[0051] For example, when the client registers, the biometric original image slices and feature value slices are uploaded and saved to the background; in the face comparison stage, based on the feature value slices / face image slices uploaded by the client and the feature value slices saved in the back-end multi-party MPC face feature comparison platform, the face similarity comparison calculation is realized based on the MPC operation rules and formula decomposition; when the background comparison algorithm is updated, the biometric original image slices saved in the multi-party secure computing platform are used to calculate the user's new feature value and upgrade the version, thereby decoupling the algorithm upgrade logic from the terminal-side business.

[0052] In one embodiment, the first or second biometric original image slice is a facial image, and preprocessing the second biometric original image slice includes: based on secret sharing, constructing MPC multiplication and MPC addition to implement operator / function operations to achieve preprocessing of the facial image.

[0053] Secret sharing is a technology for sharing secrets among a group of participants. It is mainly used to protect important information and prevent it from being lost, destroyed, or tampered with. It originates from classical cryptography theory and was first proposed by Sharmir and Blakley in 1979. In simple terms, secret sharing refers to the reasonable distribution of shared secrets among a user group so that all members can jointly manage the secrets. In the method based on Shamir's secret sharing theory, the secret sharing mechanism is mainly composed of the secret distributor D, group participants P{P1, P2, ..., Pn}, access structure, secret space, allocation algorithm, recovery algorithm and other elements.

[0054] 1. Technical Process of Secret Sharing

[0055] Objective: Assume that there is a secret S. Perform a specific operation on the secret S to obtain w secret fragments Si (0<i<=w), which are given to w people to keep. When at least t people take out their own secret fragments Si at the same time, the original secret S can be restored.

[0056] Secret generation: Construct a polynomial: F(x) = S + a1*x 1 +a2*x 2 +...+a (t-1) *x(t-1) mod(p), where S is the secret to be protected, p is a prime number, and S<p. Take w unequal x and substitute them into F(x) to obtain w groups (xi, yi). Distribute them to w people, make p public, destroy the polynomial, and each person is responsible for keeping his or her own (xi, yi) confidential.

[0057] Secret recovery: When x=0, F(0)=S, and S can be recovered; substitute the t group (xi, yi) into the following formula: Among them, the negative first power is the inverse of the term modulo p. Substituting the t group (xi, yi) into it, we can get S.

[0058] In a calculation example, suppose there are w = 4 people, set at least t = 3 people to recover the secret, secret S = 2, p = 23, construct F(x) = 2 + 3x + 2x 2 mod(23), take x1=1, x2=2, x3=3, x4=4, substitute and get y1=7, y2=16, y3=6, y4=0, use 3 groups to restore (1, 7)(3, 6)(4, 0), and calculate S=2.

[0059] S=F(0)=7*(0-3)*(0-4)*((1-3)*(1-4)) -1 +

[0060] 6*(0-1)*(0-4)*((3-1)*(3-4)) -1 +

[0061] 0*(0-1)*(0-3)*((4-1)*(4-3)) -1 mod(23)

[0062] Similarly, using the other three groups to restore (2, 16) (3, 6) (4, 0), the same calculation can yield S=2.

[0063] (II) Example of MPC addition based on secret sharing

[0064] Given: A has 100, B has 70, C has 2, goal: calculate their sum, and realize that any two parties from ABC can get the sum when they come together.

[0065] Because: Any two of ABC are solvable, then ABC can each form a 2-variable first-degree polynomial, such as A(n) = 100 + 2n, B(n) = 70 + 3n, and C(n) = 2 + 8n. Let ABC be numbered 1, 2, and 3, then A gets A(1), B(1), and C(1), which are 102, 73, and 10 respectively; B gets A(2), B(2), and C(2), which are 104, 76, and 18 respectively; C gets A(3), B(3), and C(3), which are 106, 79, and 26 respectively. Finally, A(1, 102 + 73 + 10 = 185), B(2, 104 + 76 + 18 = 198), and C(3, 106 + 79 + 26 = 211).

[0066] The decryption process (i.e. the process of calculating their sum): Since the solution can be obtained by two sides, a linear polynomial y=a0+a1*x is constructed. Then, the two sides AC, (1, 185) and (3, 211) are substituted to obtain a0=172.

[0067] It can be seen that the secret of ABC is not leaked during the entire calculation process, and the final result of A+B+C can be obtained.

[0068] (III) Example of MPC multiplication based on secret sharing

[0069] MPC multiplication under secret sharing: that is, I have a data, you have a data, and we can realize the product of the data without leaking each other's data. Figure 3 An example of the calculation process of MPC multiplication based on secret sharing is shown.

[0070] like Figure 3 As shown, because the auxiliary computing node (P2) generates three types of random numbers A, B, and C related numbers A0A1B0B1C0Cl, it is named a triple. The process is derived as follows:

[0071] 1) X*Y = (A+XA)*(B+YB);

[0072] 2) Let E = XA, F = YB, and rearrange to get: X*Y = (A+E)*(B+F) = AB+BE+AF+EF;

[0073] 3) Let C = AB, and we get: X*Y = C + BE + AF + EF. Then: let B = B0 + B1, A = A0 + A1, C = C0 + C1, and we get: X*Y = C0 + C1 + E (B0 + B1) + F (A0 + A1) + EF.

[0074] 4) Split into two parts, namely: Z0 = E*B0 + F*A0 + C0, Z1 = X*Y-Z0 = C0 + C1 + E(B0 + B1) + F(A0 + A1) + EF-C0-EB0-FA0 = C1 + EB1 + FA1 + EF. It can be seen that if P0 wants to calculate Z0 and P1 wants to calculate Z1, they need to prepare B0, A0, C0 and C1, B1, A1 respectively, and these two sets of data can be distributed by P2;

[0075] 5) E and F are E=XA, F=YB respectively, which are split into two parts E0=X0-A0, E1=X1-A1, F0=Y0-B0, F1=Y1-B1. It can be seen that P0 has E0 and F0, and P1 has E1 and F1. We only need to use P0 to generate a random number r, so that X1=Xr.

[0076] That is, to realize X*Y, you only need to prepare random numbers (A0B0C0A1B1C1) in advance. The data Z0Z1 calculated by P0 and P1 respectively is only a part of X*Y, and you can get X*Y by adding them together.

[0077] It is easy for a person skilled in the art to understand that the biometric feature extraction method 1000 for a multi-party secure computing system provided by one or more embodiments of the present invention can be implemented by a computer program. For example, the computer program is included in a computer program product, and when the computer program is executed by a processor, the biometric feature extraction method 1000 for a multi-party secure computing system according to one or more embodiments of the present invention is implemented. For another example, when a computer storage medium (such as a USB flash drive) storing the computer program is connected to a computer, the biometric feature extraction method for a multi-party secure computing system according to one or more embodiments of the present invention can be executed by running the computer program.

[0078] refer to Figure 2 , Figure 2 FIG. 2 shows a schematic diagram of the structure of a biometric feature extraction device 2000 for a multi-party secure computing system according to an embodiment of the present invention. Figure 2As shown, the biometric feature extraction device 2000 for a multi-party secure computing system includes: an acquisition device 210 and a feature extraction device 220. The acquisition device 210 is used to acquire a biometric feature pre-processing slice; and the feature extraction device 220 is used to extract features from the pre-processing slice using a neural network, wherein the feature extraction device 220 is configured to obtain intermediate data by performing data calculations on the pre-processing slices in the first N convolution and pooling layers in the neural network by the multi-party server platform in the multi-party secure computing system through multi-party secure computing, and the intermediate data is the processing result of the first N convolution and pooling layers, and the processing result cannot be reversely restored to the pre-processing slice, and N is a natural number; the feature extraction device 220 is also configured to collect the intermediate data in the convolution and pooling layers after the Nth layer of the neural network to the single-party server platform in the multi-party secure computing system for subsequent calculations, and complete feature extraction after inputting the fully connected layer.

[0079] In the context of the present invention, the so-called "multi-party secure computing" is to solve the problem of collaborative computing to protect privacy between a group of mutually distrustful participants. It must ensure the independence of input, the correctness of calculation, decentralization and other characteristics, while not leaking the input values ​​to other members participating in the calculation. Each multi-party secure computing participating node has the same status and can initiate collaborative computing tasks or choose to participate in computing tasks initiated by other parties. Routing addressing and computing logic transmission are controlled by the hub node, which searches for relevant data and transmits computing logic at the same time. Each multi-party secure computing node completes data extraction and calculation in the local database according to the computing logic, and routes the output calculation results to the designated node, so that the multi-party nodes complete the collaborative computing task and output unique results. During the whole process, all the data of all parties are local. While ensuring data privacy, the calculation results are fed back to the entire computing task system, so that all parties get correct data feedback.

[0080] In multi-party secure computation based on secret sharing, data input and computational intermediate values ​​exist in the form of "shards". For example, private data can be cut into two or more parts, and then the random shards can be distributed to the computation participants. This process protects data privacy and allows multiple parties to jointly compute the data. After that, the homomorphic computing properties between shards can be used to compute on the shards and reconstruct the computation results of the private data.

[0081] In the context of the present invention, the term "client" is also referred to as a user terminal, and refers to a device or apparatus that provides local services to a client. In one or more embodiments, the client has some basic functions, including a collection function for collecting biometric information of the user, and a data processing function for extracting feature values ​​from the collected biometric information using an extraction algorithm pre-stored in the client. The client can be a user's smart device, including but not limited to the user's mobile phone, laptop computer, and head-mounted device.

[0082] The term "server platform" is also referred to as a backend / cloud server, and refers to a device or apparatus that provides remote services to a client, corresponding to a "client". In one or more embodiments, the server platform may be an MPC face platform.

[0083] In the context of the present invention, the term "biometric" or "biometric information" refers to any information inherent to the human body that can be used for personal identification, including but not limited to physiological characteristics (e.g., fingerprints, irises, physiognomy, DNA, etc.) and behavioral characteristics (gait, typing habits, etc.). The term "feature extraction" refers to extracting or calculating a feature vector of a specific dimension from biometric information through a specific algorithm (e.g., extraction algorithm), so that the distance between two feature vectors of different biometric information (e.g., different faces) is as large as possible.

[0084] In one embodiment, the neural network includes multiple convolutional layers, pooling layers, and fully connected layers. In a neural network, a convolutional layer is composed of a group of convolutional units (also called "convolution kernels"), which are generally considered as filters. A specific feature is extracted through the filter to obtain multiple feature maps (usually deep feature maps are obtained through multi-layer convolution). The pooling layer reduces the dimension by selecting a value to represent all pixel values ​​within a range, reducing the image size (feature compression) while retaining the features, thereby simplifying the computational complexity of the network. The fully connected layer will generally convert the two-dimensional feature map output by the convolution into a one-dimensional vector to identify the features of the image.

[0085] In one embodiment, the feature extraction device 220 is configured to perform data calculations on the corresponding pre-processed slices by the multi-party server platform in the multi-party secure computing system in the calculation process of the first N convolutional layers and pooling layers, where N is a natural number, respectively, and in the subsequent calculation process of the convolutional layer and the pooling layer, it is configured to collect the intermediate data to the single-party server platform in the multi-party secure computing system for calculation. For example, in an embodiment where the multi-party secure computing system is a multi-party MPC face platform, the multi-party MPC face platform pre-processes the slices based on the input face image, and completes the data processing and calculation of the slices through the multi-party MPC face platform in the convolution, pooling and other calculation processes of the first layer of the deep neural network (Note: considering that the reverse restoration of the face image is not limited to the first layer, such as the first two layers / first three layers, etc.); in the subsequent convolution, pooling and other calculation processes, the intermediate data of the neural network is collected to one party to complete the facial feature extraction process. Thus, in this embodiment, in the feature extraction link, multi-party secure computation of image slices is performed only in the first layer. In subsequent processing, processing for an original image is formed based on the multi-party MPC platform nodes, thereby avoiding the communication overhead and computational merging overhead of the multi-party MPC platform performing multi-party secure computation based on image slices, thereby greatly reducing the processing complexity and efficiency of the MPC platform.

[0086] In a further embodiment, the feature extraction device 220 is configured to enable each server platform to obtain the sharding of the biometric value respectively by means of multi-party secure computing in the fully connected layer. In one embodiment, the feature extraction device 220 is configured to implement the sharding of the biometric value by constructing a calculation operator corresponding to the server platform of each party in the average pooling layer and the fully connected layer, wherein the sharding of the biometric value includes a first feature vector and a second feature vector, wherein the first feature vector is stored in the first server platform of the server platform of each party, and the second feature vector is stored in the second server platform of the server platform of each party. For example, the sharding of the face feature vector is implemented based on a multi-party (i.e., two or more) MPC face platform in the fully connected layer, and the face feature value sharding information of the user is saved by the multi-party. This enables the biometric feature extraction system based on the multi-party secure computing technology to protect the biometric privacy information while performing distributed computing of image features based on multiple parties.

[0087] although Figure 2 Not shown, in one embodiment, the above-mentioned device 2000 may also include: a receiving device, which is used to receive the first biometric original image fragment and the feature value fragment from the client when the client registers, and save them.

[0088] In one embodiment, the acquisition device 210 includes: a receiving unit for receiving a second biometric original image slice from a client; and a preprocessing unit for preprocessing the second biometric original image slice to obtain the preprocessed slice. That is, in this embodiment, the client only needs to provide the biometric original image slice without preprocessing the original image slice, which reduces the processing requirements of the client. In another embodiment, when the client has relevant processing capabilities, the acquisition device 210 can also directly receive the preprocessed biometric original image slice from the client.

[0089] In one embodiment, the device 2000 may further include: a comparison device, which is used to perform a similarity comparison calculation of the biometric features based on the feature value slices or the first biometric original image slices sent by the client, through the biometric value slices in the multi-party secure computing system, and according to the operation rules and formula decomposition of the multi-party secure computing. For example, in the face comparison stage, the comparison device is configured to perform a similarity comparison calculation of the face based on the feature value slices / face image slices sent by the client, based on the feature value slices stored in the back-end multi-party MPC face feature comparison platform, and based on the MPC operation rules and formula decomposition.

[0090] In one embodiment, the device 2000 may further include: a computing device for realizing the calculation of new feature values ​​based on the first biometric original image slices saved when the comparison algorithm of the multi-party secure computing system is updated. In one embodiment, the computing device may be configured to re-extract features using the neural network after preprocessing the first biometric original image slices. For example, when the client registers, the biometric original image slices and feature value slices are both saved to the background by uploading the biometric original image slices and feature value slices; in the face comparison stage, the comparison device realizes face similarity comparison calculation based on the feature value slices / face image slices sent by the client, the feature value slices saved in the back-end multi-party MPC face feature comparison platform, and the MPC operation rules and formula decomposition; when the background comparison algorithm is updated, the feature extraction device is configured to realize the calculation and version upgrade of the user's new feature value through the biometric original image slices saved in the multi-party secure computing platform, thereby decoupling the algorithm upgrade logic from the terminal side business.

[0091] In one embodiment, the first / second biometric feature original image is segmented into a face image. In this embodiment, the preprocessing unit can be configured to implement the operation of the operator / function by constructing MPC multiplication and MPC addition based on secret sharing to implement preprocessing of the face image.

[0092] The following uses face recognition as an example to specifically describe a biometric feature extraction solution for a multi-party secure computing system according to one or more embodiments of the present invention.

[0093] In the specific implementation of face recognition technology, the key process of image processing is: face image acquisition->face detection->image preprocessing->face feature extraction->face feature comparison->face recognition result. Among them, "face detection" is to accurately calibrate the position and size of the face from the input image, output the coordinates of the face circumscribed rectangle in the image, and may also include posture information such as tilt angle. "Image preprocessing" is to perform image preprocessing such as grayscale correction and noise filtering based on the face detection results, such as light compensation, grayscale transformation, histogram equalization, normalization, geometric correction, filtering and sharpening of face images. "Facial feature extraction" usually processes the face after preprocessing and face detection through a neural network to obtain a feature vector of a specific dimension (used to characterize the face), so that the distance between two feature vectors of different faces is as large as possible, and the distance between two feature vectors of the same face is as small as possible. "Facial feature matching" is to compare the facial feature data to be identified with the facial feature templates registered and saved in the database. By setting a threshold, when the similarity exceeds this threshold, the matching result is output. It can be divided into two types of face recognition: 1:1 and 1:N.

[0094] The following is a specific implementation plan for face recognition:

[0095] 1) Face image acquisition: Face images are acquired through a camera lens.

[0096] 2) Face detection: Accurately calibrate the position and size of the face in the image, and select some rectangular features that best represent the face. A typical example is the Haar classifier in OpenCV, which is an improved version of the Viola-Jones method (real-time target detection based on Haar-like features and Adaboost classifiers). Others include MTCNN, Faster-CNN, SSD, etc.

[0097] Example of using the Histogram of Directed Gradients (HOG) to detect the face position calculation process:

[0098] HOG (Histogram of Oriented Gradients) constructs features by calculating and counting the directional gradient histograms of local areas (Cells and Blocks) of the image. First, the entire image is divided into small Cell areas of equal size. For example, the image is first divided into small areas of 20pixel*20pixel, and then the gradient directional histograms of these small areas are calculated respectively; then, a certain number of small areas are combined into a slightly larger area Block, for example, 2*2 small Cell areas form a Block area, and then, the directional gradient histogram feature vectors of the Block area form the directional gradient histogram HOG features of the entire image.

[0099] For image I (x, y), calculate the gradient of the image in the horizontal and vertical directions. You can use the first-order differential operator Sobel in OpenCV to calculate the gradient image in the X and Y directions. Then calculate the gradient direction and gradient amplitude of each pixel in each small area according to the following formula (1) and formula (2).

[0100]

[0101]

[0102] The angle of the gradient direction calculated by the formula is a radian value ranging from 0 to 360 degrees. For the sake of simplicity, the range of the gradient direction is constrained to 0-180 degrees and divided into 9 directions, 20 degrees in each direction, and then the constrained angle is divided by 20, so the gradient direction angle value becomes in the range of [0, 9).

[0103] The gradient amplitude in each small area Cell is counted in 9 directions. After calculation, a directional gradient histogram will be generated with the horizontal axis X as the gradient direction and the vertical axis Y as the gradient amplitude. The directional histogram corresponding to the cell area is converted into a single-dimensional vector, and the number of corresponding directional gradients is encoded according to the specified group interval to obtain the 9 histogram HOG features of a single cell area.

[0104] 3) Image preprocessing

[0105] Image preprocessing includes image grayscale and color space transformation, normalization, denoising and enhancement, face alignment, etc., among which image grayscale and color space transformation mainly include: a) converting color images into black and white images for post-processing, Gray = 0.229R + 0.587G + 0.11B (R is the red component, G is the green component, and B is the blue component); b) in order to display the face on the LCD in real time, the YCrCb 4:2:2 format digital video data output by video decoding is switched to RGB color space data format before it can be displayed on the LCD or CRT.

[0106] The conversion formula of YCrCh 4:2:2 video data to RGB color space is as follows:

[0107] R=1.164(Y-16)+1.596(Cr-128),

[0108] G=1.164(Y-16)-0.813(Cr-128)-0.392(Cb-128),

[0109] B=1.164(Y-16)+2.017(Cb-128).

[0110] The goal of normalization is to obtain standardized face images with consistent size and the same grayscale value range. The steps of geometric normalization example are as follows: (1) Obtain feature points. For example, use the function [x, y] = ginput(3) to calibrate the three feature points of the eyes and nose, and obtain the coordinate values ​​of the three feature points; (2) Rotate the image according to the coordinate values ​​of the left and right eyes to ensure the consistency of the face direction. Let the distance between the two eyes be d, and the middle point be O; (3) Determine the rectangular feature area based on the facial feature points and the geometric model. Take O as the reference, cut d on the left and right, and take 0.5d and 1.5d rectangular areas in the vertical direction for cropping; (4) Scale the expression sub-area image to a uniform size, which is more conducive to the extraction of expression features. The captured image is standardized to a 90*100 image to achieve geometric normalization of the image.

[0111] Denoising and enhancement refers to the denoising and enhancement of noise in an image through a filtering device. Filtering devices include linear filters and nonlinear filters. Median filtering is a typical nonlinear smoothing filtering method. Gaussian smoothing filtering is the most commonly used linear smoothing filtering. The implementation process of Gaussian smoothing filtering is to first determine a neighborhood, which is a window range. The pixel is in the middle of this window, so that there is a fixed window range for each pixel. Scanning is performed on each pixel, and all pixels within the window range determined by each pixel can obtain a weighted average grayscale value, and this weighted average grayscale value is used to replace the original value of the center pixel of this window.

[0112] The input of the face alignment algorithm is "a face image" plus a "face coordinate frame", and the output is a coordinate sequence of the key points of the facial features. The number of key points of the facial features is a fixed value that is pre-set and can be defined according to different semantics (common ones are 5 points, 68 points, 90 points, etc.). For example, Dlib has a special function and model that can realize the positioning of 68 feature points of the face.

[0113] 4) Face feature extraction

[0114] The numerical string that can represent the characteristics of a person's face is called a "face feature". A neural network is trained to generate a 128-dimensional prediction value from the input face image. The general process of training is: two different photos of the same person and a photo of another person are fed into the neural network, and the training is iterated continuously to make the prediction values ​​of the two photos of the same person after encoding close, and the prediction values ​​of photos of different people are far apart. In other words, the distance within the class is reduced and the distance between classes is increased.

[0115] 5) Facial feature comparison

[0116] Input a facial feature, and compare it with the features corresponding to N identities registered in the database one by one, and find "one" feature with the highest similarity to the input feature. Compare this highest similarity value with the preset threshold. If it is greater than the threshold, return the identity corresponding to the feature, otherwise return "not in the database". There are many ways to compare, you can directly find the face with the smallest Euclidean distance within the threshold range, or train a terminal SVM or knn classifier.

[0117] Figure 4 FIG. 1 shows an architecture diagram of a biometric feature recognition platform based on MPC technology according to an embodiment of the present invention. Figure 4 As shown, when a user uses the biometric recognition function for the first time, he needs to register. In the registration process, the user's original image is collected through the mobile POS client, and the original face image (or pre-processed original image) is randomly sliced ​​(to ensure that no party can obtain the complete original face image data) while extracting and slicing the feature values ​​(for example, calculating the user's facial feature values ​​on the client, and slicing the output feature vector (such as adding, subtracting or truncating random vectors, and sending the two feature slices to the MPC platform of different institutions). Then, the feature value slices and the face original image slices are transmitted to the cloud for storing the feature values ​​and the face image data.

[0118] Optionally, during the registration process, the user's feature value is calculated only through image slicing and the MPC platform. In other words, the mobile POS client itself does not need to calculate the facial feature value, which can advantageously reduce the client's calculation requirements. For example, the client does not need to calculate multiple versions of feature value slicing, which improves the user experience. Take the two-party MPC calculation process as an example to illustrate the specific calculation process:

[0119] (1) Image input: The original face image fragment 1 and the original face image fragment 2 are input according to the user information;

[0120] (2) Image grayscale processing: According to the calculation formula for converting RGB images to grayscale images, the multi-party secure computing system implements the grayscale processing of the image by constructing MPC addition and MPC multiplication. The two parties obtain grayscale image slice 1 and grayscale image slice 2 respectively.

[0121] (3) Image face detection: According to the extraction and calculation steps of Hog face detection, gradient information is calculated, local information is counted, and the gradient map of the block composed of multiple adjacent cells is normalized to finally obtain the detection window of the image. The two MPC platforms save part of the information respectively.

[0122] (4) Normalization: Based on the two-party MPC platform, the above process is calculated, and the image is translated and rotated. The transformed image is closest to the reference face image. Translation can be done by calculating the average value and then normalizing the points; overall scaling can calculate the variance of the points and then normalize the variance; rotation can be done by an orthogonal matrix R, so that the points transformed by the R matrix are closest to the reference image; finally, the normalized face image fragments that meet the requirements of deep learning are obtained;

[0123] (5) Convolution and pooling calculation process: For the normalized face image segmentation data obtained by the two MPC platforms, the image is convolved. Taking the GoogleNet network as an example, starting from the upper left corner of the image input, the number of rows and columns of each slide is called the stride; at the same time, at the boundary position, padding is used to increase the size of the matrix, allowing the convolution kernel to exceed the boundary of the original image and making the size of the convolution result consistent with the original. The image processing calculation process of the GoogleNet network (including the addition, multiplication and function operation process in the convolution and pooling layers) is completed through the addition, multiplication and other calculation formulas constructed by the two MPC platforms. In the subsequent convolution and pooling calculations, the intermediate calculation results are collected to one of the two MPC platforms to complete the calculation of the convolution and pooling process.

[0124] For example, in convolution layer 1 (conv1), the image input size is (224, 224, 3), ReLU is used as the activation function (other models also use sigmoid activation function, etc.), after the first layer of 7*7 convolution kernel, convolution step size is 2, 64 convolution kernels with border filling, convolution with the input image ((224-7+3*2) / 2+1=112.5 (rounded down)=112), the output feature map is 112*112*64. The output 112*112*64 obtained by the ReLU activation function passes through the second layer of 3*3 convolution kernel, convolution step size is 2 convolution kernel (((112-3) / 2)+1=55.5 (rounded up)=56), the output feature map is 56*56*64. Then, the output feature map (56*56*64) of the convolutional layer 1 (conv1) is obtained after local response normalization (LRN) processing (Local Response Normalization) in the channel direction.

[0125] In convolution layer 2 (conv2), 64 1x1 convolution kernels (dimensionality reduction before 3x3 convolution kernel) are used to convert the input feature map to 56*56*64, and then the ReLU operation is performed. Then the convolution operation is performed with a convolution kernel size of 3*3, a step size of 1, a padding of 1, and an output channel number of 192. The output feature map size is (56-3+1*2) / 1+1=56, and the output feature map dimension is 56x56x192, and then the ReLU operation is performed. The pooling layer window size is 3*3, the step size is 1, the output channel number is 192, and the output feature map dimension is 56*56*192.

[0126] In convolution layer 3 (conv3), a 3*3 convolution kernel with a stride of 2 is first used. After the operation, the feature map output is 28*28*192. Using 64 1*1 convolution kernels, the feature map output is 28*28*64, and then the RuLU operation is performed (result 1). After 96 1*1 convolution kernels are operated, the feature map output is 28*28*96, ReLU calculation is performed, and then 128 3*3 convolutions are performed, and the output is 28*28*128 (result 2). 16 1*1 convolution kernels turn the feature map into 28*28*16, ReLU calculation is performed, and then 32 5*5 convolutions are performed, and the output is 28*28*32 (result 3). In the pooling layer, a 3*3 kernel is used, the output is 28*28*192, and then 32 1*1 convolutions are performed, and the output is 28*28*32 (result 4). The four results are connected / concatenated (DepthConcat), and the third dimension of the four parts of the output results is connected in parallel, that is, 64+128+32+32=256, and the final output is 28*28*256.

[0127] The subsequent fourth layer (4a, 4b, 4c, 4d, 4e), fifth layer (5a, 5b) ... of GoogleNet are similar to the calculation process of 3a and 3b and will not be repeated.

[0128] (6) Fully connected layer: In the neural network structure, after the feature extraction layer extracts features, it will be connected to a fully connected layer. Each neuron in this layer is interconnected with all neurons in the previous layer, flattening the convolution output into a one-dimensional vector. In GoogleNet, the network finally uses Average Pooling to replace the fully connected layer. At the same time, by adding a fully connected layer, the output feature value can be flexibly adjusted. In the calculation of the fully connected layer, by constructing two-party MPC platform calculation operators in the average pooling layer and the fully connected layer, the feature vectors after full connection are obtained by the two MPC platforms respectively.

[0129] (7) Output layer: After the calculation of the fully connected layer, the final output feature vector is divided into feature value slices (feature vector 1 and feature vector 2) by the two MPC parties respectively. The final output result feature vector 1 is saved by MPC platform 1, and feature vector 2 is saved by MPC platform 2.

[0130] In one embodiment, when the user performs payment authentication, the mobile POS client collects the original image of the biometric features and extracts the eigenvalues, and transmits them to the cloud after random slicing. The MPC platform on the cloud compares the user features with the feature library in a 1:1 or 1:N ratio through the eigenvalue comparison algorithm, and confirms through the comparison results that the compared eigenvalues ​​and the registered eigenvalues ​​are within the allowed threshold range. In an optional embodiment, in the comparison link, the mobile POS client can still slice the image and transmit the image slices to the cloud for feature vector calculation and feature vector comparison (completed within the MPC platform).

[0131] In one embodiment, when an algorithm update is required, no operation is required from the client. Since the user has provided the sliced ​​data of the original face image during the registration process, when the algorithm is upgraded, the face image can be preprocessed based on the image slices saved in the cloud, so that the preprocessed data is consistent with the data for feature extraction of the original face image. Then, feature extraction is performed based on the MPC platform according to the standard face feature extraction method. The final result is randomly sliced ​​into feature vector 1 and feature vector 2, and saved to the face feature libraries of different institutions respectively. Using this method, batch feature updates can be implemented in the back-end without substantially changing the business process and interaction process of the mobile POS client, without user perception. At the same time, since there is no coupling between the feature update and the comparison logic, it will not affect the normal operation of the previous business logic, and the complexity of the transformation is low.

[0132] In summary, the biometric feature extraction scheme of one or more embodiments of the present application is targeted at a multi-party secure computing platform. When the algorithm is upgraded, it is only necessary to slice two / multiple biometric feature original images and perform detection, preprocessing and other processes. In the feature extraction stage, multi-party secure computing of image slices is performed only in the first layer or the first n layers. In subsequent processing, the processing for one original image is formed based on the multi-party MPC platform nodes, avoiding the communication overhead and computing merging overhead of two / multiple MPC platforms performing multi-party secure computing based on image slices, greatly reducing the processing complexity and efficiency of the MPC platform. This enables the biometric feature recognition system based on multi-party secure computing technology to protect biometric privacy information while reducing the network communication overhead and computing logic complexity of the MPC platform, and realizes the commercial availability and ease of use of algorithm upgrades based on the distributed computing of image features by two / multiple parties.

[0133] Although the above specification only describes some of the embodiments of the present invention, it should be understood by those skilled in the art that the present invention can be implemented in many other forms without departing from its subject matter and scope. For example, on the client side, each time a new comparison request is made, it is possible to submit only feature value fragments, only biometric original image (e.g., face image) fragments, or both, so as to be compatible with the needs and security requirements of different application scenarios. Correspondingly, on the server side, corresponding processing logic can also be given to different request methods.

[0134] Therefore, the shown examples and embodiments are to be regarded as illustrative rather than restrictive, and the present invention may be capable of various modifications and substitutions without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A biometric feature extraction method for a multi-party secure computing system, characterized in that: The method comprises: Obtaining biometric preprocessing slices; and The pre-processed slices are subjected to feature extraction using a neural network, wherein in the first N convolution layers and pooling layers in the neural network, the multi-party server platform in the multi-party secure computing system respectively performs data calculations on the pre-processed slices by means of multi-party secure computing to obtain intermediate data, the intermediate data being the processing results of the first N convolution layers and pooling layers, and the processing results cannot be reversely restored to the pre-processed slices, and N is a natural number; In the convolution and pooling layers after the Nth layer of the neural network, the intermediate data is collected to the single-party server platform in the multi-party secure computing system for subsequent calculations, and feature extraction is completed after inputting the fully connected layer.

2. The method of claim 1, wherein: Completing feature extraction after inputting the fully connected layer includes: enabling each party's server platform to obtain slices of the biometric feature value respectively through multi-party secure computing in the fully connected layer.

3. The method of claim 2, wherein: The sharding of the biometric value is achieved by constructing computing operators corresponding to the various server platforms in the average pooling layer and the fully connected layer, wherein the sharding of the biometric value includes a first feature vector and a second feature vector, wherein the first feature vector is stored on a first-party server platform of the various server platforms, and the second feature vector is stored on a second-party server platform of the various server platforms.

4. The method of claim 1, further comprising: When the client registers, the first biometric original image slice and the feature value slice are received from the client and saved.

5. The method of claim 1, wherein: Obtaining biometric preprocessing slices includes: Receive a second biometric original image fragment from the client; and The second biometric original image slice is preprocessed to obtain the preprocessed slice.

6. The method of claim 2, further comprising: Based on the feature value fragments or the first biometric original image fragments sent by the client, the biometric similarity comparison calculation is realized through the fragments of the biometric value in the multi-party secure computing system according to the operation rules and formula decomposition of the multi-party secure computing.

7. The method of claim 4, further comprising: When the comparison algorithm of the multi-party secure computing system is updated, calculation of new feature values ​​is implemented based on the saved first biometric feature original image fragments.

8. The method of claim 7, wherein: The calculation of the new feature value based on the saved first biometric feature original image fragment includes: After preprocessing the first biometric original image fragments, the neural network is used to re-extract features.

9. The method of claim 1, wherein: The multi-party secure computing system is a multi-party MPC face platform.

10. The method of claim 5, wherein: The second biometric feature original image slice is a face image, and preprocessing the second biometric feature original image slice includes: Based on the secret sharing method, the operation of the operator / function is realized by constructing MPC multiplication and MPC addition, so as to realize the preprocessing of the face image.

11. A biometric feature extraction device for a multi-party secure computing system, characterized in that: The device comprises: An acquisition device, used to acquire a biometric pre-processing slice; and A feature extraction device, used to extract features from the pre-processed slices using a neural network, wherein the feature extraction device is configured to obtain intermediate data by performing data calculations on the pre-processed slices in the first N convolution layers and pooling layers in the neural network by the multi-party server platform in the multi-party secure computing system through multi-party secure computing, wherein the intermediate data is the processing result of the first N convolution layers and pooling layers, and the processing result cannot be reversely restored to the pre-processed slices, and N is a natural number; The feature extraction device is also configured to collect the intermediate data to a single-party server platform in the multi-party secure computing system for subsequent calculations in the convolution and pooling layers after the Nth layer of the neural network, and complete feature extraction after inputting the fully connected layer.

12. The device of claim 11, wherein: The feature extraction device is configured to enable each server platform to obtain slices of the biometric feature value respectively through multi-party secure computing in the fully connected layer.

13. The device of claim 12, wherein: The feature extraction device is configured to implement slicing of the biometric value by constructing calculation operators corresponding to the party server platforms in the average pooling layer and the fully connected layer, wherein the slicing of the biometric value includes a first feature vector and a second feature vector, wherein the first feature vector is stored on a first party server platform of the party server platforms, and the second feature vector is stored on a second party server platform of the party server platforms.

14. The apparatus of claim 11, further comprising: The receiving device is used to receive and save the first biometric original image fragment and the feature value fragment from the client when the client registers.

15. The apparatus of claim 11, wherein: The acquisition device comprises: A receiving unit, configured to receive a second biometric original image fragment from a client; and The preprocessing unit is used to preprocess the second biometric original image slice to obtain the preprocessed slice.

16. The apparatus of claim 12, further comprising: The comparison device is used to realize the similarity comparison calculation of the biometrics based on the feature value slices or the first biometric original image slices sent by the client, through the slices of the biometric value in the multi-party secure computing system, according to the operation rules and formula decomposition of the multi-party secure computing.

17. The apparatus of claim 14, further comprising: The computing device is used to calculate the new feature value based on the saved first biometric feature original image fragment when the comparison algorithm of the multi-party secure computing system is updated.

18. The apparatus of claim 17, wherein: The computing device is configured to re-extract features using the neural network after pre-processing the first biometric original image slice.

19. The apparatus of claim 11, wherein: The multi-party secure computing system is a multi-party MPC face platform.

20. The apparatus of claim 15, wherein: The second biometric original image is segmented into facial images, and the preprocessing unit is configured to implement operator / function operations based on secret sharing by constructing MPC multiplication and MPC addition to achieve preprocessing of the facial images.

21. A computer storage medium, characterized in that The medium includes instructions which, when executed, perform the method of any one of claims 1 to 10.

22. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images