Feature comparison method and apparatus, electronic device, and computer storage medium

Abstract

The embodiment of the present specification provides a feature comparison method and device, electronic equipment and computer storage medium, the method comprises: obtaining the target query feature by carrying out noise processing and encryption processing on the target query feature in the trusted execution environment to obtain the target ciphertext noise feature, transmitting the target ciphertext noise feature to the general execution environment, obtaining the first preset number of candidate ciphertext noise features from the left-over ciphertext noise features according to the similarity between the left-over ciphertext noise features and the target ciphertext noise features in the feature database, obtaining the original ciphertext feature corresponding to the candidate ciphertext noise feature from the feature database, sending the original ciphertext feature to the trusted execution environment, decrypting the original ciphertext feature to obtain the left-over feature, and then comparing the left-over feature with the target query feature to obtain the comparison result of the target query feature.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to feature comparison methods, apparatus, electronic devices and computer storage media. Background Technology

[0002] With the widespread adoption of biometric technologies such as facial recognition payment, the security of user privacy information is also facing threats. In facial recognition payment and identity verification scenarios, to ensure the speed of facial recognition, it is usually necessary to distribute the facial feature database locally for local comparison to improve the speed. However, there is a risk of feature theft when it is calculated locally.

[0003] Therefore, there is an urgent need to propose a feature comparison method that can effectively protect data security and improve the level of security. Summary of the Invention

[0004] The main purpose of this specification is to provide a feature comparison method, apparatus, electronic device, and computer storage medium, aiming to provide a highly efficient and reliable feature comparison method. The technical solution is as follows:

[0005] Firstly, embodiments of this specification provide a feature comparison method, including:

[0006] In a trusted execution environment, a target dense-state denoised feature is obtained, and the target dense-state denoised feature is transmitted to a general execution environment. The target dense-state denoised feature is obtained by adding noise and encrypting the target query feature obtained in the trusted execution environment.

[0007] In the general execution environment, based on the similarity between the residual dense noise feature in the feature database and the target dense noise feature, a first preset number of candidate dense noise features are obtained from the residual dense noise features. The similarity between the candidate dense noise features and the target dense noise feature is greater than that of the other residual dense noise features in the residual dense noise features excluding the candidate dense noise features.

[0008] Obtain the original dense state feature corresponding to the candidate dense state noise-adding feature from the feature database, and send the original dense state feature to the trusted execution environment;

[0009] The original dense state features are decrypted to obtain the residual features;

[0010] The baseline feature is compared with the target query feature to obtain the comparison result of the target query feature.

[0011] Secondly, embodiments of this specification provide a feature comparison device, comprising:

[0012] The first acquisition module is used to acquire target dense-state denoised features in a trusted execution environment and transmit the target dense-state denoised features to a general execution environment. The target dense-state denoised features are obtained by adding noise and encryption to the target query features acquired in the trusted execution environment.

[0013] The first retrieval module is used to obtain a first preset number of candidate dense noise features from the base-retained dense noise features in the general execution environment based on the similarity between the base-retained dense noise features in the feature database and the target dense noise features. The similarity between the candidate dense noise features and the target dense noise features is greater than that of the other base-retained dense noise features in the base-retained dense noise features excluding the candidate dense noise features.

[0014] The second acquisition module is used to acquire the original dense state feature corresponding to the candidate dense state noise feature from the feature database, and send the original dense state feature to the trusted execution environment;

[0015] The decryption module is used to decrypt the original encrypted features to obtain the retained features;

[0016] The second retrieval module is used to compare the retained features with the target query features to obtain the comparison result of the target query features.

[0017] Thirdly, embodiments of this specification provide an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described above.

[0018] Fourthly, embodiments of this specification provide a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described above.

[0019] Fifthly, embodiments of this specification provide a computer program product, including: a computer program that, when executed by a processor of an electronic device, enables the processor to at least implement the feature comparison method as described in the first and second aspects.

[0020] In the embodiments of this specification, target encrypted features are obtained by adding noise and encrypting the acquired target query features in a trusted execution environment. The target encrypted features are then transmitted to a general execution environment. In the general execution environment, based on the similarity between the target encrypted features and the retained encrypted features in the feature database, a first preset number of candidate encrypted features are obtained from the retained encrypted features. The similarity between the candidate encrypted features and the target encrypted features is greater than that of the remaining retained encrypted features. The original encrypted features corresponding to the candidate encrypted features are obtained from the feature database. The original encrypted features are sent to the trusted execution environment for decryption to obtain retained features. The retained features are then compared with the target query features to obtain the comparison result of the target query features. By distributing the feature comparison process across trusted execution environments and general execution environments, encrypted feature data is initially retrieved in the general execution environment to obtain a first preset number of candidate encrypted and noisy features. Then, the corresponding original encrypted features in the database are sent to the trusted execution environment for decryption and feature comparison. This not only enables the processing of a large number of features, but also ensures the security of the features by decrypting the original encrypted features only in the trusted execution environment throughout the entire process. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of a scenario for a feature comparison method provided in the embodiments of this specification;

[0023] Figure 2 This is a flowchart illustrating a feature comparison method provided in an embodiment of this specification;

[0024] Figure 3 This is a detailed flowchart illustrating a feature comparison method provided in the embodiments of this specification;

[0025] Figure 4 This is a detailed flowchart illustrating a feature comparison method provided in the embodiments of this specification;

[0026] Figure 5 This is a detailed flowchart illustrating a feature comparison method provided in the embodiments of this specification;

[0027] Figure 6This is a detailed flowchart illustrating a feature comparison method provided in the embodiments of this specification;

[0028] Figure 7 This is a detailed flowchart illustrating a feature comparison method provided in the embodiments of this specification;

[0029] Figure 8 This is a detailed flowchart illustrating a feature comparison method provided in the embodiments of this specification;

[0030] Figure 9 This is a flowchart of a feature comparison method provided in the embodiments of this specification;

[0031] Figure 10 This is a schematic diagram of the structure of a feature comparison device provided in the embodiments of this specification;

[0032] Figure 11 This is a schematic diagram of the structure of a feature comparison device provided in the embodiments of this specification. Detailed Implementation

[0033] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0034] Furthermore, it should be noted that the acquisition, storage, use, and processing of data involved in biometric identification and facial recognition in the embodiments of this specification all comply with the relevant provisions of national laws and regulations.

[0035] The feature comparison device can be a terminal device such as a mobile phone, computer, tablet, smartwatch, or in-vehicle device, or it can be a module in the terminal device used to implement the feature comparison method. The feature comparison device can perform noise addition and encryption processing on the acquired target query features in a trusted execution environment to obtain target encrypted noise features. The target encrypted noise features are then transmitted to a general execution environment. In the general execution environment, based on the similarity between the retained encrypted noise features in the feature database and the target encrypted noise features, a first preset number of candidate encrypted noise features are obtained from the retained encrypted noise features. The similarity between the candidate encrypted noise features and the target encrypted noise features is greater than that of the remaining retained encrypted noise features other than the candidate encrypted noise features. The original encrypted features corresponding to the candidate encrypted noise features are obtained from the feature database. The original encrypted features are sent to the trusted execution environment for decryption processing to obtain retained features. The retained features are then compared with the target query features to obtain the comparison result of the target query features.

[0036] Please see also Figure 1 This document provides a schematic diagram illustrating a feature comparison method in an embodiment. After obtaining target query features from a terminal, the feature comparison device encrypts and adds noise to these features in a trusted execution environment to obtain target encrypted and noisy features. These encrypted and noisy features are then sent to a general execution environment for preliminary retrieval to obtain a first preset number of candidate encrypted and noisy features in the feature database. These first preset number of candidate encrypted and noisy features are then sent to the trusted execution environment for decryption and feature comparison, thereby obtaining the comparison result of the target query features and sending it back to the terminal. It should be noted that the terminal can also be used as a feature comparison device; the terminal can collect target query features and perform comparisons.

[0037] The feature comparison method provided in this specification will be described in detail below with reference to specific embodiments.

[0038] Please see Figure 2 This is a flowchart illustrating a feature comparison method provided in an embodiment of this specification. Figure 2 As shown, the method described in the embodiments of this specification may include the following steps S102-S110.

[0039] S102, Obtain the target dense-state noise-added features in the trusted execution environment, and transmit the target dense-state noise-added features to the general execution environment. The target dense-state noise-added features are obtained by adding noise and encrypting the target query features obtained in the trusted execution environment.

[0040] S104, in the general execution environment, based on the similarity between the residual dense noise feature in the feature database and the target dense noise feature, a first preset number of candidate dense noise features are obtained from the residual dense noise features. The similarity between the candidate dense noise feature and the target dense noise feature is greater than that of the other residual dense noise features in the residual dense noise features excluding the candidate dense noise features.

[0041] S106, Obtain the original dense state feature corresponding to the candidate dense state noise feature from the feature database, and send the original dense state feature to the trusted execution environment;

[0042] S108, the original encrypted features are decrypted to obtain the retained features;

[0043] S110, compare the retained features with the target query features to obtain the comparison result of the target query features.

[0044] This invention uses a face recognition scenario as an example to illustrate the feature comparison method. Of course, the feature comparison method of this invention can also be used in other application scenarios that require feature security. It is understood that in the face recognition process, the user's biometric features collected by the client need to be compared with the features stored in the server's feature library. The features in the feature library are usually encrypted and stored, and then decrypted on the client side after being sent to it. However, during the decryption calculation, the plaintext is stored in memory, posing a risk of theft. Operating systems are typically divided into Trusted Execution Environments (TEEs) and Rich Execution Environments (REEs). A TEE can provide a secure area within a connected device, ensuring that sensitive data is stored, processed, and protected in an isolated and trusted environment. End-to-end security is achieved by providing isolated, securely executed authorized software. Although loading all features into the TEE for feature comparison can also achieve feature security protection, in actual business scenarios, the feature library is large, and the TEE memory is limited, making it impossible to load all features. In other words, for feature comparison processes with large amounts of data, it is impossible to implement everything within the TEE. Therefore, the embodiments of this specification provide a feature comparison method that can protect data privacy and has high security.

[0045] The following will provide a detailed explanation of each step:

[0046] S102, Obtain the target dense-state noise-added features in the trusted execution environment, and transmit the target dense-state noise-added features to the general execution environment. The target dense-state noise-added features are obtained by adding noise and encrypting the target query features obtained in the trusted execution environment.

[0047] Specifically, the target query feature is the feature information that needs to be compared. In this embodiment, it can be the biometrics of the target object. Understandably, since the target query feature is the target object's private information, it needs to be stored and processed in a trusted execution environment. For example, facial video and images of the target object are acquired through a camera, and then the target query feature is obtained by extracting features using a facial feature extraction model in the trusted execution environment. After obtaining the target query feature, considering its security, it undergoes noise addition and encryption processing to generate a target encrypted noisy feature. This encrypted noisy feature is then transmitted to a general execution environment for matching. Because the general execution environment has lower security than the trusted execution environment, the encrypted noisy feature is at risk of being stolen. However, in the general execution environment, matching is not performed directly based on the target query feature, but rather based first on the encrypted and noisy encrypted noisy feature. Even if the encrypted noisy feature is stolen in the general execution environment, it cannot be used to decipher the target query feature, thus protecting the user's privacy.

[0048] S104, in the general execution environment, based on the similarity between the residual dense noise feature in the feature database and the target dense noise feature, a first preset number of candidate dense noise features are obtained from the residual dense noise features. The similarity between the candidate dense noise feature and the target dense noise feature is greater than that of the other residual dense noise features in the residual dense noise features excluding the candidate dense noise features.

[0049] Specifically, when the client of the face recognition machine starts the face recognition function or starts the face recognition task, the server detects that the client has started and can work normally. Then, it sends the feature data required by the face recognition machine (backed-up dense-state noisy features and original dense-state features) to the client and stores them in the feature database of the general execution environment.

[0050] For example, if the feature database contains 10,000 data entries, the process of comparing the target's noisy, dense features involves a large amount of data that cannot be stored in the TEE's memory. Storing it in the REE would also be insecure, as decrypting the original dense features within the REE would be risky. Therefore, the features are noisy and placed in the REE for initial comparison, and the target's noisy, dense features can guarantee recall. The encryption and noisy processing methods used for the retained noisy, dense features in the feature database are the same as those used for the target query features in the trusted execution environment. Therefore, by calculating the similarity between the retained noisy, dense features and the target noisy, dense features, a first predetermined number of candidate noisy, dense features similar to the target noisy, dense features can be obtained from the retained noisy features.

[0051] For example, setting the first preset number to 200, the TOPK algorithm retrieves the top 200 candidate dense-state noisy features with the highest similarity from the retained dense-state loaded features in the feature database. These candidate dense-state noisy features do not need to be returned in sorted order. After obtaining the candidate dense-state noisy features, although it's unknown which retained dense-state noisy feature is the TOP1, it's guaranteed that TOP1 is within TOPK. Therefore, the subsequent feature comparison range in the TEE is TOPK, narrowing the search scope and ensuring that matching requirements are met within the TEE's memory limit. The first preset number can be set according to actual needs and is not specifically limited.

[0052] S106, Obtain the original dense state feature corresponding to the candidate dense state noise feature from the feature database, and send the original dense state feature to the trusted execution environment;

[0053] Specifically, in the feature database, the backed-up, noisy, encrypted features and the original encrypted features are stored in a one-to-one correspondence. The backed-up, noisy, encrypted features are obtained by encrypting and adding noise to the backed-up features, and the original encrypted features are obtained by encrypting the backed-up features. Understandably, feature data in the feature database exists in encrypted form, and the decryption key is not stored in the general execution environment. Therefore, even if stolen, the backed-up features cannot be deciphered. After the backed-up, noisy, encrypted features are coarsely sorted in the general execution environment to obtain candidate encrypted features, their corresponding original encrypted features are retrieved from the database and transmitted to the trusted execution environment for further feature decryption and comparison.

[0054] S108, the original encrypted features are decrypted to obtain the retained features;

[0055] Specifically, after obtaining the original encrypted features, they are decrypted in the trusted execution environment. When the server sends feature data to the feature database in the general execution environment, it also sends a key to the trusted execution environment for decrypting the original encrypted features. Therefore, by decrypting the original encrypted features corresponding to the candidate encrypted noisy features using the key, the corresponding retained features are obtained.

[0056] S110, compare the retained features with the target query features to obtain the comparison result of the target query features.

[0057] Specifically, based on the decrypted baseline features, a feature comparison is performed with the target query features. In one feasible implementation, the similarity between the baseline features and the target query features is calculated, and the baseline feature with the highest similarity is used as the matching feature for the target query feature, which is then used as the comparison result for the target query feature. After obtaining the comparison result for the target query feature, the user ID corresponding to the baseline feature can be obtained, and the recognition result of this face recognition task can be output, such as face recognition passing, or invoking a face-scanning payment program to complete the payment, etc.

[0058] In the embodiments of this specification, target encrypted features are obtained by adding noise and encrypting the acquired target query features in a trusted execution environment. The target encrypted features are then transmitted to a general execution environment. In the general execution environment, based on the similarity between the target encrypted features and the retained encrypted features in the feature database, a first preset number of candidate encrypted features are obtained from the retained encrypted features. The similarity between the candidate encrypted features and the target encrypted features is greater than that of the remaining retained encrypted features. The original encrypted features corresponding to the candidate encrypted features are obtained from the feature database. The original encrypted features are sent to the trusted execution environment for decryption to obtain retained features. The retained features are then compared with the target query features to obtain the comparison result of the target query features. By distributing the feature comparison process across trusted execution environments and general execution environments, the encrypted and noisy target encrypted features are initially retrieved in the general execution environment to obtain a first preset number of candidate encrypted features. Then, the original encrypted features corresponding to the candidate encrypted features in the feature database are obtained and sent to the trusted execution environment for decryption and feature comparison. This not only enables the processing of a large number of features, but also ensures the security of the features by decrypting the original encrypted features only in the trusted execution environment throughout the entire process.

[0059] Please see Figure 3 This document provides a detailed flowchart of a feature comparison method for embodiments of this specification. Figure 3As shown, the method described in the embodiments of this specification may include the following steps S202-S206.

[0060] S202, In a trusted execution environment, the target query features are denoised using a differential privacy algorithm to obtain denoised target features;

[0061] In one or more embodiments, in order not to disclose the privacy of any user, feature noise is added in the TEE through differential privacy, that is, interference data (i.e. noise) is added to the target query feature, such as adding Laplace noise, to obtain the target noisy feature.

[0062] S204, perform binary conversion on the target noise-added features to obtain binary features;

[0063] Specifically, the target noise-adding features are numerical values. To simplify calculations, these features are processed into binary features. For example, the `clip` function can be used. The `clip` function is used to control the elements in an array within a given range. Given the upper and lower boundaries (0,1) of the range to be controlled, the `clip` function changes all values ​​less than the lower boundary to the lower boundary and all values ​​greater than the upper boundary to the upper boundary. Understandably, other methods can also be used for binary conversion.

[0064] S206, perform encryption calculation on the binary feature to obtain the target dense-state noise-added feature.

[0065] Specifically, the binary feature is further encrypted. In the TEE (Transparent Environment), a 128-bit key is used to XOR the binary feature to obtain the target encrypted noisy feature. For example, using AES encryption, let the AES encryption function be E, then C = E(K, P), where P is the plaintext, K is the key, and C is the ciphertext. That is, by using the plaintext P and the key K as input parameters to the encryption function, the encryption function E will output the ciphertext C. In this embodiment, the plaintext P is the binary feature, and the ciphertext is the target encrypted noisy feature. By using a reversible feature transformation technique, the original feature template is transformed by an irreversible function before being stored. This ensures that the transformed template cannot be recovered from the original template, guaranteeing the security of the feature information.

[0066] Please see Figure 4 This document provides a detailed flowchart of a feature comparison method for embodiments of this specification. Figure 4 As shown, in one embodiment, the step of obtaining a first preset number of candidate dense-state denoising features from the retained dense-state denoising features based on the similarity between the retained dense-state denoising features in the feature database and the target dense-state denoising features includes:

[0067] S302, calculate the Hamming distance between the base-dense noisy feature in the feature database and the target dense noisy feature, wherein the Hamming distance is used to characterize the similarity between the base-dense noisy feature and the target dense noisy feature;

[0068] S304, obtain a first preset number of candidate dense-state noise features from the base-dense noise features according to the Hamming distance, wherein the Hamming distance between the candidate dense-state noise features and the target dense-state noise features is less than the remaining base-dense noise features other than the candidate dense-state noise features.

[0069] Specifically, Hamming distance represents the number of distinct characters at the same position in two strings of the same length. Let d(x,y) represent the Hamming distance between strings x and y. Hamming distance can be used to calculate the similarity between two texts, judging their similarity based on the number of distinct characters. For example, the distance between (00) and (01) is 1, and the distance between (110) and (101) is 2; the smaller the Hamming distance, the more similar they are. By calculating the Hamming distance between the base dense-state noisy feature and the target dense-state noisy feature, feature similarity is represented based on the Hamming distance. Then, based on the magnitude of the Hamming distance, the TOPK algorithm selects the K (first preset number) candidate dense-state noisy features with the smallest Hamming distance.

[0070] Please see Figure 5 This document provides a detailed flowchart of a feature comparison method for embodiments of this specification. Figure 5 As shown, in one embodiment, obtaining the original dense-state feature corresponding to the candidate dense-state noisy feature from the feature database and sending the original dense-state feature to the trusted execution environment includes:

[0071] S402, Obtain the feature identifier corresponding to the candidate dense-state noise-adding feature;

[0072] S404, using the feature identifier as an index, retrieve the original dense state feature corresponding to the candidate dense state noise feature from the feature database.

[0073] Understandably, when obtaining the original dense features corresponding to the candidate dense-state noisy features, it is faster to perform comparison queries after establishing an index compared to the method of directly comparing features.

[0074] Specifically, the implementation involves building a binary tree. Each candidate dense-state noisy feature corresponds to a user ID, i.e., a feature identifier. Each ID represents a node, and each node manages a dataset containing: retained dense-state noisy features and original dense-state features. Once the binary tree is built, during retrieval in the REE, a coarse ranking is performed based on the retained dense-state noisy features. After generating the TOPK candidate dense-state loaded features, the IDs of the TOPK are obtained, and the original dense-state features corresponding to the K candidate dense-state loaded features are found based on these IDs and sent to the TEE. Optionally, a tree can be built within the TEE for these TOPK original dense-state features to facilitate retrieval and matching within the TEE.

[0075] Please see Figure 6 This document provides a detailed flowchart of a feature comparison method for embodiments of this specification. Figure 6 As shown, further, in one embodiment, obtaining the target query features in a trusted execution environment includes:

[0076] S502, acquire the target image of the target object through a preset acquisition device;

[0077] Specifically, in facial recognition scenarios, target query features are extracted based on the target image of the target object. When a user makes a facial recognition payment or verifies their identity in a physical store, the camera on the preset acquisition device will capture one or more target images of the target object, or capture a video of the target object and extract the target image from the video.

[0078] S504, in the general execution environment, the first convolutional network based on the pre-trained image recognition model performs feature processing on the target image to obtain the first feature;

[0079] S506, in the trusted execution environment, the first feature is processed by the second convolutional network based on the pre-trained image recognition model to obtain the target query feature.

[0080] Specifically, after acquiring the target image, features need to be extracted using an image recognition model. However, directly placing the image recognition model in memory (REE) is not very secure. But placing the entire model in TEE would consume too much memory, making it difficult to implement. Therefore, the model is split and deployed, with some functions deployed in the TEE and others in the REE. Understandably, image recognition models typically consist of multiple convolutional layers or networks. Therefore, some convolutional networks closer to the input can be placed in the REE, while those closer to the output can be placed in the TEE. For example, if the image recognition model's function is to transform a 12x12 image of the acquired user into a 256-dimensional vector, then the first convolutional network of the pre-trained image recognition model performs feature processing on the target image to obtain a first feature. This first feature can have over 1000 dimensions. After inputting the first feature into the second convolutional network of the pre-trained image recognition model, the output is a 256-dimensional target query feature.

[0081] Please see Figure 7 This document provides a detailed flowchart of a feature comparison method for embodiments of this specification. Figure 7 As shown, further, in one embodiment, before obtaining a first preset number of candidate dense-state denoising features from the retained dense-state denoising features based on the similarity between the retained dense-state denoising features in the feature database and the target dense-state denoising features, the method further includes:

[0082] S602, confirm the preset coverage area of ​​the preset acquisition device;

[0083] Specifically, when the server sends feature data to the client of the preset collection device, the coverage area can be determined based on the geographical location of the preset collection device. The coverage area can be set manually or determined based on the location of other preset collection devices. For example, if there are three office buildings A, B, and C in an office area, and each building has a preset collection device, then the coverage area of ​​preset collection device A can be the users who frequently visit that building. If only building A has a preset collection device, then the coverage area can be the users who frequently visit the office area.

[0084] S604, obtain the retained dense state noise-added features and the original dense state features of registered users within the preset coverage area, and store the retained dense state noise-added features and the original dense state features into the feature database of the general execution environment.

[0085] Specifically, once the preset coverage area is confirmed, meaning we know who might be using the preset data collection device, we acquire the noisy, backed-up features and original features of these registered users. Understandably, users need to have registered for facial recognition, and their features need to be collected for them to be identified in subsequent facial recognition tasks. The collected user features are encrypted to obtain the original features, and then noisy encryption is applied to obtain the noisy, backed-up features. These are then stored in the server-side database. Once the coverage area is confirmed, the noisy, backed-up features and original features of these registered users are acquired and stored in the feature database of the general execution environment.

[0086] Please see Figure 8 This document provides a detailed flowchart of a feature comparison method for embodiments of this specification. Figure 8 As shown, in one embodiment, the step of comparing the retained features with the target query features to obtain the comparison result of the target query features includes:

[0087] S702, the background features are coarsely ranked according to the similarity between the background features and the target query features, and a second preset number of candidate background features are obtained from the background features, wherein the similarity between the candidate background features and the target query features is greater than that of the other background features in the background features;

[0088] S704, The candidate retention features are finely sorted to identify the unique retention feature;

[0089] S706, Generate the comparison result of the target query feature based on the unique retention feature.

[0090] Specifically, when performing matching in TEE, a combination of coarse ranking and fine ranking can be used to accelerate the sorting process. First, a coarse ranking using the TOPK algorithm is performed, obtaining a second preset number of candidate features without regard to order. Then, a fine ranking is performed on these second preset number of candidate features to obtain a unique feature, i.e., the feature with the highest similarity. For example, in REE, coarse ranking is performed based on dense-state noisy features to obtain candidate dense-state noisy features for TOPK. The original dense-state features corresponding to the candidate dense-state noisy features of TOPK are packaged together with TOPK and sent to TEE. TEE decrypts these original dense-state features of TOPK, performs coarse ranking to obtain the TOP10 candidate features, and then performs fine ranking to obtain the TOP1. The TOP1 feature is used as the comparison result for the target query feature.

[0091] Based on the above embodiments, referring to Figure 9 , Figure 9 A flowchart is provided for the feature comparison method of this invention. The comparison process is implemented in TEE and REE. Figure 9 The light gray area represents the part executed within the REE, and the dark gray area represents the part executed within the REE. First, the REE performs startup operations and client initialization (end-side initialization), and receives feature data sent by the server to build a feature library. Then, it begins collecting target images of the target user and dividing the model. One part of the model is the feature library, and the other part is the facial recognition query features. The facial recognition query features are extracted in the TEE, then subjected to noise addition and encryption to obtain the target dense-state noise-added features. These are sent to the REE for coarse ranking to obtain the top 10 candidate dense-state noise-added features. These features are packaged together with the corresponding original dense-state features and sent to the TEE for feature decryption. An index is built based on the decrypted retained features, and coarse ranking is performed to obtain the top 10 candidate retained features. Fine ranking is then performed to obtain the unique retained features from the top 10, and the comparison is completed.

[0092] In the embodiments of this specification, the target query features are denoised using a differential privacy algorithm in a Trusted Execution Environment (TEE) to obtain denoised target features. These denoised features are then binary-converted to obtain binary features. Encryption calculations are performed on the binary features to obtain denoised target features, thus protecting the security of the target query features. During REE matching, coarse ranking is performed based on Hamming distance. After obtaining candidate denoised features, the original denoised features corresponding to the candidate denoised features are retrieved from the feature database based on the feature identifiers corresponding to the candidate denoised features, improving retrieval efficiency. In the TEE, the original denoised features are decrypted to obtain retained features. The similarity between the retained features and the target query features is calculated to obtain candidate retained features through coarse ranking. Finally, the candidate retained features are fine-ranked to obtain a unique retained feature as the comparison result. This not only effectively protects facial information but also... Furthermore, the system boasts fast retrieval speed. It acquires target images of the target detection object through a pre-set acquisition device. In the general execution environment, the first convolutional network based on the pre-trained image recognition model performs feature processing on the target image to obtain the first feature. In the trusted execution environment, the second convolutional network based on the pre-trained image recognition model performs feature processing on the first feature to obtain the target query feature. By splitting the pre-trained image recognition model, the target query feature is not exposed in the REE, while also meeting the memory requirements of the TEE. Additionally, by confirming the pre-set coverage area of ​​the pre-set acquisition device, it acquires the retained dense-state noisy features and original dense-state features of registered users within the pre-set coverage area. These features are then stored in the feature database of the general execution environment, eliminating the need to store all feature data in the feature database, reducing the retrieval processing volume, and improving data processing efficiency.

[0093] The following will be combined with the appendix Figure 10 This specification provides a detailed description of the feature comparison device provided in the embodiments. It should be noted that the appendix... Figure 10 The feature comparison device in the specification is used to perform the functions described herein. Figures 2-9 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 2-9 The example shown.

[0094] Please see Figure 10 This diagram illustrates a feature comparison device provided in an exemplary embodiment of this specification. The feature comparison device can be implemented as all or part of a device through software, hardware, or a combination of both. The device 1 includes a first acquisition module 11, a first retrieval module 12, a second acquisition module 13, a decryption module 14, and a second retrieval module 15.

[0095] The first acquisition module 11 is used to acquire target dense-state noise-added features in a trusted execution environment and transmit the target dense-state noise-added features to a general execution environment. The target dense-state noise-added features are obtained by adding noise and encryption to the target query features acquired in the trusted execution environment.

[0096] The first retrieval module 12 is used to obtain a first preset number of candidate dense noise features from the retained dense noise features in the feature database based on the similarity between the retained dense noise features and the target dense noise features in the general execution environment. The similarity between the candidate dense noise features and the target dense noise features is greater than that of the other retained dense noise features in the retained dense noise features excluding the candidate dense noise features.

[0097] The second acquisition module 13 is used to acquire the original dense state feature corresponding to the candidate dense state noise feature from the feature database, and send the original dense state feature to the trusted execution environment;

[0098] The decryption module 14 is used to decrypt the original encrypted features to obtain the retained features;

[0099] The second retrieval module 15 is used to compare the retained features with the target query features to obtain the comparison result of the target query features.

[0100] Optionally, the first acquisition module 11 is specifically used to add noise to the target query features in a trusted execution environment using a differential privacy algorithm to obtain target noisy features;

[0101] The target noise-added features are converted into binary features.

[0102] The binary features are encrypted to obtain the target dense-state noisy features.

[0103] Optionally, the first retrieval module 12 is specifically used to calculate the Hamming distance between the base-dense noisy feature in the feature database and the target dense noisy feature, wherein the Hamming distance is used to characterize the similarity between the base-dense noisy feature and the target dense noisy feature;

[0104] A first preset number of candidate dense-state noise features are obtained from the base-dense noise features based on the Hamming distance, wherein the Hamming distance between the candidate dense-state noise features and the target dense-state noise features is less than the remaining base-dense noise features other than the candidate dense-state noise features.

[0105] Optionally, the second acquisition module 13 is specifically used to acquire the feature identifier corresponding to the candidate dense-state noise feature;

[0106] Using the feature identifier as an index, the original dense state feature corresponding to the candidate dense state noise feature is obtained from the feature database.

[0107] Optionally, the first acquisition module 11 is specifically used to acquire the target image of the target detection object through a preset acquisition device;

[0108] In the general execution environment, a first convolutional network based on a pre-trained image recognition model performs feature processing on the target image to obtain a first feature;

[0109] In the trusted execution environment, the first feature is processed by a second convolutional network based on the pre-trained image recognition model to obtain the target query feature.

[0110] Optionally, the first acquisition module 11 and the acceptance module 15 are further used to confirm the preset coverage area of ​​the preset acquisition device;

[0111] Obtain the retained, noisy, and original dense features of registered users within the preset coverage area, and store the retained, noisy, and original dense features into the feature database of the general execution environment.

[0112] Optionally, the second retrieval module 15 is specifically used to perform coarse sorting of the background features based on the similarity between the background features and the target query features, and to obtain a second preset number of candidate background features from the background features, wherein the similarity between the candidate background features and the target query features is greater than that of the other background features in the background features;

[0113] The candidate retention features are refined to identify the unique retention feature;

[0114] The comparison result of the target query feature is generated based on the unique retention feature.

[0115] It should be noted that the feature comparison device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the feature comparison method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the feature comparison device and the feature comparison method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0116] The embodiment numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0117] This specification also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the above-described functionality. Figures 2-9 The feature comparison method described in the illustrated embodiment can be found in the following documentation for its specific execution process. Figures 2-9 The specific details of the illustrated embodiments will not be elaborated here.

[0118] Please refer to Figure 11 This diagram illustrates the structure of a feature comparison device provided in an exemplary embodiment of this specification. The feature comparison device in this specification may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0119] Processor 110 may include one or more processing cores. Processor 110 connects various parts within the feature comparison device using various interfaces and lines, and executes various functions of terminal 100 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 110 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user page, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0120] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described above, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems.

[0121] The memory 120 can be divided into operating system space and user space. The operating system runs in the operating system space, while native and third-party applications run in user space. To ensure that different third-party applications can achieve good running performance, the operating system allocates corresponding system resources for each application. However, different application scenarios within the same third-party application have different requirements for system resources. For example, in local resource loading scenarios, third-party applications have high requirements for disk read speed; in animation rendering scenarios, third-party applications have high requirements for GPU performance. Since the operating system and third-party applications are independent of each other, the operating system often cannot promptly perceive the current application scenario of a third-party application, resulting in the operating system's inability to adapt system resources accordingly.

[0122] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0123] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 can be a touch display screen.

[0124] The touch display screen can be designed as a full-screen, curved screen, or irregularly shaped screen. It can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen; however, this specification does not limit the specific design of the embodiments.

[0125] In addition, those skilled in the art will understand that the structure of the feature comparison device shown in the above figures does not constitute a limitation on the feature comparison device. The feature comparison device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the feature comparison device may also include radio frequency circuits, input units, sensors, audio circuits, WiFi modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0126] exist Figure 11 In the feature matching device shown, the processor 110 can be used to call the feature matching application stored in the memory 120 and specifically perform the following operations:

[0127] In a trusted execution environment, a target dense-state denoised feature is obtained, and the target dense-state denoised feature is transmitted to a general execution environment. The target dense-state denoised feature is obtained by adding noise and encrypting the target query feature obtained in the trusted execution environment.

[0128] In the general execution environment, based on the similarity between the residual dense noise feature in the feature database and the target dense noise feature, a first preset number of candidate dense noise features are obtained from the residual dense noise features. The similarity between the candidate dense noise features and the target dense noise feature is greater than that of the other residual dense noise features in the residual dense noise features excluding the candidate dense noise features.

[0129] Obtain the original dense state feature corresponding to the candidate dense state noise-adding feature from the feature database, and send the original dense state feature to the trusted execution environment;

[0130] The original dense state features are decrypted to obtain the residual features;

[0131] The baseline feature is compared with the target query feature to obtain the comparison result of the target query feature.

[0132] In one embodiment, when the processor 110 acquires the target dense-state noisy features in a trusted execution environment, it specifically performs the following operations:

[0133] In a trusted execution environment, the target query features are denoised using a differential privacy algorithm to obtain denoised target features.

[0134] The target noise-added features are converted into binary features.

[0135] The binary features are encrypted to obtain the target dense-state noise-added features.

[0136] In one embodiment, when the processor 110 performs the operation of obtaining a first preset number of candidate dense-state denoising features from the retained dense-state denoising features based on the similarity between the retained dense-state denoising features in the feature database and the target dense-state denoising features, the processor 110 specifically performs the following operations:

[0137] Calculate the Hamming distance between the base-dense noisy feature in the feature database and the target dense noisy feature. The Hamming distance is used to characterize the similarity between the base-dense noisy feature and the target dense noisy feature.

[0138] A first preset number of candidate dense-state noise features are obtained from the base-dense noise features based on the Hamming distance, wherein the Hamming distance between the candidate dense-state noise features and the target dense-state noise features is less than the remaining base-dense noise features other than the candidate dense-state noise features.

[0139] In one embodiment, when the processor 110 retrieves the original dense-state feature corresponding to the candidate dense-state noisy feature from the feature database and sends the original dense-state feature to the trusted execution environment, it specifically performs the following operations:

[0140] Obtain the feature identifier corresponding to the candidate dense-state noise feature;

[0141] Using the feature identifier as an index, the original dense state feature corresponding to the candidate dense state noise feature is obtained from the feature database.

[0142] In one embodiment, when the processor 110 performs the operation of obtaining target query features in a trusted execution environment, it specifically performs the following operations:

[0143] The target image of the target object is acquired through a preset acquisition device;

[0144] In the general execution environment, a first convolutional network based on a pre-trained image recognition model performs feature processing on the target image to obtain a first feature;

[0145] In the trusted execution environment, the first feature is processed by a second convolutional network based on the pre-trained image recognition model to obtain the target query feature.

[0146] In one embodiment, before the processor 110 performs the operation of obtaining a first preset number of candidate dense-state denoising features from the retained dense-state denoising features based on the similarity between the retained dense-state denoising features in the feature database and the target dense-state denoising features, the processor 110 further performs the following operations:

[0147] Confirm the preset coverage area of ​​the preset data acquisition device;

[0148] Obtain the retained, noisy, and original dense features of registered users within the preset coverage area, and store the retained, noisy, and original dense features into the feature database of the general execution environment.

[0149] In one embodiment, when the processor 110 performs a feature comparison between the retained feature and the target query feature to obtain the comparison result of the target query feature, it specifically performs the following operations:

[0150] The background features are coarsely ranked based on their similarity to the target query features, and a second preset number of candidate background features are obtained from the background features. The similarity between the candidate background features and the target query features is greater than that of the other background features in the background features.

[0151] The candidate retention features are refined to identify the unique retention feature;

[0152] The comparison result of the target query feature is generated based on the unique retention feature.

[0153] In the embodiments of this specification, by distributing the feature comparison process to a trusted execution environment and a general execution environment, the encrypted feature data is placed in the general execution environment for preliminary retrieval to obtain a first preset number of candidate encrypted and noisy features. Then, the corresponding original encrypted features in the database are sent to the trusted execution environment for decryption and feature comparison. This not only enables the processing of a large number of features, but also ensures the security of the features by decrypting the original encrypted features only in the trusted execution environment throughout the entire process. By adding noise to the target query features using a differential privacy algorithm within a Trusted Execution Environment (TEE), noisy target features are obtained. These noisy features are then converted to binary form to obtain binary features. Encryption calculations are performed on the binary features to obtain the target encrypted noisy target features, thus protecting the security of the target query features. During REE matching, coarse ranking is performed based on Hamming distance. After obtaining candidate encrypted noisy features, the original encrypted features corresponding to the candidate encrypted noisy features are retrieved from the feature database based on their corresponding feature identifiers, improving retrieval efficiency. In the TEE, the original encrypted features are decrypted to obtain retained features. Similarity is calculated between the retained features and the target query features to obtain candidate retained features. Finally, the candidate retained features are finely ranked to obtain a unique retained feature as the comparison result. This approach not only effectively protects facial information but also improves retrieval efficiency. It is fast; in addition, the target image of the target detection object is acquired through a preset acquisition device. In the general execution environment, the first convolutional network based on the pre-trained image recognition model performs feature processing on the target image to obtain the first feature. In the trusted execution environment, the first feature is processed by the second convolutional network based on the pre-trained image recognition model to obtain the target query feature. By splitting the pre-trained image recognition model, the target query feature is not exposed in the REE, while meeting the memory requirements of the TEE. In addition, by confirming the preset coverage area of ​​the preset acquisition device, the backed-up dense-state noise-added features and the original dense-state features of registered users within the preset coverage area are acquired. The backed-up dense-state noise-added features and the original dense-state features are stored in the feature database of the general execution environment, so that the feature database does not need to store all feature data, reducing the amount of retrieval processing and improving data processing efficiency.

[0154] Additionally, embodiments of this specification provide a computer program product comprising a computer program that, when executed by a processor of an electronic device, enables the processor to at least perform the functions described above. Figures 2 to 9 The feature comparison method provided in the illustrated embodiment.

[0155] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0156] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. A feature comparison method, comprising: In a trusted execution environment, a target dense-state denoised feature is obtained, and the target dense-state denoised feature is transmitted to a general execution environment. The target dense-state denoised feature is obtained by adding noise and encrypting the target query feature obtained in the trusted execution environment. In the general execution environment, based on the similarity between the residual dense noise feature in the feature database and the target dense noise feature, a first preset number of candidate dense noise features are obtained from the residual dense noise features. The similarity between the candidate dense noise features and the target dense noise feature is greater than that of the other residual dense noise features in the residual dense noise features excluding the candidate dense noise features. Obtain the original dense state feature corresponding to the candidate dense state noise-adding feature from the feature database, and send the original dense state feature to the trusted execution environment; The original dense state features are decrypted to obtain the residual features; The baseline feature is compared with the target query feature to obtain the comparison result of the target query feature.

2. The method as described in claim 1, wherein acquiring the target dense-state noise-added features in a trusted execution environment comprises: In a trusted execution environment, the target query features are denoised using a differential privacy algorithm to obtain denoised target features. The target noise-added features are converted into binary features. The binary features are encrypted to obtain the target dense-state noise-added features.

3. The method as described in claim 1, wherein obtaining a first preset number of candidate dense-state denoising features from the retained dense-state denoising features based on the similarity between the retained dense-state denoising features in the feature database and the target dense-state denoising features includes: Calculate the Hamming distance between the base-dense noisy feature in the feature database and the target dense noisy feature. The Hamming distance is used to characterize the similarity between the base-dense noisy feature and the target dense noisy feature. A first preset number of candidate dense-state noise features are obtained from the base-dense noise features based on the Hamming distance, wherein the Hamming distance between the candidate dense-state noise features and the target dense-state noise features is less than the remaining base-dense noise features other than the candidate dense-state noise features.

4. The method as described in claim 1, wherein obtaining the original dense-state feature corresponding to the candidate dense-state noisy feature from the feature database and sending the original dense-state feature to the trusted execution environment comprises: Obtain the feature identifier corresponding to the candidate dense-state noise feature; Using the feature identifier as an index, the original dense state feature corresponding to the candidate dense state noise feature is obtained from the feature database.

5. The method as described in claim 1, wherein obtaining the target query features in a trusted execution environment includes: The target image of the target object is acquired through a preset acquisition device; In the general execution environment, a first convolutional network based on a pre-trained image recognition model performs feature processing on the target image to obtain a first feature; In the trusted execution environment, the first feature is processed by a second convolutional network based on the pre-trained image recognition model to obtain the target query feature.

6. The method as described in claim 5, further comprising, before obtaining a first preset number of candidate dense-state denoising features from the retained dense-state denoising features based on the similarity between the retained dense-state denoising features in the feature database and the target dense-state denoising features: Confirm the preset coverage area of ​​the preset data acquisition device; Obtain the retained, noisy, and original dense features of registered users within the preset coverage area, and store the retained, noisy, and original dense features into the feature database of the general execution environment.

7. The method as described in claim 1, wherein comparing the retained feature with the target query feature to obtain the comparison result of the target query feature includes: The background features are coarsely ranked based on their similarity to the target query features, and a second preset number of candidate background features are obtained from the background features. The similarity between the candidate background features and the target query features is greater than that of the other background features in the background features. The candidate retention features are refined to identify the unique retention feature; The comparison result of the target query feature is generated based on the unique retention feature.

8. A feature comparison device, the device comprising: The first acquisition module is used to acquire target dense-state denoised features in a trusted execution environment and transmit the target dense-state denoised features to a general execution environment. The target dense-state denoised features are obtained by adding noise and encryption to the target query features acquired in the trusted execution environment. The first retrieval module is used to obtain a first preset number of candidate dense noise features from the base-retained dense noise features in the general execution environment based on the similarity between the base-retained dense noise features in the feature database and the target dense noise features. The similarity between the candidate dense noise features and the target dense noise features is greater than that of the other base-retained dense noise features in the base-retained dense noise features excluding the candidate dense noise features. The second acquisition module is used to acquire the original dense state feature corresponding to the candidate dense state noise feature from the feature database, and send the original dense state feature to the trusted execution environment; The decryption module is used to decrypt the original encrypted features to obtain the retained features; The second retrieval module is used to compare the retained features with the target query features to obtain the comparison result of the target query features.

9. An electronic device, comprising: Processor and memory; The memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as described in any one of claims 1 to 7.

10. A computer storage medium storing a computer program that, when executed by a processor, implements the steps of the method as claimed in any one of claims 1 to 7.

11. A computer program product, comprising: A computer program, when executed by a processor of an electronic device, causes the processor to perform the feature comparison method as described in any one of claims 1 to 7.

Images