A face recognition method and system

Abstract

The present invention relates to a face recognition method and system, and to the field of face recognition technology, and is used to solve the problem that the sensitive information of the person photographed is easily leaked after taking a photo of a person for face recognition. The method includes: collecting a facial image of a person to be identified; performing image recognition on the facial image to obtain first image information and second image information; obtaining the sensitive information of the person to be identified from a pre-created blockchain through the first image information, and the pre-created blockchain pre-stores the sensitive information of the person to be identified; performing de-identification processing on the first image information to obtain third image information; using the second image information and the third image information to generate a facial picture of the person to be identified; and displaying the facial picture and sensitive information of the person to be identified. The technical solution provided by the present invention can determine the sensitive information of others by recognizing faces, while avoiding the leakage of customer privacy information.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a face recognition method and system. Background Art

[0002] As the scale of enterprises continues to expand and the communication between enterprises increases, how to efficiently recognize customers and employees has become a focus of managers.

[0003] Typically, managers and their assistants are required to prepare detailed information and then memorize it solely through memory.

[0004] However, the process of making detailed data may lead to the problem of personal information leakage. For example, as the business develops, the customer photos taken previously may be discarded. In the case that face recognition technology has become popular, the discarded photos mean that the customer's privacy information is at risk of being leaked. Summary of the invention

[0005] In view of the above analysis, the present invention aims to propose a face recognition method and system, which can determine the sensitive information of others by recognizing faces while avoiding the leakage of customer privacy information.

[0006] The purpose of the present invention is mainly achieved through the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a face recognition method, comprising:

[0008] Collecting facial images of persons to be identified;

[0009] Performing image recognition on the facial image to obtain first image information and second image information; wherein the first image information includes location information of facial feature points and facial texture information; and the second image information includes one or more of expression, age, gender and skin color information;

[0010] Acquire the sensitive information of the person to be identified from a pre-created blockchain through the first image information, wherein the pre-created blockchain pre-stores the sensitive information of the person to be identified;

[0011] performing a de-identification process on the first image information to obtain third image information;

[0012] Generate a facial image of the person to be identified using the second image information and the third image information;

[0013] Display the facial image and sensitive information of the person to be identified.

[0014] Further, performing image recognition on the facial image to obtain the first image information and the second image information includes:

[0015] The facial image is input into a pre-trained feature extraction neural network, and the first image information and the second image information are output using the feature extraction neural network.

[0016] Furthermore, before obtaining the sensitive information of the person to be identified from the pre-created blockchain through the first image information, the method further includes:

[0017] Collecting standard facial image information of the person to be identified, wherein the standard facial image includes: location information of facial feature points and facial texture information;

[0018] Encrypting the standard facial image information to obtain a first ciphertext;

[0019] The first ciphertext and sensitive information are stored in the blockchain in correspondence, where the sensitive information includes: resume information and work information.

[0020] Furthermore, obtaining the sensitive information of the person to be identified from a pre-created blockchain through the first image information includes:

[0021] Encrypting the first image information extracted to obtain a second ciphertext;

[0022] Sending the second ciphertext to the blockchain;

[0023] After the blockchain determines that the first ciphertext is the same as the second ciphertext, the sensitive information returned by the blockchain is received.

[0024] Further, sending a download request message to the blockchain;

[0025] After the download request message is approved by the blockchain, a third ciphertext is downloaded, where the third ciphertext is a ciphertext obtained by encrypting the sensitive information using the first ciphertext as a key;

[0026] The third ciphertext is stored in a local database or terminal device.

[0027] Further, encrypting the first image information extracted to obtain a second ciphertext;

[0028] The third ciphertext is decrypted in the local database or on the terminal device according to the second ciphertext to obtain the sensitive information.

[0029] Further, the first image information is input into a de-identified neural network, and the de-identified neural network outputs the third image information;

[0030] The de-identification neural network is used to modify the position information of the feature points of the face and the image features corresponding to the face texture information.

[0031] In a second aspect, an embodiment of the present invention provides a face recognition system, including: a face recognition device and a blockchain storing sensitive information of a person to be recognized; the face recognition device includes: a camera device, a communication device, and an image processing device;

[0032] The camera device is used to collect facial images of the person to be identified;

[0033] The communication device is used to send the collected facial image of the person to be identified to the image processing device;

[0034] The image processing device is used to perform face recognition with de-identified information on the collected facial image of the person to be identified, to obtain first image information and second image information; send the first image information to the blockchain storing sensitive information of the person to be identified; perform de-identified information processing on the first image information to obtain third image information; and generate a facial picture of the person to be identified using the second image information and the third image information;

[0035] The blockchain storing the sensitive information of the person to be identified is used to return the corresponding sensitive information to the image processing device according to the first image information;

[0036] The image processing device is also used to display the facial image and sensitive information of the person to be identified;

[0037] The first image information includes: location information of feature points of a human face and image information of human face texture; the second image information includes: one or more of a person's expression, age, gender and skin color.

[0038] Further, the image processing device comprises: a first image processing module and a second image processing module;

[0039] The first image processing module is provided with a feature extraction neural network for respectively extracting first image information and second image information from the facial image of the person to be identified in a multi-task learning manner;

[0040] The second image processing module is provided with an identity information neural network for performing de-identification processing on the first image information.

[0041] Furthermore, the image processing device further comprises: a standard image acquisition module;

[0042] The standard image acquisition module is used to collect standard facial image information of the person to be identified, wherein the standard facial image includes: location information of facial feature points and facial texture information; encrypt the standard facial image information to obtain ciphertext; and store the ciphertext and sensitive information in the blockchain in correspondence, wherein the sensitive information includes: resume information and work information.

[0043] Beneficial effects of the technical solution of the present invention:

[0044] 1. By extracting the first image information and the second image information respectively, the facial features that can reveal the identity are distinguished from the ordinary facial features. Then the first image information is processed to remove the identity information to obtain the third image information, and the facial features that can reveal the identity are hidden. Finally, the third image information and the second image information are used to synthesize the photo of the person, which retains the person's appearance to the greatest extent, while also ensuring that the photo does not leak personal privacy.

[0045] 2. Utilize multi-task learning of neural networks to realize the separate extraction of the first image information and the second image information, so as to ensure the accuracy of the first image information and the second image information.

[0046] 3. Using the immutability of blockchain information, the standard facial image information is stored on the blockchain, which can ensure both the accuracy and security of the information. At the same time, the first image information and sensitive information are transmitted in ciphertext to further prevent the leakage of personal information.

[0047] 4. The facial recognition device can be carried with you to adapt to various scenarios such as banquets and meetings, and has strong adaptability.

[0048] Other features and advantages of the present invention will be described in the following description, and part of them will become obvious from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the entire drawings, the same reference symbols represent the same components;

[0050] Figure 1 A flow chart of a face recognition method provided by an embodiment of the present invention;

[0051] Figure 2 A structural schematic diagram of a face recognition device provided by an embodiment of the present invention.

[0052] Reference numerals:

[0053] 1- Camera device; 2- Communication device; 3- Image processing device; 4- Front view of the camera device. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0055] In today's society, managers usually face the following scenarios:

[0056] Scenario 1: As a company continues to grow and develop, it usually has dozens of branches at all levels. When holding large-scale internal events, such as annual meetings, managers need to contact the main persons in charge of each branch. However, many times, because they do not know the other party's name, resume, main achievements and main responsibilities, there are communication barriers and even embarrassment for the other party.

[0057] Scenario 2: Many companies have established long-term cooperative relationships through the platform and hold regular communication meetings. With the turnover of personnel and the increase in partners, it is inevitable that they are not familiar with each other at the communication meetings. Usually, managers will inevitably be tentative and reserved with unfamiliar people, which increases communication time and reduces communication efficiency.

[0058] Based on the above scenario, the most direct way is to obtain the other party's photo and then obtain the other party's name, resume, main achievements and main responsibilities through photo comparison. However, while obtaining the other party's facial information, it may lead to the leakage of the other party's facial information. At the same time, the other party's name, resume, main achievements and main responsibilities are also personal privacy. If handled improperly, it will cause immeasurable harm to the other party.

[0059] In order to solve the above technical problems, an embodiment of the present invention provides a face recognition method, comprising:

[0060] Step 101: collect the facial image of the person to be identified.

[0061] Since the standard facial image information needs to be compared with the first image information, it is necessary to be able to capture a high-quality frontal face image when shooting. Therefore, in the embodiment of the present invention, the device for obtaining the facial image of the person to be identified can be carried with you. In this way, the manager does not need to shoot by himself, but can hand over the shooting work to his assistant, so that a high-quality facial image can be captured to facilitate the subsequent verification process.

[0062] Step 102: Perform image recognition on the facial image to obtain first image information and second image information.

[0063] In the embodiment of the present invention, the first image information refers to information in a facial image that can be used to identify a person, such as the geometric description of the eyes, nose, mouth, chin and the structural relationship between them. The second image information refers to image information of other faces other than the first image information, including: one or more of a person's expression, age, gender and skin color. It should be noted that in the embodiment of the present invention, age does not refer to a person's real age, but the age expressed by facial features. For example, many people look very young, and the age reflected by their facial features is inconsistent with their real age.

[0064] In order to obtain the first image information and the second image information simultaneously, the embodiment of the present invention uses a neural network to perform image processing on the collected facial image according to a multi-task learning method.

[0065] Multi-task learning is a method of inductive transfer. Inductive transfer is a method of using related knowledge to improve the learning efficiency in the main task. For example, practicing water skills can help people master various swimming styles faster, practicing long and short distance running can improve football and basketball skills, and reading a lot can improve writing skills. This method of transferring common knowledge between related learning tasks uses the idea of ​​inductive transfer.

[0066] In machine learning, multi-task learning achieves this by making multiple tasks share the same feature representation and coordinating them for parallel learning, so that the learning content of each task can help other tasks learn better.

[0067] In the embodiment of the present invention, the first image information includes the location of the feature points of the face and the face texture image, and the second image information includes: one or more of the expression, age, gender and skin color of the person. The second image information helps to analyze and determine the location of the feature points of the face, so a multi-task learning method can be used to determine the first image information and the second image information.

[0068] Specifically, 3-4 convolutional layers and 3-4 pooling layers are first used to obtain a shared feature vector, and then the shared feature vector is reduced in dimension through a fully connected layer. Finally, two parallel output layers import the shared vector into two tasks respectively, one of which is used to obtain the first image information and the other is used to obtain the second image information.

[0069] Step 103: Acquire sensitive information of the person to be identified from a pre-created blockchain through the first image information.

[0070] In the embodiment of the present invention, in order to ensure the security of private information, the sensitive information of the person to be identified is stored in the blockchain. With the help of the information immutability of the blockchain, the privacy of the person to be identified is not violated and the accuracy of the sensitive information is guaranteed. A private chain can be established within the enterprise, or a consortium chain can be established between enterprises. In addition, a private chain within the enterprise and a consortium chain between enterprises can be set up under the consortium chain led by the national security department.

[0071] Specifically, taking the alliance chain as an example, in scenario 1, the companies under the enterprise form an alliance chain, and each branch is responsible for uploading the standard facial image information, name, resume, main achievements and main responsibilities of their main personnel to the blockchain. In scenario 2, the companies with cooperative relationships form an alliance chain and upload the first image information, name, resume, main achievements and main responsibilities of the person in charge of contact to the blockchain. Among them, resume and main achievements belong to resume information, and main responsibilities belong to work information.

[0072] When storing standard facial image information and sensitive information in the block, each enterprise encrypts the standard facial image information to be uploaded in advance to obtain the first ciphertext, and then stores the first ciphertext and sensitive information in correspondence for verification. For example, the manager encrypts the extracted first image information to obtain the second ciphertext, and sends the second ciphertext to the blockchain. The blockchain directly compares the two ciphertexts to see if they are the same, thereby preventing the first image information from being intercepted and deciphered by a third party during the transmission process.

[0073] All operations on the blockchain require consensus among all nodes, so whether it is scenario one or scenario two, the above rules will result in the manager being unable to quickly obtain the sensitive information of the person to be identified. Therefore, in an embodiment of the present invention, the sensitive information of the person to be identified in the alliance chain is downloaded in advance. When downloading, each node must first reach a consensus, and then encrypt the sensitive information with the first ciphertext as the key, so that each node on the alliance chain can only obtain the third ciphertext. Only when the first image information of the corresponding person is obtained and the first image can be converted into the key corresponding to the third ciphertext, can the sensitive information be obtained, thereby ensuring that personal privacy will not be leaked. The downloaded third ciphertext can be stored in the company's local database or terminal device. When the manager performs identity authentication, he only needs to send the second ciphertext to the local database or terminal device and decrypt the corresponding third ciphertext to obtain the sensitive information in the third ciphertext.

[0074] Personnel changes in enterprises can also lead to deviations in the data on the blockchain. Therefore, in the embodiment of the present invention, each node of the alliance chain needs to regularly update the information on the blockchain and add an identifier in the corresponding block so that each node can update the third ciphertext in real time.

[0075] Step 104: De-identify the first image information to obtain third image information.

[0076] In the embodiment of the present invention, the first image information is de-identified using a convolutional neural network to obtain the third image information, and the third image information does not leak sensitive information of the person to be identified. Specifically:

[0077] Input: Given a face image data set {x i ; 1≤i≤k}, the face image data set contains multiple face images, and the face feature set corresponding to the face image data set {S i ; 1≤i≤k}, where S i =x i ³, face feature set {S i ; 1≤i≤k}includes the first image information.

[0078] Output: third image information.

[0079] 1. Set an integer value m, 1≤m≤k.

[0080] 2. Divide the face dataset into k / m groups. If k is not divisible by m, the final number of groups is m+mod(k / m).

[0081] In the embodiment of the present invention, random number m is used for random grouping.

[0082] 3. Apply the k-anonymity algorithm to the facial features corresponding to each group and obtain the average features of each group:

[0083] S 均 =(x 1 ³+ x 2 ³+ x 3 ³+…+x i ³) / m.

[0084] The key to removing identity information is how to change the position information of the feature points of the face. If the position information of the feature points is changed too much, it will cause face distortion or even a non-face image. If it is changed too little, it is easy for others to reversely infer the position information of the feature points and the facial texture information of the real face.

[0085] During face recognition, feature points are usually set on the face image, and changing the position information of the feature points is actually to make the feature points deviate from the original positions within a certain range. Therefore, in the embodiments of the present invention, the offset is used to represent the specific degree of modification.

[0086] It can be seen that when the offset exceeds the preset error range of face recognition, face recognition cannot identify the identity information corresponding to the face image. Of course, the offset will cause face distortion or even non-face images while making the face recognition function ineffective. The reason why the face is distorted is that the offset is so large that people can recognize the offset with the naked eye. However, the recognition accuracy of the human eye is far lower than that of face recognition, which means that if the order of magnitude of the offset can be controlled to reach the level of face recognition, the probability of people recognizing the offset with the naked eye can be effectively reduced.

[0087] Based on the above theory, in an embodiment of the present invention, feature points of multiple facial images are extracted, and the feature points of the person to be identified are used as a reference to determine the offsets of other feature points, and finally an average offset is obtained. The offset is then used to determine new feature points, so that the order of magnitude of the offset can be controlled to reach the level of facial recognition.

[0088] 4. In the reconstruction layer of the neural network, use the S 均 The third image information is obtained.

[0089] In the embodiment of the present invention, the offset with a relatively large value is removed in the reconstruction layer, and the remaining offset is used to obtain the third image information.

[0090] It should be noted that, in the embodiment of the present invention, the elements in the face image data set can be random, and since the m value is also random within a certain range, the grouping method is also random grouping. Through the above method, it is ensured that others cannot obtain the first image information by inferring the third image information.

[0091] Step 105: Generate a facial image of the person to be identified using the second image information and the third image information.

[0092] In order to ensure that the image is not distorted, in the embodiment of the present invention, only the first image information is modified, while the features of expression, age, gender and skin color information are not modified, and they are combined with the third image information to obtain a new face image. The third image information is the face features synthesized from multiple face photos, so even if others can extract the third image information, they cannot infer the first image information, so as to ensure that the information of the relevant person will not be leaked. Therefore, the facial picture of the person to be identified is synthesized using the third image information that will not leak the sensitive information of the person to be identified and the second image information, so as to obtain a face picture without identity information.

[0093] Step 106: Display the facial image and sensitive information of the person to be identified.

[0094] In the embodiment of the present invention, only the face picture without identity information is displayed. In this way, even if other people get the photo, since the first image information has been replaced by the third image information, other people can only extract the third image information. However, the third image information can neither pass the blockchain authentication nor be used for other identity authentication, thereby ensuring that the privacy of the person to be identified will not be violated.

[0095] like Figure 2 As shown, an embodiment of the present invention provides a face recognition device, including: a camera device 1, a communication device 2 and an image processing device 3, wherein 4 is a front view of the camera device 1. The camera device 1 can be set on the ear or chest of a person. The communication device 2 is used for data transmission and communication, so as to transmit the face image obtained by the camera device 1 to the image processing device 3 and communicate. For example, if it is inconvenient for the manager to move around at a banquet, the manager can talk to the assistant through the communication device 2 to help quickly obtain the image of the target person. The image processing device 3 is provided with a communication module, a neural network module and a processor. Among them, the communication module is used to generate a second ciphertext and interact with a peripheral device, a local database or a terminal device of the blockchain. The neural network module is used to extract the first image information and the second image information from the face image, and to generate the third image information.

[0096] The embodiment of the present invention further provides a face recognition system, comprising: a face recognition device and a blockchain storing sensitive information of a person to be recognized. The face recognition device comprises: a camera device 1, a communication device 2 and an image processing device 3.

[0097] The camera device 1 is used to capture the facial image of the person to be identified.

[0098] The communication device 2 is used to send the collected facial image of the person to be identified to the image processing device.

[0099] The image processing device 3 is used to perform face recognition on the collected facial image of the person to be identified, and obtain first image information and second image information; send the first image information to the blockchain that stores sensitive information of the person to be identified; perform face recognition on the first image information to obtain third image information; and use the second image information and the third image information to generate a facial picture of the person to be identified.

[0100] The blockchain storing the sensitive information of the person to be identified is used to return the corresponding sensitive information to the image processing device based on the first image information.

[0101] The image processing device 3 is also used to display the facial image and sensitive information of the person to be identified.

[0102] In the embodiment of the present invention, the image processing device 3 includes: a first image processing module and a second image processing module.

[0103] The first image processing module is provided with a feature extraction neural network, which is used to extract first image information and second image information from the facial image of the person to be identified in a multi-task learning manner, wherein the first image information includes: location information of feature points of the face and image information of the face texture; the second image information includes: one or more of the person's expression, age, gender and skin color. The second image processing module is provided with an identity information neural network, which is used to perform de-identification processing on the first image information.

[0104] In the embodiment of the present invention, the image processing device 3 further includes: a standard image acquisition module;

[0105] The standard image acquisition module is used to collect standard facial image information of the person to be identified. The standard facial image includes: the location information of the facial feature points and the facial texture information; encrypt the standard facial image information to obtain ciphertext; store the ciphertext and sensitive information in the blockchain in correspondence. The sensitive information includes: resume information and work information.

[0106] Those skilled in the art will appreciate that all or part of the process of the method in the above embodiment can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0107] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A face recognition method, characterized in that: include: Collecting facial images of persons to be identified; Performing image recognition on the facial image to obtain first image information and second image information; wherein the first image information includes location information of feature points of the face and face texture information; and the second image information includes one or more of expression, age, gender and skin color information; Acquire the sensitive information of the person to be identified from a pre-created blockchain through the first image information, wherein the pre-created blockchain pre-stores the sensitive information of the person to be identified; The first image information is processed to remove the identity information to obtain the third image information; wherein the first image information is processed to remove the identity information to obtain the third image information includes: Inputting the first image information into a de-identified neural network, and the de-identified neural network outputs the third image information; The de-identification neural network is used to modify the position information of the feature points of the face and the image features corresponding to the face texture information; Generate a facial image of the person to be identified using the second image information and the third image information; Display the facial image and sensitive information of the person to be identified.

2. The method according to claim 1, characterized in that The step of performing image recognition on the facial image to obtain first image information and second image information includes: The facial image is input into a pre-trained feature extraction neural network, and the first image information and the second image information are output using the feature extraction neural network.

3. The method according to claim 1, characterized in that Before obtaining the sensitive information of the person to be identified from the pre-created blockchain through the first image information, the method further includes: Collecting standard facial image information of the person to be identified, wherein the standard facial image includes: location information of facial feature points and facial texture information; Encrypting the standard facial image information to obtain a first ciphertext; The first ciphertext and sensitive information are stored in the blockchain in correspondence, where the sensitive information includes: resume information and work information.

4. The method according to claim 3, characterized in that: The obtaining the sensitive information of the person to be identified from a pre-created blockchain through the first image information includes: Encrypting the first image information extracted to obtain a second ciphertext; Sending the second ciphertext to the blockchain; After the blockchain determines that the first ciphertext is the same as the second ciphertext, the sensitive information returned by the blockchain is received.

5. The method according to claim 3, characterized in that: The method further comprises: Sending a download request message to the blockchain; After the download request message is approved by the blockchain, a third ciphertext is downloaded, where the third ciphertext is a ciphertext obtained by encrypting the sensitive information using the first ciphertext as a key; The third ciphertext is stored in a local database or terminal device.

6. The method according to claim 5, characterized in that The method further comprises: Encrypting the first image information extracted to obtain a second ciphertext; The third ciphertext is decrypted in the local database or on the terminal device according to the second ciphertext to obtain the sensitive information.

7. A face recognition system, characterized in that: include: Facial recognition devices and blockchains that store sensitive information about people to be identified; The face recognition device comprises: a camera device, a communication device and an image processing device; The camera device is used to collect facial images of the person to be identified; The communication device is used to send the collected facial image of the person to be identified to the image processing device; The image processing device is used to perform face recognition with de-identified information on the collected facial image of the person to be identified, to obtain first image information and second image information; send the first image information to the blockchain storing sensitive information of the person to be identified; perform de-identified information processing on the first image information to obtain third image information; and generate a facial picture of the person to be identified using the second image information and the third image information; The blockchain storing the sensitive information of the person to be identified is used to return the corresponding sensitive information to the image processing device according to the first image information; The image processing device is also used to display the facial image and sensitive information of the person to be identified; The first image information includes: location information of facial feature points and facial texture image information; the second image information includes: one or more of a person's expression, age, gender and skin color; The performing de-identification processing on the first image information to obtain the third image information includes: Inputting the first image information into a de-identified neural network, and the de-identified neural network outputs the third image information; The de-identification neural network is used to modify the position information of the feature points of the face and the image features corresponding to the face texture information.

8. The system according to claim 7, characterized in that The image processing device comprises: a first image processing module and a second image processing module; The first image processing module is provided with a feature extraction neural network for respectively extracting first image information and second image information from the facial image of the person to be identified in a multi-task learning manner; The second image processing module is provided with an identity information neural network for performing de-identification processing on the first image information.

9. The system according to claim 7, characterized in that include: The image processing device also includes: a standard image acquisition module; The standard image acquisition module is used to collect standard facial image information of the person to be identified, wherein the standard facial image includes: location information of facial feature points and facial texture information; encrypt the standard facial image information to obtain ciphertext; and store the ciphertext and sensitive information in the blockchain in correspondence, wherein the sensitive information includes: resume information and work information.

Images