A facial recognition method and system

Abstract

This invention provides a face recognition method and system. The face recognition method includes: acquiring a spectral image of a face to be recognized; inputting the spectral image of the face to be recognized into a trained face spectral image feature extraction model to obtain a target feature vector of the face spectral image, wherein the trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with face information tags and true / false category spectral information tags; and recognizing the target feature vector according to a similarity evaluation standard to obtain face recognition and liveness detection results. This invention overcomes the security vulnerabilities of traditional face detection and improves the accuracy of face recognition results and the security of the face recognition system.

Description

Technical Field

[0001] This invention relates to the field of spectral imaging technology, and in particular to a face recognition method and system. Background Technology

[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. It involves collecting and analyzing images or videos containing faces to automatically infer attributes such as identity, expression, age, and gender. As a type of biometric identification technology, facial recognition is characterized by its non-intrusiveness, non-contact nature, ease of operation, and good concealment, making it widely used in security, management and supervision, and multimedia entertainment.

[0003] Currently, most facial recognition technologies are based on grayscale or RGB images, which limit the image information available. Changes in lighting conditions and shooting angle directly affect the recognition results. Changes in facial expressions, hairstyles, makeup, and glasses can all lead to a decrease in recognition accuracy. Furthermore, malicious actors can severely interfere with the recognition results through disguise, occlusion, masks, and printed photos. Facial recognition technologies based on traditional imaging systems, which only utilize the spatial geometric features of the observed object, are highly sensitive to uncertainties caused by various changes in conditions. The system has poor robustness, and its recognition performance drops sharply in complex environments. Even with the introduction of 3D facial information, the problem of silicone masks cannot be solved, making it impossible to recognize live faces.

[0004] Therefore, there is an urgent need for a facial recognition method and system to solve the above problems. Summary of the Invention

[0005] To address the problems existing in the prior art, the present invention provides a face recognition method and system.

[0006] This invention provides a face recognition method, comprising:

[0007] Obtain the spectral image of the face to be identified;

[0008] The face spectral image to be identified is input into a trained face spectral image feature extraction model to obtain the target feature vector of the face spectral image. The trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with identity information tags and true / false category spectral information tags.

[0009] Based on the similarity evaluation criteria, the target feature vector is identified to obtain face recognition and liveness detection results.

[0010] This invention provides a face recognition method, wherein acquiring the spectral image of the face to be recognized includes:

[0011] The spectral image of the face to be identified is obtained using a facial recognition spectral imaging chip.

[0012] The face recognition spectral imaging chip includes a light modulation layer, an image sensor layer, and a signal processing circuit layer, which are connected vertically from top to bottom.

[0013] The optical modulation layer is used to receive the light signal reflected from the face to be identified and to perform optical modulation.

[0014] The image sensor layer is used to convert the light signal reflected by the light-modulated face to be identified into an electrical signal, and the electrical signal includes facial image spatial information and skin spectral information;

[0015] The signal processing circuit layer is used to process the spatial information of the face image and the skin spectrum information output by the image sensor layer to obtain the face recognition result.

[0016] According to a face recognition method provided by the present invention, the face spectral imaging chip further includes a lens group, wherein the lens group is located on the upper surface of the light modulation layer and connected to the light modulation layer, and is used to focus and image the light signal reflected by the face to obtain the light signal reflected by the face to be recognized.

[0017] According to a face recognition method provided by the present invention, the optical modulation layer includes at least one optical modulation unit, the optical modulation unit includes multiple micro-nano structure arrays, each micro-nano structure array has uniformly distributed through holes arranged according to different preset arrangement rules, and the through hole shapes of each micro-nano structure array are different.

[0018] According to a face recognition method provided by the present invention, the upper surface of the image sensor layer is distributed with a plurality of photosensitive pixel units, and each micro-nano structure array corresponds to at least one photosensitive pixel unit.

[0019] According to the face recognition method provided by the present invention, the trained face spectral image feature extraction model is obtained through the following steps:

[0020] A sample training set is constructed based on sample face spectral images labeled with identity information tags and authenticity category spectral information tags. The authenticity category spectral information tags are spectral information tags for live faces, simulated faces, and non-face categories.

[0021] The sample training set is input into the machine learning model for training to obtain a trained face image feature extraction model, wherein the machine learning model is a convolutional neural network.

[0022] According to a face recognition method provided by the present invention, the step of inputting the sample training set into the machine learning model for training to obtain a trained face spectral image feature extraction model includes:

[0023] Based on deep learning algorithms, the convolutional kernels in the convolutional neural network are trained using the sample training set. If the preset training conditions are met, a well-trained face spectral image feature extraction model is obtained, wherein the convolutional kernels are used to detect the contours of face corners and skin spectral characteristics.

[0024] The present invention also provides a face recognition system, comprising:

[0025] The spectral image acquisition module acquires the spectral image of the face to be identified.

[0026] The feature vector extraction module is used to input the face spectral image to be identified into the trained face spectral image feature extraction model to obtain the target feature vector of the face spectral image. The trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with face information tags and true / false category spectral information tags.

[0027] The recognition module is used to identify the target feature vector according to the similarity evaluation criteria and obtain the face recognition result.

[0028] According to a face recognition system provided by the present invention, the system further includes:

[0029] A sample training set construction module is used to construct a sample training set based on sample face spectral images labeled with face information tags and true / false category spectral information tags. The face information tags are the identity information tags to which the face belongs, and the true / false category spectral information tags are spectral information tags for live faces, simulated faces, and non-face categories.

[0030] The training feature extraction model module is used to input the sample training set into the machine learning model for training to obtain a trained face image feature extraction model, wherein the machine learning model is a convolutional neural network.

[0031] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described face recognition methods.

[0032] This invention provides a face recognition method and system. By acquiring a face spectral image and inputting it into a face spectral image feature extraction model, a target feature vector based on the face's spatial characteristics and spectral reflectance characteristics is obtained. Using a similarity evaluation standard, the target feature vector is compared with feature vectors in a database for identification. Compared with traditional face recognition methods, this method utilizes skin spectral characteristics to simultaneously achieve face recognition and liveness detection, thus addressing the security vulnerabilities of traditional face detection, improving the accuracy of face recognition results, and enhancing the security of the face recognition system. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0034] Figure 1 A schematic flowchart illustrating the face recognition method provided by this invention;

[0035] Figure 2 This is a schematic diagram of the structure of the face recognition spectral imaging chip provided by the present invention;

[0036] Figure 3 This is a schematic diagram of the micro / nano structure array of the optical modulation layer in the face recognition spectral imaging chip provided by the present invention.

[0037] Figure 4 This is a schematic diagram of the structure of the face recognition system provided by the present invention;

[0038] Figure 5 This is a schematic diagram of the structure of an electronic device provided by the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0040] Ordinary RGB cameras can only capture images with three RGB channels. Liveness detection algorithms based on ordinary RGB cameras only use image information from these three channels. They generally require more complex algorithms such as video sequence analysis for recognition, resulting in poor real-time performance and reliability, and a low recognition rate for highly realistic faces.

[0041] Figure 1 This is a flowchart illustrating the face recognition method provided by the present invention, as shown below. Figure 1 As shown, the present invention provides a face recognition method, including:

[0042] Step 101: Obtain the spectral image of the face to be identified;

[0043] Step 102: Input the face spectral image to be identified into the trained face spectral image feature extraction model to obtain the target feature vector of the face spectral image. The trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with identity information tags and true / false category spectral information tags.

[0044] Step 103: Based on the similarity evaluation criteria, identify the target feature vector to obtain face recognition and liveness detection results.

[0045] In this invention, in step 101, the spectral image of the face to be identified can be obtained by a computational spectral device or by a hyperspectral device.

[0046] Further, in step 102, the identity information label of the face is a label containing the spatial characteristics of the face with different identity IDs. Based on sample face spectral images labeled with face identity information labels and authenticity category spectral information labels, a machine learning model is trained to obtain a face spectral image feature extraction model. The face spectral image to be identified is input into the trained face spectral image feature extraction model. The face spectral image feature extraction model is a machine learning model. By training the machine learning model, the face spectral image is transformed into a target feature vector in a high-dimensional space. The obtained target feature vector includes face spatial characteristics and spectral reflectance characteristics. It should be noted that the target feature vector can be a feature vector of a live face, a feature vector of a simulated face, or a feature vector of a non-face.

[0047] It should be noted that the facial spectral image feature extraction model can be a convolutional neural network or any machine learning model. The machine learning model can also be a support vector machine (SVM) or a perceptron. This invention uses the training process of a convolutional neural network as an example.

[0048] Further, in step 103, the similarity evaluation criterion can be the Euclidean distance between feature vectors, the angle between feature vectors in high-dimensional space, or an algorithm used to measure the similarity between vectors in high-dimensional space. The sample feature vector with the highest similarity to the target feature vector in an existing face image database is selected as the candidate recognition result. The sample feature vectors in this database include face spatial characteristics and spectral reflectance characteristics. If the similarity reaches a set threshold, the recognition is considered successful; otherwise, it is considered a failure. Through the above similarity evaluation criteria, the target feature vector is compared and identified, thereby obtaining face recognition and liveness detection results.

[0049] The face recognition method provided by this invention acquires a face spectral image, inputs the face spectral image into a face spectral image feature extraction model, and obtains a target feature vector of face spatial characteristics and spectral reflectance characteristics. Using a similarity evaluation standard, the target feature vector is compared with feature vectors in the database for identification. Compared with traditional face recognition methods, this method utilizes skin spectral characteristics to simultaneously achieve face recognition and liveness detection, making up for the security vulnerabilities of traditional face detection, improving the accuracy of face recognition results, and enhancing the security of the face recognition system.

[0050] Preferably, the face recognition method provided by the present invention can be implemented through a face recognition spectral imaging chip, specifically through the signal processing circuit layer in the face recognition spectral imaging chip. The face recognition spectral imaging chip acquires the spectral image of the face to be recognized. The face recognition spectral imaging chip is used to capture images of the face to be recognized, obtaining a face spectral image with up to hundreds of channels. The face spectral image includes spatial information of the face image and skin spectral information; the face to be recognized can be a live face, a face image, a face in a video, or an object.

[0051] Figure 2 This is a schematic diagram of the structure of the face recognition spectral imaging chip provided by the present invention, as shown below. Figure 2 As shown, this invention provides a face recognition spectral imaging chip, including a light modulation layer 2021, an image sensor layer 2022, and a signal processing circuit layer 2023. The light modulation layer 2021, the image sensor layer 2022, and the signal processing circuit layer 2023 are connected sequentially from top to bottom along the vertical direction, wherein:

[0052] The optical modulation layer 2021 is used to receive the light signal reflected by the face to be identified 203 and perform optical modulation.

[0053] The image sensor layer 2022 is used to convert the light signal reflected by the light-modulated face 203 to be identified into an electrical signal. The electrical signal includes the spatial information of the face image and the light-modulated face spectral information.

[0054] The signal processing circuit layer 2023 is used to process the spatial information of the face image output by the image sensor layer 2022 and the light-modulated face spectral information to obtain the face recognition result.

[0055] Optionally, the face spectral imaging chip also includes a lens group 201, wherein the lens group 201 is located on the upper surface of the light modulation layer 2021 and connected to the light modulation layer 2021, and is used to focus and image the light signal reflected by the face 203 to obtain the light signal reflected by the face to be identified.

[0056] In this invention, when the face recognition spectral imaging chip captures an image of the face to be recognized 203, the lens group in the face recognition spectral imaging chip faces the face to be recognized. In the face recognition spectral imaging chip, a series of lens groups 201 are disposed on one side of the internal structure 202 of the chip, as shown in the reference. Figure 2 As shown, the light modulation layer 2021, image sensor layer 2022, and signal processing circuit layer 2023 constitute the internal structure 202 of the face recognition spectral imaging chip. Light reflected from the face passes through the lens group 201 to obtain a focused image of the light signal reflected from the face, which is then used as the light signal reflected from the face to be recognized. The light modulation layer 2021 has several light modulation units, each containing multiple micro / nano structure arrays, which modulate the received image of the light signal reflected from the face to be recognized. The image sensor layer 2022... Multiple photosensitive pixel units are set on the upper surface. Micro-nano structure arrays that modulate light of different wavelengths are directly fabricated on the surface of the photosensitive pixel unit area. Thus, the light signal reflected by the light-modulated face to be identified can be converted into an electrical signal through the image sensor layer 2022. The electrical signal includes the spatial information of the face image and the light-modulated skin spectrum information. The signal processing circuit layer 2023 performs data analysis and processing on the spatial information of the face image and the light-modulated face spectrum information output by the image sensor layer 2022 to determine whether the target to be identified is a live face, a simulated face, or a non-human face.

[0057] Furthermore, the optical modulation layer 2021 is fabricated directly on the image sensor layer 2022. For example, the optical modulation layer is attached, bonded, bonded, or deposited on the image sensor layer 2022. The image sensor layer 2022 and the signal processing circuit layer 2023 are connected by electrical contact.

[0058] Optionally, the micro-nano structure arrays can be of different types, and the modulation methods of different micro-nano structure arrays can be different. The modulation methods include, but are not limited to, scattering, absorption, transmission, reflection, interference, excitation and resonance enhancement.

[0059] Optionally, the micro / nano structure array includes, but is not limited to, one-dimensional photonic crystals, two-dimensional photonic crystals, surface plasmon resonances, metamaterials, and metasurfaces. Specific materials may include silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, and III-V group materials, or metals. Among them, silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, and silicon carbide.

[0060] Optionally, the light modulation layer 2021 is formed by directly growing one or more layers of material on the image sensor layer 2022 and then etching to prepare the micro-nano structure, for example, by deposition followed by etching; or the micro-nano structure can be prepared by directly etching on the image sensor layer 2022.

[0061] Optionally, the image sensor layer can be a CIS wafer or a CCD image sensor.

[0062] Furthermore, a facial recognition spectral imaging chip is used to capture images of the face to be identified, obtaining facial spectral images with up to hundreds of channels. The information contained in these facial spectral images is far greater than that of images captured by ordinary RGB cameras. In addition to identifying the identity of the face, the facial spectral information obtained by the spectral imaging chip, and then processed by the signal processing circuit layer, can be easily used for liveness detection to identify disguised faces.

[0063] In this invention, the light modulation layer is monolithically integrated with the image sensor, eliminating the need for discrete components and external collimation elements. The spectral imaging chip can be fabricated in a single CMOS fabrication process, which improves device stability, greatly promotes the miniaturization and weight reduction of imaging spectrometers, and reduces the cost of face recognition devices.

[0064] Based on the above embodiments, the optical modulation layer includes at least one optical modulation unit, and the optical modulation unit includes multiple micro-nano structure arrays. Each micro-nano structure array has through holes arranged in a uniformly distributed manner according to different preset arrangement rules, and the shape of the through holes in each micro-nano structure array is different.

[0065] Figure 3 This is a schematic diagram of the micro / nano structure array of the light modulation layer in the face recognition spectral imaging chip provided by the present invention, as shown below. Figure 3As shown, the optical modulation layer is etched with several optical modulation units, each unit containing an array of various micro / nano structures. These micro / nano structures can be holes penetrating the flat plate or micro / nano structures with a certain depth. The optical modulation effect can be altered by changing the structural size parameters and / or shape of the micro / nano structure units within the array. The unit geometry can include, but is not limited to, circles, crosses, regular polygons, rectangles, and any combination thereof. Furthermore, this modulation effect can be changed by altering the parameters of the micro / nano structures. These structural parameters can include, but are not limited to, the micro / nano structure's period, radius, side length, duty cycle, thickness, and any combination thereof.

[0066] Optionally, the optical modulation layer is made of silicon or silicon compounds with a thickness of 300 nm, containing 1000 optical modulation units, each with an overall size of 400 μm. 2 Each optical modulation unit contains an array of 25 micro / nano structures. Each micro / nano structure can be arranged according to different preset rules, and each micro / nano structure can be a periodic arrangement of the same shape with a duty cycle between 10% and 90%. Each micro / nano structure array can be any one of one-dimensional photonic crystals, two-dimensional photonic crystals, surface plasmon resonances, metamaterials, and metasurfaces.

[0067] Based on the above embodiments, the upper surface of the image sensor layer is distributed with multiple photosensitive pixel units, and each micro-nano structure array corresponds to at least one photosensitive pixel unit.

[0068] In this invention, multiple photosensitive pixel units are disposed on the upper surface of the image sensor layer 2022. Micro-nano structure arrays that have different modulation effects on light of different wavelengths are directly prepared on the surface of the photosensitive pixel unit area. Each micro-nano structure array corresponds to one or more photosensitive pixel units in the vertical direction. The image sensor layer 2022 can convert the light signal reflected by the light-modulated face to be identified into an electrical signal.

[0069] Based on the above embodiments, the trained face image feature extraction model is obtained through the following steps:

[0070] A sample training set is constructed based on sample face spectral images labeled with identity information tags and authenticity category spectral information tags. The authenticity information tags are spectral information tags for live faces, simulated faces, and non-face categories.

[0071] The sample training set is input into the machine learning model for training to obtain a trained face image feature extraction model, wherein the machine learning model is a convolutional neural network.

[0072] In this invention, the spectral images of faces in the training sample set are labeled. Each label contains two parts: an identity information label for the face, used for face recognition; and a spectral information label indicating whether the face is a live, simulated, or non-human face, used for liveness detection. Both parts of the label information are used simultaneously to train the same face spectral image feature extraction model.

[0073] Furthermore, the training set of sample face spectral images labeled with face spectral image information is input into a convolutional neural network for training to obtain a trained face spectral image feature extraction model, which is used to extract the target feature vector of the face spectral image to be identified.

[0074] Optionally, the simulated human face can be 3D or an image, while the non-human face can be an animal or an object.

[0075] Optionally, the category of the entire label can be {non-human face, fake human face, human face 1, human face 2, human face 3, ...}. If there are n different facial feature information in the training sample set, then there should be a total of n+2 label categories. The label information includes the identity information of the human face and the spectral reflectance characteristics of the true and false categories.

[0076] Based on the above embodiments, the step of inputting the sample training set into the machine learning model for training to obtain a trained face spectral image feature extraction model includes:

[0077] Based on deep learning algorithms, the convolutional kernels in the convolutional neural network are trained using the sample training set. If the preset training conditions are met, a well-trained face spectral image feature extraction model is obtained, wherein the convolutional kernels are used to detect the contours of face corners and skin spectral characteristics.

[0078] In this invention, based on deep learning algorithms, a large number of real human faces and objects are collected as a sample training set. The sample training set is input into a convolutional neural network to automatically train the convolutional kernel and obtain the loss function value of the convolutional neural network. If it is determined that the obtained loss function value meets the training convergence condition, a well-trained face spectral image feature extraction model is obtained.

[0079] In one embodiment, face recognition and liveness detection processes can be performed independently. Specifically, the spatial image of the face to be recognized is input into a trained face image feature extraction model to obtain the target feature vector of the face spatial image. The trained face image feature extraction model is obtained by training a convolutional neural network with sample face spatial images labeled with facial information tags. This target feature vector includes face spatial characteristics used to identify the identity of the face. According to a similarity evaluation criterion, the target feature vector is compared with feature vectors in a database (which may only contain face spatial characteristics). If the similarity reaches a set threshold, face recognition is considered successful; otherwise, face recognition is considered a failure.

[0080] Furthermore, the target feature vector is input into the trained classifier to obtain the liveness detection result. The trained classifier is obtained by automatically training a deep neural network using sample feature vectors labeled with spectral information of true and false categories.

[0081] Alternatively, the classifier can also be trained using machine learning algorithms such as Support Vector Machines (SVM) and Perceptrons.

[0082] It is understandable that there are significant differences in the spectral reflectance characteristics between living and non-living organisms. Therefore, a classifier can be trained to find differences in the spectral reflectance characteristics corresponding to living and non-living organisms.

[0083] In one embodiment, a face image database is constructed. Based on a similarity evaluation standard, the target feature vector and the feature vector in the face image database are matched. Based on the matching result, the spectral reflectance characteristic curve corresponding to the target feature vector is obtained. The spectral reflectance characteristic curve is then judged. If the characteristic peak of the spectral reflectance characteristic curve is a minimum point at the 545nm and 575nm bands, then the face spectral image to be identified is determined to be a live face image; or if the characteristic peak of the spectral reflectance characteristic curve is a maximum point at the 850nm band, then the face spectral image to be identified is determined to be a live face image.

[0084] Optionally, by utilizing the spectral reflectance characteristics of facial skin, a convolutional kernel with the same "W-shaped" filtering characteristics is designed to perform matched filtering on the facial skin spectrum. A threshold is set to make a decision and obtain the liveness detection result.

[0085] It should be noted that the skin spectrum of a human face primarily reflects the spectral characteristics of human skin. Hemoglobin in the skin absorbs light at 545nm and 575nm, causing the skin's reflectance curve to exhibit a "W" shape in the visible light spectrum. Human skin reflectance reaches its maximum around 850nm, then decreases rapidly with increasing wavelength, before slightly increasing again around 1450nm. Therefore, in the visible light spectrum, liveness detection can be performed based on at least one characteristic peak of hemoglobin, such as the characteristic absorption peaks near wavelengths of 545nm and 575nm; and / or in the near-infrared spectrum, liveness detection can also be performed through the extreme point at 850nm.

[0086] In one embodiment, since the spectral imaging chip uses computational imaging rather than direct imaging, it can also use the original grayscale image output by the image sensor for face recognition. The original image is processed through preprocessing methods such as equalization and noise reduction. Then, face recognition is performed on the processed original grayscale image. Then, the face spectrum in the face recognition image is restored (some key points in the face spectrum are restored) according to the face recognition spectral imaging chip, that is, the spectral information of the face key points is obtained. Finally, the operation and analysis of the signal processing circuit layer in the face spectral imaging chip are used to identify whether it is a live face.

[0087] In another embodiment, the multi-channel image captured by the spectral imaging chip can extract the RGB channels. Therefore, face recognition algorithms applicable to ordinary RGB cameras are also applicable to face spectral images. Using image analysis techniques, corner detectors and contour detectors are designed to abstract face images into feature vectors in a high-dimensional space. These feature vectors are then compared with feature vectors in an existing face image database based on a similarity evaluation criterion. The similarity evaluation criterion can be the Euclidean distance between feature vectors, the angle between feature vectors in high-dimensional space, or an algorithm used to measure the similarity between vectors in high-dimensional space. The sample feature vector in the existing face image database with the highest similarity to the target feature vector is selected as the candidate recognition result. If the similarity reaches a set threshold, the preliminary face recognition is considered successful; otherwise, the preliminary face recognition is considered a failure.

[0088] Furthermore, based on the aforementioned similarity evaluation criteria, a preliminary face image of successfully recognized face is obtained. This image is then captured by a spectral imaging chip to obtain spectral image information of key facial points. Since the spectral image contains RGB information and has up to hundreds of channels, a 2D or 3D convolutional kernel can be constructed to extract the feature vector of the spectral image from the spectral image data cube. Optionally, this convolutional kernel can be set in the signal processing circuit layer of the face recognition spectral imaging chip. Based on the spatial corner contours of the face spectral image and the hemoglobin absorption characteristics in the spectral dimension as prior knowledge, a convolutional kernel for detecting facial corner contours and skin spectral characteristics is constructed using the principle of matched filtering. For example, based on the "W"-shaped feature of the skin spectrum between 500nm and 600nm, a "W"-shaped convolutional kernel is used to perform matched filtering on the facial skin spectrum to obtain the filtered skin spectral reflectance curve. By utilizing the spectral reflectance characteristics of human skin, the system determines whether the captured face target is a live human face by detecting the spectrum at the characteristic peak of the skin's spectral reflectance curve.

[0089] In this invention, face recognition algorithm and liveness detection algorithm are combined. The spectral reflectance characteristics of the face are obtained by capturing the face through a spectral imaging chip, which greatly enriches the face information, makes up for the shortcomings of traditional imaging technology, and has strong anti-interference ability against the influence of factors such as lighting conditions, shooting angle, facial expression, hairstyle and makeup. It has good recognition effect on disguise, occlusion, masks and printed photos, and greatly improves the accuracy of face recognition results.

[0090] Figure 4 This is a schematic diagram of the structure of the face recognition system provided by the present invention, as shown below. Figure 4 As shown, the present invention provides a face recognition system, including a spectral image acquisition module 401, a feature vector extraction module 402, and a recognition module 403. The spectral image acquisition module 401 acquires a spectral image of a face to be recognized. The feature vector extraction module 402 inputs the spectral image of the face to be recognized into a trained face spectral image feature extraction model to obtain a target feature vector of the face spectral image. The trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with identity information tags and authenticity category spectral information tags. The recognition module 403 identifies the target feature vector according to a similarity evaluation standard to obtain face recognition and liveness detection results.

[0091] Optionally, the lens group, light modulation layer, and image sensor layer in the face recognition spectral imaging chip can be regarded as the spectral image acquisition module 401. Through the lens group, light modulation layer, and image sensor layer, the face spectral image of the face to be recognized is acquired. The face spectral image includes face image spatial information and face spectral information. The feature vector extraction module 402 and the recognition module 403 can be set in the signal processing circuit layer. They are used to acquire the target feature vector of the face spectral image and match the target feature vector with the feature vector of the face image database according to the similarity evaluation standard. The feature vector with the highest similarity in the face image database is obtained as the candidate recognition result. If the similarity meets the set threshold, the face recognition and liveness recognition results are obtained.

[0092] This invention provides a face recognition system that acquires a face spectral image and inputs it into a face spectral image feature extraction model to obtain target feature vectors based on the face's spatial characteristics and spectral reflectance characteristics. Using a similarity evaluation standard, the target feature vectors are compared with feature vectors in a database for identification. Compared to traditional face recognition methods, this system utilizes skin spectral characteristics to simultaneously achieve face recognition and liveness detection, thus addressing security vulnerabilities in traditional face detection, improving the accuracy of face recognition results, and enhancing the security of the face recognition system.

[0093] Based on the above embodiments, the system further includes a sample training set construction module and a feature extraction model training module. The sample training set construction module is used to construct a sample training set based on sample face spectral images labeled with identity information tags and authenticity category spectral information tags. The authenticity category spectral information tags are spectral information tags for live faces, simulated faces, and non-face categories. The feature extraction model training module is used to input the sample training set into the machine learning model for training to obtain a trained face image feature extraction model. The machine learning model is a convolutional neural network.

[0094] Based on the above embodiments, the training feature extraction model module further includes a training feature extraction model unit. The training feature extraction model unit is used to train the convolutional kernel in the convolutional neural network based on the deep learning algorithm and the sample training set. If the preset training conditions are met, a trained face image feature extraction model is obtained. The convolutional kernel is used to detect the contour of face corners and skin spectral characteristics.

[0095] The system provided by this invention is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0096] Figure 5 This is a schematic diagram of the structure of an electronic device provided by the present invention, such as... Figure 5As shown, the electronic device may include: a processor 501, a communication interface 502, a memory 503, and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504. The processor 501 can call logical instructions in the memory 503 to execute a face recognition method, which includes: acquiring a spectral image of a face to be recognized; inputting the spectral image of the face to be recognized into a trained face spectral image feature extraction model to obtain a target feature vector of the face spectral image, wherein the trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with face identity information tags and true / false category spectral information tags; and recognizing the target feature vector according to a similarity evaluation standard to obtain face recognition and liveness detection results.

[0097] Furthermore, the logical instructions in the aforementioned memory 503 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the face recognition method provided by the above methods, the method comprising: acquiring a face spectral image to be recognized; inputting the face spectral image to be recognized into a trained face spectral image feature extraction model to obtain a target feature vector of the face spectral image, wherein the trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with face identity information tags and true / false category spectral information tags; and recognizing the target feature vector according to a similarity evaluation standard to obtain face recognition and liveness recognition results.

[0099] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned face recognition methods. The method includes: acquiring a spectral image of a face to be recognized; inputting the spectral image of the face to be recognized into a trained face spectral image feature extraction model to obtain a target feature vector of the face spectral image, wherein the trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with identity information tags and authenticity category spectral information tags; and recognizing the target feature vector according to a similarity evaluation standard to obtain face recognition and liveness detection results.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0101] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0102] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A face recognition method, characterized in that, include: Acquire a spectral image of a face to be identified; wherein, acquiring the spectral image of a face to be identified includes: acquiring the spectral image of a face to be identified through a face recognition spectral imaging chip; The face spectral image to be identified is input into a trained face spectral image feature extraction model to obtain the target feature vector of the face spectral image. The trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with face identity information tags and true / false category spectral information tags. The multi-task learning uses n+2 categories of labels to train the machine learning model. The n+2 categories include n face identity categories, one fake face category, and one non-face category. Based on the similarity evaluation criteria, the target feature vector is identified to obtain face recognition and liveness detection results; The sample training set is input into the machine learning model for training to obtain a trained face spectral image feature extraction model, including: Based on deep learning algorithms, the convolutional kernels in the convolutional neural network are trained using the sample training set. If the preset training conditions are met, a well-trained face spectral image feature extraction model is obtained. The convolutional kernels are used to detect the contours of face corners and the spectral reflectance characteristics of skin. Liveness recognition is performed based on the extreme points of the spectral reflectance characteristic curve in the preset band. The filtering characteristics of the convolutional kernels are matched with the spectral reflectance characteristics of skin. The face recognition and liveness detection processes are completed independently. Obtaining the face recognition and liveness detection results includes: comparing the similarity between the target feature vector and the feature vector in the database to obtain the face recognition result; and inputting the target feature vector into a trained classifier to obtain the liveness detection result.

2. The face recognition method according to claim 1, characterized in that, The face recognition spectral imaging chip includes a light modulation layer, an image sensor layer, and a signal processing circuit layer, which are connected vertically from top to bottom. The optical modulation layer is used to receive the light signal reflected from the face to be identified and to perform optical modulation. The image sensor layer is used to convert the light signal reflected by the light-modulated face to be identified into an electrical signal, and the electrical signal includes the face image spatial information and the light-modulated skin spectrum information; The signal processing circuit layer is used to process the spatial information of the face image and the light-modulated skin spectrum information output by the image sensor layer to obtain the face recognition result.

3. The face recognition method according to claim 2, characterized in that, The face spectral imaging chip also includes a lens group, which is located on the upper surface of the light modulation layer and connected to the light modulation layer. The lens group is used to focus and image the light signal reflected by the face to obtain the light signal reflected by the face to be identified.

4. The face recognition method according to claim 2, characterized in that, The optical modulation layer includes at least one optical modulation unit, and the optical modulation unit includes multiple micro-nano structure arrays. Each micro-nano structure array has uniformly distributed through holes arranged according to different preset rules, and the through hole shapes of each micro-nano structure array are different.

5. The face recognition method according to claim 2, characterized in that, The upper surface of the image sensor layer is distributed with multiple photosensitive pixel units, and each micro-nano structure array corresponds to at least one photosensitive pixel unit.

6. The face recognition method according to claim 1, characterized in that, The trained face spectral image feature extraction model is obtained through the following steps: A sample training set is constructed based on sample face spectral images labeled with identity information tags and authenticity category spectral information tags. The authenticity category spectral information tags are spectral information tags for live faces, simulated faces, and non-face categories. The sample training set is input into the machine learning model for training to obtain a trained face image feature extraction model, wherein the machine learning model is a convolutional neural network.

7. A face recognition system, characterized in that, include: A spectral image acquisition module acquires a spectral image of the face to be identified; wherein, acquiring the spectral image of the face to be identified includes: acquiring the spectral image of the face to be identified through a face recognition spectral imaging chip; The feature vector extraction module is used to input the face spectral image to be identified into the trained face spectral image feature extraction model to obtain the target feature vector of the face spectral image. The trained face spectral image feature extraction model is obtained by training a machine learning model with sample face spectral images labeled with face identity information tags and true / false category spectral information tags. The multi-task learning uses n+2 categories of labels to train the machine learning model. The n+2 categories include n face identity categories, one fake face category, and one non-face category. The recognition module is used to recognize the target feature vector according to the similarity evaluation criteria and obtain face recognition and liveness recognition results; The sample training set is input into the machine learning model for training to obtain a trained face spectral image feature extraction model, including: Based on deep learning algorithms, the convolutional kernels in the convolutional neural network are trained using the sample training set. If the preset training conditions are met, a well-trained face spectral image feature extraction model is obtained. The convolutional kernels are used to detect the contours of face corners and the spectral reflectance characteristics of skin. Liveness recognition is performed based on the extreme points of the spectral reflectance characteristic curve in the preset band. The filtering characteristics of the convolutional kernels are matched with the spectral reflectance characteristics of skin. The face recognition and liveness detection processes are completed independently. Obtaining the face recognition and liveness detection results includes: comparing the similarity between the target feature vector and the feature vector in the database to obtain the face recognition result; and inputting the target feature vector into a trained classifier to obtain the liveness detection result.

8. The face recognition system according to claim 7, characterized in that, The system also includes: A sample training set construction module is used to construct a sample training set based on sample face spectral images labeled with identity information tags and authenticity category spectral information tags. The authenticity category spectral information tags are spectral information tags for live faces, simulated faces, and non-face categories. The training feature extraction model module is used to input the sample training set into the machine learning model for training to obtain a trained face image feature extraction model, wherein the machine learning model is a convolutional neural network.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the face recognition method as described in any one of claims 1 to 6.

Images