Biological feature recognition method and apparatus, medium and product

Abstract

Disclosed in the present application are a biological feature recognition method and apparatus, a medium and a product. The method comprises: acquiring a biological feature image; generating a physiological signal feature representation of the biological feature image, the physiological signal feature representation being used for representing, in the form of a feature vector, a biological pulse signal contained in the biological feature image; on the basis of the intensity of the biological pulse signal in the physiological signal feature representation, generating a skin attention feature representation corresponding to the biological feature image, the skin attention feature representation being used for representing an active region of the biological pulse signal in the biological feature image; and fusing the physiological signal feature representation and the skin attention feature representation to obtain a fused feature representation, and, on the basis of the fused feature representation, obtaining a liveness recognition result of the biological feature image.

Description

Biometric identification methods, devices, media and products

[0001] This application claims priority to Chinese Patent Application No. 202510207307.7, filed on February 24, 2025, entitled “Method, Apparatus, Medium and Product for Identifying Biometric Features”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image recognition, and in particular to a method, apparatus, medium, and product for identifying biometric features. Background Technology

[0003] Biometric identification is widely used in the security field. For example, in scenarios such as facial recognition payment, palm recognition payment, fingerprint unlocking, and facial recognition access control, it is necessary to identify biometric features and complete security verification based on the identification results.

[0004] Among related technologies, remote photoplethysmography (rPPG) is applied to biometric detection. rPPG is a technique that uses reflected light to measure changes in skin brightness, thereby analyzing signals such as heart rate expressed by biometrics.

[0005] Technical content

[0006] This application provides a method, apparatus, medium, and product for biometric identification, which can improve the accuracy of biometric identification. The technical solution is as follows:

[0007] This application provides a method for biometric identification, the method comprising:

[0008] Acquire biometric images, wherein the biometric images are images acquired from the subject being collected;

[0009] Generate a physiological signal feature representation of the bio-image, wherein the physiological signal feature representation is used to characterize the biological pulse signal contained in the bio-image in the form of a feature vector;

[0010] Based on the intensity of the biopulse signal in the physiological signal feature representation, a skin attention feature representation corresponding to the biofeature image is generated, and the skin attention feature representation is used to characterize the active area of ​​the biopulse signal in the biofeature image;

[0011] The physiological signal feature representation and the skin attention feature representation are fused to obtain a fused feature representation. Based on the fused feature representation, the liveness recognition result of the biometric image is obtained. The liveness recognition result is used to indicate whether the acquired object in the biometric image is a live body.

[0012] This application also provides a biometric identification device, the device comprising:

[0013] The acquisition module is used to acquire biometric images, which are images acquired from the target object.

[0014] A generation module is used to generate a physiological signal feature representation of the bio-feature image, wherein the physiological signal feature representation is used to characterize the biological pulse signal contained in the bio-feature image in the form of a feature vector;

[0015] The recognition module is used to generate a skin attention feature representation corresponding to the bio-feature image based on the intensity of the bio-pulse signal in the physiological signal feature representation, wherein the skin attention feature representation is used to characterize the active area of ​​the bio-pulse signal in the bio-feature image;

[0016] The recognition module is further configured to fuse the physiological signal feature representation and the skin attention feature representation to obtain a fused feature representation, and to obtain a liveness recognition result of the biometric image based on the fused feature representation. The liveness recognition result is used to indicate whether the collected object in the biometric image is a live body.

[0017] This application also provides a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the biometric identification method provided in the various embodiments of this application.

[0018] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the biometric identification method provided in the various embodiments of this application.

[0019] This application also provides a computer program product that, when run on a computer, causes the computer to execute the biometric identification method provided in the various embodiments of this application.

[0020] Brief description of the attached figures

[0021] Figure 1 is a schematic diagram of a biometric identification system provided in some embodiments of this application;

[0022] Figure 2 is a schematic diagram of the face liveness recognition process provided in some embodiments of this application;

[0023] Figure 3 is a flowchart of a biometric identification method provided in some embodiments of this application;

[0024] Figure 4 shows a comparative schematic diagram of biometric images and skin mask images provided in some embodiments of this application;

[0025] Figure 5 is a flowchart of a biometric identification method provided in some embodiments of this application;

[0026] Figure 6 is a schematic diagram of physiological signals related to skin attention provided in some embodiments of this application;

[0027] Figure 7 is a schematic diagram of the overall process of liveness detection provided in some embodiments of this application;

[0028] Figure 8 is a flowchart of a biometric identification method provided in some other embodiments of this application;

[0029] Figure 9 is a schematic diagram of facial images provided in some embodiments of this application;

[0030] Figure 10 is a schematic diagram of the loss calculation process provided in some embodiments of this application;

[0031] Figure 11 is a flowchart of a facial recognition method provided in some embodiments of this application;

[0032] Figure 12 is a structural block diagram of a biometric identification device provided in some embodiments of this application;

[0033] Figure 13 is a structural block diagram of a biometric identification device provided in some other embodiments of this application;

[0034] Figure 14 is a structural block diagram of a computer device provided in some embodiments of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0036] First, a brief introduction to the terms used in the embodiments of this application will be given.

[0037] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0038] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0039] Liveness detection is a biometric technology designed to verify whether the detected object is a real, living person, and not a forgery such as a photograph, video, or model. Its principle is based on analyzing vital characteristics, such as subtle changes in facial expressions, eye movements, skin texture, and blood flow. Liveness detection has a wide range of applications, including secure verification in payment processes, unlocking smartphones, security checks, and online identity authentication. It effectively enhances the security and reliability of biometric systems.

[0040] Remote photoplethysmography (rPPG) is a technique that uses reflected ambient light to measure subtle changes in skin brightness caused by blood flow due to heartbeats. For images or videos captured from a living organism, rPPG can obtain signals similar to a heartbeat, which can then be used to predict physiological characteristics such as heart rate.

[0041] In related technologies, rPPG technology is used to train an rPPG model. The video obtained by collecting biometric data of a first object is input into the rPPG model, and the model analyzes the physiological characteristics exhibited by the first object in the video to identify whether the first object is a living person. The video obtained by collecting biometric data of the first object includes at least one of the following: a video of the first object's face and a video of the first object's palm print. This application does not limit the type of biometric data.

[0042] However, in the above methods, the rPPG model needs to analyze the changes in the physiological characteristics of the first object over a period of time to determine whether the object is alive. This results in a long time consumption for liveness detection using the rPPG model, reducing the efficiency of biometric identification. Furthermore, when faced with forged biometric features, such as 3D masks masquerading as faces or gloves masquerading as hands, the accuracy of liveness detection using the rPPG model is low.

[0043] In some embodiments, the application of biometric identification methods in security access control scenarios is illustrated as an example.

[0044] In a schematic representation, a security access control scenario is one where an individual unlocks the access control system using biometric features to enter a specific area, such as facial recognition, palm print recognition, or iris recognition. Compared to traditional passwords or keys, biometric unlocking is more convenient and secure, effectively preventing identity theft and unauthorized intrusion. This is often combined with technologies such as liveness detection to further prevent bypassing the system by forging biometric features (such as photos, videos, masks, or fake fingerprints), thereby enhancing the security of the access control system.

[0045] In this embodiment, a biometric video of the subject is collected, which includes a biometric image. The physiological signal feature representation implicit in the biometric image of the biometric video is analyzed by an rPPG model, that is, the feature representation of the rPPG signal of the biometric image is extracted by the rPPG model. Based on the skin attention model, the feature representation of the rPPG signal and the biometric image are analyzed to obtain the skin attention feature representation. The physiological signal feature representation and the skin attention feature representation are fused to obtain the fused feature representation. Liveness recognition is performed on the fused feature representation to obtain the liveness recognition result of the subject collected by the biometric video. The liveness recognition result is used to indicate whether the subject collected is alive.

[0046] The biometric identification system involved in the embodiments of this application will be described below. The biometric identification method provided in the embodiments of this application can be implemented by the terminal alone, by the server, or by the terminal and the server through data interaction. The embodiments of this application do not limit this. In some embodiments, the biometric identification method is described by example with the terminal and the server interacting to execute it.

[0047] For illustrative purposes, please refer to Figure 1. This biometric identification system involves a terminal 110 and a server 120, which are connected via a communication network 130.

[0048] In some embodiments, terminal 110 has a biometric data acquisition function, which is used to collect biometric features. For example, terminal 110 can collect facial multimedia data through a camera, such as facial video or Graphics Interchange Format (GIF) data, which can be used to determine dynamically displayed facial changes. Alternatively, terminal 110 can collect palmprint multimedia data through a camera, such as palmprint video or GIF data. The above-mentioned facial multimedia data and palmprint multimedia data are merely illustrative examples; biometric features also include iris data, fingerprint data, etc., which are not limited in this application embodiment.

[0049] In some embodiments, taking facial liveness recognition as an example, terminal 110 can capture facial video through a camera. Terminal 110 can analyze the facial video locally; or, terminal 110 can send the facial video to server 120 through communication network 130, and server 120 can perform the analysis process. The following description uses the facial video analysis process performed by server 120 as an example.

[0050] Server 120 receives a face video sent by terminal 110. First, it analyzes the rPPG signal features of each frame in the face video to generate a physiological signal feature representation (rPPG signal feature representation) for each frame. Based on the face attention mechanism, it analyzes the physiological signal feature representation and the image frame to obtain the face attention feature representation corresponding to each image frame. The physiological signal feature representation and the face attention feature representation are fused to obtain a fused feature representation. Liveness detection is then performed on the fused feature representation to obtain a liveness detection result, which is used to indicate whether the captured object is alive.

[0051] Figure 2 is a schematic diagram of the face liveness recognition process provided in some embodiments of this application. As shown in Figure 2, the process mainly includes the following steps.

[0052] Step 210: Capture a video of the face.

[0053] The facial video can be a pre-captured video of a preset duration, or it can be a video continuously captured in real time. Taking the continuous real-time capture of facial video as an example, terminal 110 continuously captures facial video in real time and sends the captured image frames to server 120. Server 120 performs liveness detection on the face based on the continuously received image frames until it obtains the liveness detection result and the facial recognition result, at which point terminal 110 terminates the capture of facial video.

[0054] Step 220, Preprocessing.

[0055] Preprocessing refers to the cleaning and normalization of the raw data of a face video. Illustratively, this involves removing image frames with resolutions below a threshold from the face video and enhancing the contrast of image frames in the face video by a preset adjustment range. This application does not limit the specific processing steps of the preprocessing.

[0056] Step 230: Call the model to perform liveness detection.

[0057] In this embodiment, a physiological signal recognition model (rPPG model) and a skin attention recognition model are used to perform liveness detection on face videos.

[0058] In some embodiments, physiological signal feature representations of image frames in a face video are first extracted using a physiological signal recognition model; the physiological signal feature representations are then input into a skin attention recognition model to obtain skin attention feature representations corresponding to each image frame; the skin attention feature representations are then input into a physiological signal recognition model, and the physiological signal feature representations and skin attention feature representations are fused using the physiological signal recognition model to obtain a fused feature representation; finally, a classifier in the physiological signal recognition model is used to perform liveness detection on the fused feature representation to obtain the face recognition classification result.

[0059] Step 240: Obtain the classification results.

[0060] In some embodiments, the classification result is used to indicate whether a face in a face video is a live person.

[0061] In some embodiments, when the liveness detection result indicates that the face in the face video is live, further face recognition can be performed to obtain a face recognition result, which is used to indicate the identity information corresponding to the face. In some embodiments, the database stores face feature data corresponding to at least one identity information; the face feature representation corresponding to the face video is extracted, the face feature representation is matched with the face feature data in the database, and the face feature data matching the face feature representation and the identity information corresponding to the face feature data are determined from the database as the identity information obtained from the face video recognition.

[0062] The terminal 110 can be a mobile terminal, such as a mobile phone, tablet, vehicle terminal, smartwatch, access control device, payment verification device, or other device with biometric data collection capabilities. The terminal 110 runs a first application program, which is used to collect biometric images and perform liveness detection on the biometric images.

[0063] It is worth noting that the aforementioned server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0064] In some embodiments, the server described above can also be implemented as a node in a blockchain system.

[0065] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant regions. For example, the facial data, palm print data, etc. involved in this application were obtained with full authorization.

[0066] Figure 3 is a flowchart of a biometric identification method provided in some embodiments of this application. The method is executed by a computer device, which includes at least one of a terminal and a server. Taking the method executed by a server as an example, as shown in Figure 3, the method includes the following steps.

[0067] Step 310: Obtain biometric images.

[0068] Biometric images are images obtained from the acquired object.

[0069] In some embodiments, the method of acquiring biometric images includes at least one of the following: first, receiving a biometric video uploaded by a terminal, the biometric video including multiple frames of biometric images; second, receiving a biometric video stream acquired and transmitted by the terminal in real time, the biometric video stream including multiple frames of biometric images.

[0070] Taking facial recognition payment as an example, the receiving terminal uploads a facial video, which includes multiple frames of facial images. These facial images are the biometric images for liveness detection. Taking palm recognition payment as an example, the receiving terminal uploads a palmprint video, which includes multiple frames of palmprint images. These palmprint images are the biometric images for liveness detection.

[0071] When the biometric image is a face image, the object of acquisition is the face; when the biometric image is a palmprint image, the object of acquisition is the hand. In some embodiments, the object of acquisition can be a living person, that is, the terminal directly acquires the biometric image of a living person. The object of acquisition can also be an attack target, that is, the terminal acquires the biometric image of an attack target using attack methods. For example: when the biometric image includes a face image, the object of acquisition can be a face mask, that is, the attack target wears a face mask to complete the acquisition of the face image; when the biometric image includes a palmprint image, the object of acquisition can be a palmprint glove, that is, the attack target wears a palmprint glove to complete the acquisition of the palmprint image; when the biometric image includes a fingerprint image, the object of acquisition can be a fingerprint glove, that is, the attack target wears a fingerprint glove to complete the acquisition of the fingerprint image; when the biometric image includes an iris image, the object of acquisition can be a colored contact lens, that is, the attack target wears a colored contact lens to complete the acquisition of the iris image.

[0072] Step 320: Generate physiological signal feature representations of biometric images.

[0073] Physiological signal feature representation is used to characterize the physiological signals contained in a biometric image, such as a biological pulse signal, in the form of a feature vector.

[0074] In some embodiments, the physiological signal feature representation may include physiological signal feature representations extracted from each pixel in the biometric image; or, it may include physiological signal feature representations extracted from sampling points obtained from the biometric image. That is, physiological signal feature representations can be extracted from each pixel in the biometric image, which can comprehensively cover the entire image area; or, physiological signal feature representations can be extracted from sampling points obtained from the biometric image, which can reduce computational load and improve processing speed.

[0075] In some embodiments, a physiological signal feature representation of a biomarker image can be generated using a pre-trained physiological signal recognition model. Specifically, the biomarker image can be input into the physiological signal recognition model, which analyzes the biological pulse signals it contains, thereby extracting the physiological signal feature representation from the biomarker image.

[0076] In some embodiments, the physiological signal recognition model can directly identify the biological pulse signal of a frame of biofeature image to obtain the physiological signal feature representation of the biofeature image.

[0077] In other embodiments, the physiological signal recognition model can also obtain the physiological signal feature representation of a biomarker image by recognizing changes in the biological pulse signal in multiple frames of biomarker images.

[0078] Taking the physiological signal recognition model implemented as the rPPG model as an example, the working principle of the rPPG model is to measure physiological indicators, such as heart rate and respiratory rate, by capturing minute color changes on the skin surface. The recognition principle of the rPPG model includes the following:

[0079] 1. Light Reflection and Absorption: When the heart beats, changes in blood volume cause periodic changes in the oxygen saturation of the skin, resulting in subtle changes in the skin's color. These changes can be captured by a camera.

[0080] 2. Signal Extraction: By recognizing the RGB channel signals in a face video, the blood volume pulse wave signal, also known as the biological pulse signal, is extracted from the heartbeat. Common methods include estimating blood oxygen content using the ratio of DC to AC signals of different color channels, and recovering the blood volume pulse wave signal (biological pulse signal) from the AC signal.

[0081] 3. Signal Processing: Frequency domain feature analysis is performed on the extracted blood volume pulse wave signal (biological pulse signal) to calculate physiological indicators such as heart rate and respiratory rate. Remote photoplethysmography (rPPG) captures changes in light reflection on the skin surface of a biological image (such as a human body) using a camera to estimate physiological signals such as heart rate and respiratory rate. Unlike traditional contact photoplethysmography (PPG), rPPG does not require direct contact with the skin; instead, it uses a regular camera (such as a mobile phone camera) to collect biometric data from a distance to identify physiological signal data.

[0082] In some embodiments, a pre-trained physiological signal recognition model can identify the bio-pulse signals contained in at least two image frames in a biometric video, thereby obtaining a physiological signal feature representation of the biometric image. In some embodiments, the at least two image frames include the current biometric image and at least one adjacent image frame. Illustratively, if the i-th frame in the biometric video is taken as the biometric image, the physiological signal recognition model identifies the changes in the bio-pulse signals between the (i-2)-th, (i-1)-th, and i-th image frames, thereby obtaining a physiological signal feature representation of the i-th image frame, where i is an integer greater than 2.

[0083] In some embodiments, by identifying color changes in the skin surface color corresponding to the biofeature in at least two image frames, the bio-pulse signal of the biofeature image is identified, and the bio-pulse signal expressed in a feature representation manner is obtained, thus obtaining a physiological signal feature representation. In some embodiments, the physiological signal feature representation is used to express the bio-pulse signal in the form of a feature vector.

[0084] In some embodiments, the contraction and relaxation of the heart propels blood through blood vessels. As blood flows through arteries, the heart's pumping action creates pressure on the arterial walls, forming a pulse. The physiological signal feature representation includes the relationship between pixels in the biometric image and the pulse. For example, when a pixel is within the pulse region where blood pressure is exerted on the arterial wall, it is represented by a first feature vector; when a pixel is within a region where blood flow does not exert pressure on the arterial wall, it is represented by a second feature vector.

[0085] In some embodiments, visual feature representations of the biometric image are first extracted, and then physiological signal feature representations are extracted using a physiological signal branch network. In some embodiments, the physiological signal branch network is implemented as PhysNet (Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Network). The feature processing procedure implemented by the physiological signal branch network on the visual feature representation includes: performing convolution and pooling operations on the visual feature representation to obtain a multi-channel feature representation, which includes the physiological features of the first object in both spatial and temporal dimensions; and projecting the multi-channel feature representation onto the signal space to obtain the physiological signal feature representation. Here, the visual feature representation is used to indicate the visual information represented by the biometric image.

[0086] PhysNet is an end-to-end spatiotemporal network. The input to this network architecture is a T-frame image with RGB channels (such as a face image). After multiple convolution and pooling operations, a multi-channel manifold is formed to represent the spatiotemporal features. The latent manifold is projected onto the signal space using the channel-wise convolution operation of the convolution kernel to generate a representation of physiological signal features.

[0087] Step 330: Based on the intensity of the biopulse signal in the physiological signal feature representation, generate the skin attention feature representation corresponding to the biofeature image. The skin attention feature representation is used to characterize the active area of ​​the biopulse signal in the biofeature image.

[0088] In some embodiments, the skin attention feature representation includes:

[0089] The skin attention weights of the target pixels in the biometric image, or

[0090] The biometric image contains skin attention weights for target pixels and skin feature representations for those target pixels; the skin feature representations include at least one of texture feature representations and semantic feature representations for the target pixels.

[0091] The target pixel is the pixel in the biometric image whose skin attention weight meets a preset condition.

[0092] In some embodiments, based on the intensity of the biopulse signal in the physiological signal feature representation, the skin attention feature representation corresponding to the target pixel in the biofeature image is obtained based on the skin attention mechanism.

[0093] Skin attention feature representation is used to characterize active regions of bio-pulse signals in bio-images, such as the distribution of bio-pulse signal intensity. Skin attention feature representation can include feature representations (e.g., skin attention weights, or skin feature representations such as texture feature representations and / or semantic feature representations) of pixels in active regions (or important regions, key regions) of bio-pulse signals in the bio-image. The intensity of the bio-pulse signal corresponding to the pixels in this active region satisfies predetermined conditions.

[0094] In some embodiments, the skin attention feature representation is obtained by analyzing the physiological signal feature representation through a pre-trained skin attention recognition model.

[0095] In some embodiments, the skin attention recognition model analyzes the intensity of the bio-pulse wave signal in the physiological signal feature representation and converts the intensity of the bio-pulse wave signal into skin attention weights to obtain a skin attention feature representation. The intensity of the bio-pulse wave signal is used to express the signal intensity corresponding to the bio-pulse wave signal expressed by a pixel in the bio-feature image. In some embodiments, the closer the pixel is to the pulse region where blood exerts pressure on the arterial wall, the stronger the bio-pulse wave signal. In some embodiments, when converting the intensity of the bio-pulse wave signal into skin attention weights, the bio-pulse wave signal can be converted according to a preset mapping relationship to obtain the skin attention weights corresponding to the pixel or pixel block, thereby obtaining the skin attention feature representation. In some embodiments, the skin attention feature representation may also include: the skin attention weights corresponding to the pixel or pixel block and a skin feature representation; the skin feature representation includes at least one of the texture feature representation and image semantic feature representation of the bio-feature image extracted according to the skin attention weights. Extraction according to skin attention weights refers to selecting pixels whose skin attention weights meet preset conditions as target pixels for feature extraction based on the skin attention weights of different pixels. In some embodiments, the skin attention feature representation may also include feature vectors corresponding to the skin attention weights of pixels or pixel blocks in the biometric image.

[0096] In some embodiments, generating a skin attention feature representation corresponding to the biometric image based on the intensity of the biopulse signal in the physiological signal feature representation includes:

[0097] Based on the intensity of the biopulse signal in the physiological signal feature representation, skin attention weights are generated for each pixel in the biofeature image, and a skin mask image corresponding to the biofeature image is generated. In the skin mask image, pixels belonging to the biofeature region are set as the first pixel value, and pixels outside the biofeature region are set as the second pixel value. Based on the skin mask image, the biofeature region in the biofeature image is determined, and the skin attention feature representation corresponding to the biofeature region is generated.

[0098] By using skin mask images, we can initially distinguish skin-related regions from non-skin-related regions in biometric images. This allows us to focus the recognition region representing physiological signal features, as well as the attention mechanism and skin feature extraction, on the skin-related regions, avoiding the influence of non-skin-related regions and improving computational efficiency, accuracy, and anti-interference capabilities.

[0099] Taking biometric images, including face images, as an example, pixels in the skin mask image that belong to the face area are set to the first pixel value, such as 0; pixels in the skin mask image that belong to the area outside the face are set to the second pixel value, such as 255.

[0100] Schematic, Figure 4 shows a comparison diagram of biometric images and skin mask images provided in some embodiments of this application. As shown in Figure 4, the face image 410 includes face regions and non-face regions (such as hair regions, background regions, etc.). After representing the physiological signal features based on skin attention recognition, the face image 410 can be converted into a skin mask image 420. In the skin mask image 420, the face regions are represented by the first pixel value, and the non-face regions are represented by the second pixel value.

[0101] In other embodiments, the skin mask image corresponding to the biometric image can also be obtained by analyzing the physiological signal feature representation and the biometric image based on the skin attention mechanism. That is, the skin mask image corresponding to the biometric image is obtained by combining and analyzing the physiological signal feature representation and the biometric image.

[0102] After obtaining the skin mask image, the biometric regions in the biometric image can be determined. Then, the skin attention weights corresponding to the pixels in the biometric regions can be obtained. The skin attention weights are used to express the importance of the pixels in the biometric regions. Target pixels whose skin attention weights meet the preset weight requirements can be obtained, and the skin attention feature representations corresponding to the target pixels can be obtained.

[0103] In some embodiments, the skin attention feature representation may include the skin attention weights of the target pixels, that is, the skin attention weights corresponding to the target pixels can be used as the skin attention feature representation of the target pixels. Alternatively, the skin attention feature representation may include the skin feature representation and skin attention weights of the target pixels. Specifically, at least one of the texture feature representation and image semantic feature representation of the target pixels is extracted as the skin feature representation, and the skin attention weights corresponding to the target pixels are obtained, and the skin attention weights are used as the subsequent fusion weights of the skin feature representations.

[0104] By using skin attention weights to select target pixels with stronger physiological signal representations from the biometric region, the recognition area is concentrated in the region related to the target pixel during the liveness detection process. This reduces the amount of physiological signal feature representations to be recognized during liveness detection and improves recognition efficiency.

[0105] In some embodiments, a pre-trained skin attention recognition model identifies physiological signal feature representations to obtain skin attention weights corresponding to pixels in the biometric image. Pixels in the biometric image can be assigned a first pixel value and a second pixel value (i.e., divided into biometric and non-biometric regions) based on their skin attention weights, thereby determining the biometric region, such as the face region. Illustratively, when the skin attention weight reaches a first weight threshold, the pixel is assigned the first pixel value, indicating that the pixel is within the biometric region; when the skin attention weight is less than the first weight threshold, the pixel is assigned the second pixel value, indicating that the pixel is outside the biometric region. After obtaining the pixels within the biometric region, target pixels whose skin attention weights meet preset weight requirements are extracted from the biometric region, thereby extracting the skin attention feature representation corresponding to the target pixels.

[0106] Based on skin attention weights, the location of biometric regions in biometric images is divided. Pixels with higher predicted skin attention weights are designated as pixels within biometric regions, while pixels with lower skin attention weights are designated as pixels outside biometric regions. This improves the efficiency of dividing biometric regions from biometric images. By expressing the probability that a pixel belongs to a skin region using skin attention weights, the accuracy of subsequent target pixel selection and liveness detection is improved.

[0107] When acquiring target pixels that meet the preset weight requirements, at least one of the following methods is included:

[0108] First, the n pixels with the highest skin attention weight in the biometric region are obtained as target pixels, where n is a positive integer.

[0109] Where n is a preset value, or n is a value determined based on the total number of pixels in the biometric region.

[0110] When n is a value determined based on the total number of pixels in the biometric region, in some embodiments, the value of n is a preset proportion of the total number of pixels in the biometric region. For example, 25% of the total number of pixels in the biometric region is the value of n, that is, the top 25% of pixels with the highest skin attention weight in the biometric region are obtained as target pixels.

[0111] Alternatively, the value of n can be determined based on the numerical range of the total number of pixels within the biometric region. For example, when the total number of pixels within the biometric region is between 501 and 600, the value of n is 120; when the total number of pixels within the biometric region is between 601 and 700, the value of n is 160. The numerical range of the total number of pixels and the corresponding value of n are predetermined.

[0112] The number of target pixels is determined based on a ratio or the total number of pixels, which improves the accuracy of target pixel selection and avoids the problem that too many or too few target pixels will affect the accuracy of liveness detection, thus improving the efficiency and accuracy of liveness detection.

[0113] Second, the target pixels are obtained by acquiring the pixels in the biometric region whose skin attention weight reaches a preset weight threshold.

[0114] In some embodiments, the preset weight threshold is greater than the first weight threshold mentioned above.

[0115] In some embodiments, the preset weight threshold and the first weight threshold conform to a preset proportional relationship; for example, the preset weight threshold is 1.25 times the first weight threshold. Alternatively, the preset weight threshold and the first weight threshold conform to a preset difference relationship; for example, the preset weight threshold is 0.3 greater than the first weight threshold. This application does not limit the relationship between the preset weight threshold and the first weight threshold.

[0116] After determining the biometric region based on the first weight threshold, the target pixel is then determined based on the preset weight threshold.

[0117] Among them, the target pixel is the pixel in the key area that represents the biological pulse signal during the liveness detection process. Therefore, the liveness detection process of biometric images is based on the skin attention features of the target pixel.

[0118] Target pixels are selected from the biometric region based on a preset weight threshold, thereby ensuring that the target pixels determined from the biometric region meet the requirements of skin attention weight and improving the accuracy of subsequent liveness detection.

[0119] Step 340: The physiological signal feature representation and the skin attention feature representation are fused to obtain the fused feature representation, and the liveness recognition result of the biometric image is obtained based on the fused feature representation.

[0120] In some embodiments, the fusion of physiological signal feature representation and skin attention feature representation includes:

[0121] 1. Feature splicing: The skin attention feature representation and the physiological signal feature representation are spliced ​​together to obtain the fused feature representation;

[0122] 2. Feature addition: The skin attention feature representation and the physiological signal feature representation are added element by element to obtain the fused feature representation;

[0123] 3. Feature averaging: The skin attention feature representation and the physiological signal feature representation are averaged element by element to obtain the fused feature representation;

[0124] 4. Feature stacking: The skin attention feature representation and the physiological signal feature representation are stacked to obtain a fused feature representation;

[0125] 5. Based on deep learning fusion, the skin attention feature representation and physiological signal feature representation are input into a pre-trained convolutional neural network (or convolutional layer) for feature fusion to obtain the fused feature representation.

[0126] After fusing physiological signal feature representation and skin attention feature representation to obtain fused feature representation, liveness detection is performed on the fused feature representation to obtain liveness detection results of biometric images.

[0127] Liveness detection results are used to indicate whether the captured object in a biometric image is a living person. In some embodiments, the obtained liveness detection results can be applied to downstream tasks, such as identity verification, object recognition, video screening, etc.

[0128] In some embodiments, the above-mentioned liveness detection result includes: the collected object is a live object, or the collected object is a fraudulent object, that is, the collected object is not a real live object.

[0129] In some embodiments, when the liveness detection result indicates that the subject being collected is a fraudulent subject, the liveness detection result may also include the type of fraud to which the subject belongs, such as model fraud, mask fraud, headgear fraud, image fraud, etc.

[0130] In summary, the method provided in this application, when performing liveness detection on biometric images, on the one hand, extracts physiological signal feature representations of the biometric images, which represent the bio-pulse signals contained in the biometric images in vector form; on the other hand, performs skin attention analysis on the physiological signal feature representations, so that liveness detection focuses on the active areas of physiological signals in the biometric images, emphasizing the identification of the spatial distribution of bio-pulse signals, improving the accuracy of liveness detection from the image dimension, reducing the dependence on the identification of bio-pulse signals in the temporal dimension, thereby improving the efficiency of liveness detection of biometrics.

[0131] In some embodiments, the fusion of the above-mentioned physiological signal feature representation and skin attention feature representation can be a weighted fusion. Figure 5 is a flowchart of a biometric identification method provided in some embodiments of this application. The method is executed by a computer device, which includes at least one of a terminal and a server. Taking the method being executed by a server as an example, as shown in Figure 5, step 340 can be implemented as steps 342 to 344.

[0132] Step 342: The physiological signal feature representation and the skin attention feature representation are fused to obtain the fused feature representation.

[0133] In some embodiments, the skin attention weights corresponding to the target pixels in the biometric region are obtained. The skin attention weights are used to express the importance of the target pixels in the biometric region. The skin attention weights are used as the weights of the skin feature representation. The weighted skin feature representation is fused with the physiological signal feature representation to obtain the fused feature representation.

[0134] In some embodiments, a pre-trained skin attention recognition model identifies physiological signal feature representations to obtain skin feature representations and skin attention weights corresponding to target pixels. The skin attention weights are weighted data obtained by recognizing the intensity of the bio-pulse signal expressed in the physiological signal feature representation. For example, skin attention weights can be obtained by mapping the intensity of the bio-pulse signal expressed in the physiological signal feature representation according to a preset mapping relationship. In some embodiments, the stronger the bio-pulse signal, the greater the skin attention weight corresponding to the pixel. Specifically, the closer the pixel is to the pulse region where blood exerts pressure on the arterial wall, the stronger the bio-pulse signal. Alternatively, the skin attention recognition model is trained to assign high weights to preset bio-features and low weights to other bio-features. For example, in a face recognition scenario, the skin attention recognition model is trained to assign high weights to the cheek region and low weights to the eye sockets, lips, etc., so that when recognizing physiological signal feature representations, the skin attention recognition model predicts the positions of the cheek region, eye socket region, and lip region, and assigns corresponding skin attention weights to pixels at those positions.

[0135] In some embodiments, a pre-trained skin attention recognition model identifies physiological signal feature representations and biometric images to obtain skin feature representations and skin attention weights corresponding to each pixel. In some embodiments, the skin feature representation is a feature representation extracted from the biometric image by the skin attention recognition model, and the skin attention weight is weight data obtained by recognizing the bio-pulse signal expressed in the physiological signal feature representation.

[0136] The embodiments of this application do not limit the method of generating skin feature representation and skin attention weight.

[0137] Skin attention weights are used as fusion weights to fuse skin feature representations and physiological signal feature representations.

[0138] In some embodiments, the skin feature representation is a feature representation obtained for a target pixel in a biometric region, and a weighted fusion is performed on the skin feature representation and the physiological signal feature representation corresponding to the target pixel. In some embodiments, the fusion process is not performed on pixels other than the target pixel, or pixels other than the target pixel are discarded.

[0139] In some embodiments, skin attention weights are used as weights for the skin feature representation, and the weighted skin feature representation is fused with the physiological signal feature representation to obtain a fused feature representation. In some embodiments, the weighted skin feature representation and the physiological signal feature representation are concatenated to obtain a fused feature representation; or, the skin feature representation and the physiological signal feature representation are implemented as feature vectors of the same dimension, and the skin feature representation of the target pixel is weighted using skin attention weights to obtain a weighted feature representation, which is then added to the physiological signal feature representation to obtain a fused feature representation.

[0140] Alternatively, in some embodiments, when skin attention weight is used as the skin attention feature representation, the feature vectors corresponding to the target pixels in the physiological signal feature representation can be weighted according to the skin attention weight to obtain the fused feature representation. The weighting calculation method includes product calculation, summation calculation, etc., which are not limited in this application embodiment.

[0141] By fusing feature representations, physiological signal feature representations are focused on key regions of biological pulse signals. Vital identification based on fused feature representations improves the efficiency and accuracy of live identification of biological features.

[0142] By fusing physiological signal feature representation and skin attention feature representation, the skin feature representation with higher skin attention weight is fused with the physiological signal feature representation with higher weight, resulting in a fused feature representation that better meets the requirements of the analysis focus, thus improving the accuracy of liveness detection.

[0143] The embodiments of this application do not limit the fusion method of skin attention feature representation and physiological signal feature representation.

[0144] Figure 6 is a schematic diagram of skin attention for physiological signals provided in some embodiments of this application. As shown in Figure 6, the face image 610 includes a face region and a non-face region (such as a hair region, a background region, etc.). After representing the physiological signal features based on skin attention analysis, the face image 610 is converted into a skin mask image 620. In the skin mask image 620, the face region is set as the first pixel value, and the non-face region is set as the second pixel value. After fusing the skin attention feature representation, a physiological signal image 630 is obtained. In the physiological signal image 630, region 631 represents the pulse region range where blood exerts pressure on the arterial wall, and region 632 represents the region range where blood flows.

[0145] Step 344: Perform liveness detection on the fused feature representation to obtain the liveness detection result of the biometric image.

[0146] In some embodiments, a physiological signal recognition model is used to perform liveness recognition on the fused feature representation to obtain liveness recognition results from the biometric image.

[0147] In some embodiments, the method of performing liveness detection on the fused feature representation may include at least one of the following:

[0148] 1. Analyze the fusion feature representation of biometric images on an image frame-by-frame basis to obtain the liveness recognition results of biometric images;

[0149] In some embodiments, the physiological signal recognition model can perform liveness detection on the fused feature representation of a single-frame biometric image to obtain a liveness detection result of the biometric image, indicating whether the sampled object in the biometric image is a live body.

[0150] In some embodiments, the physiological signal recognition model can also determine the final liveness detection result of the acquired object based on the liveness detection results corresponding to at least two image frames in the biometric video. The final liveness detection result is used to indicate whether the acquired object is alive. Illustratively, when the number of image frames indicating that the acquired object is alive reaches a first threshold, the final liveness detection result is determined to be that the acquired object is alive; when the number of image frames indicating that the acquired object is not alive reaches a second threshold, the final liveness detection result is determined to be that the acquired object is not alive.

[0151] In some embodiments, when the physiological signal recognition model performs liveness detection on an image frame-by-frame basis, it can determine the liveness detection result of the biometric image based on the matching relationship between the distribution of the fused feature representation and a preset distribution rule. Illustratively, during the training process, the physiological signal recognition model learns the characteristics of the liveness image frames, including: the difference between the maximum and minimum values ​​of the physiological signal feature representation reaching a difference threshold, the average value of the physiological signal feature representation reaching an average threshold, and the distribution area of ​​the maximum value of the physiological signal representation. The physiological signal recognition model compares the fused feature representation with the learned characteristics of the liveness image frames to obtain the recognition result corresponding to the biometric image.

[0152] The system directly identifies the liveness detection results of biometric images on an image frame-by-frame basis, and combines the liveness detection results of at least two image frames to obtain the final liveness detection result of the acquired object. It identifies whether the acquired object is alive from the dimension of image frames, avoiding reliance on long video segments to identify whether the acquired object is alive, thus improving the efficiency of liveness detection.

[0153] 2. Using video as a unit, analyze the fusion feature representations corresponding to multiple image frames sampled continuously or at predetermined time intervals in the biometric video to obtain the liveness recognition results of the biometric video.

[0154] In some embodiments, the fused feature representations corresponding to multiple image frames in a biometric video are identified to obtain a liveness detection result for the acquired object. The liveness detection result is used to indicate whether the acquired object is alive. The multiple image frames can be continuous in time or conform to a set interval.

[0155] In some embodiments, when the physiological signal recognition model performs liveness detection on a per-video-frame basis, it determines the recognition result of the biometric video based on the matching relationship between the changes in the fused feature representation across different image frames and preset change rules. Illustratively, during training, the physiological signal recognition model learns the characteristics of liveness image frames, including: the positional variation patterns of physiological signal feature representations across consecutive image frames, and the variation period of physiological signal feature representations across consecutive image frames. The physiological signal recognition model compares the fused feature representation with the learned characteristics of the liveness image frames to obtain the liveness detection result corresponding to the biometric video.

[0156] Using video as a unit, the changes between fused feature representations of consecutive image frames are identified. Since the fused feature representation is based on the skin attention feature representation and is a feature representation corresponding to the target pixel in the biometric region, the amount of comparison data between the fused feature representations is small, which improves the efficiency of liveness detection.

[0157] In some embodiments, the physiological signal recognition model includes a classifier to classify the fused features, thereby obtaining a liveness detection result for the fused features. The classifier can be implemented as at least one of the following:

[0158] 1. Classifiers based on linear models, such as linear regression classifiers, logistic regression classifiers, support vector machines (SVM), etc.

[0159] 2. Decision tree-based classifiers, such as Decision Tree Classifier, Random Forest Classifier, Gradient Boosting Tree (GBT) Classifier, etc.

[0160] 3. Neural network-based classifiers, such as multilayer perceptrons (MLP), convolutional neural networks, recurrent neural networks, etc.

[0161] 4. Nearest neighbor-based classifiers, such as Nearest Neighbor Classifier, K-Nearest Neighbors (KNN), etc.

[0162] 5. Reinforcement learning-based classifiers, such as Q-learning-based classifiers and deep Q-networks.

[0163] In summary, the method provided in this application, when performing liveness detection on biometric images, on the one hand, extracts physiological signal feature representations of the biometric images, wherein the physiological signal feature representations characterize the bio-pulse signals exhibited in the biometric images in the form of feature vectors; on the other hand, performs skin attention analysis on the physiological signal feature representations, so that liveness detection focuses on the active areas of bio-pulse signals in the biometric images, emphasizing the analysis of the spatial distribution of bio-pulse signals, improving the accuracy of liveness detection from the image dimension, reducing the dependence on the analysis of bio-pulse signals in the temporal dimension, thereby improving the efficiency of liveness detection of biometrics.

[0164] The method provided in this application focuses physiological signal feature representation on the region with strong biological pulse signal by fusing feature representation. By performing liveness recognition on the fused feature representation, the efficiency and accuracy of liveness recognition of biological features are improved.

[0165] In some embodiments, physiological signal feature representations are obtained through a physiological signal recognition model, and skin attention feature representations are obtained through a skin attention recognition model. Figure 7 is a schematic diagram of the overall process of liveness detection provided in some embodiments of this application. As shown in Figure 7, taking the application of this liveness detection process to a face recognition process as an example, the process includes the following steps.

[0166] Step 710: Obtain the face image.

[0167] A face image is image data obtained by capturing a target object using an image acquisition device. In some embodiments, a face image is an image frame from video data captured by the image acquisition device. The image acquisition device may include at least one of a monocular camera, a binocular camera, a depth camera, an infrared camera, or a video camera.

[0168] Step 720, Preprocessing.

[0169] In some embodiments, the preprocessing operation includes at least one of the following processing methods: image format conversion, cropping, noise reduction, contrast enhancement, etc.

[0170] Step 730: Analyze using a physiological signal recognition model.

[0171] The preprocessed face image is input into the physiological signal recognition model, and the physiological signal feature representation of the face image is extracted by the physiological signal recognition model. The physiological signal feature representation represents the biological pulse signal contained in the face image in the form of feature vector.

[0172] In some embodiments, by analyzing the RGB channel signals in multiple consecutive frames of face images, the blood volume pulse wave signal related to heartbeat, i.e., the biological pulse signal, is extracted and converted into a feature vector to obtain a physiological signal feature representation.

[0173] Step 740: Analyze using the skin attention recognition model.

[0174] The physiological signal feature representation extracted by the physiological signal recognition model is input into the skin attention recognition model, which then extracts the skin feature representation and skin attention weights of the face image. Before extracting the skin feature representation, the face image can be divided into face regions and non-face regions, allowing feature extraction to be performed only on pixels within the face regions.

[0175] For example, skin attention weights are obtained in units of pixels or pixel blocks, and the face image is converted into a mask image based on the skin attention weights. Pixels or pixel blocks belonging to the face region are set to the first value, and pixels or pixel blocks not belonging to the face region are set to the second value, thereby distinguishing between the face region and the non-face region.

[0176] After obtaining the skin attention weights of the face region, a preset number of target pixels with the highest skin attention weights in the face region are obtained, and the skin feature representations of the target pixels are obtained. The skin feature representations of the target pixels and the skin attention weights are then input into the physiological signal recognition model.

[0177] A physiological signal recognition model is used to weight and fuse the skin feature representation and physiological signal feature representation of the target pixel to obtain a fused feature representation. Specifically, the skin feature representation is fused with the physiological signal feature representation according to skin attention weights. Alternatively, the skin feature representation and physiological signal representation are weighted and fused according to preset weights.

[0178] Step 750: Obtain the classification results.

[0179] By analyzing the fused feature representation through a physiological signal recognition model, the liveness classification result of the face image is obtained. This liveness classification result is used to indicate whether the captured object in the face image is a live person.

[0180] The aforementioned physiological signal recognition model and skin attention recognition model require training with sample biological images during the training phase. Figure 8 is a flowchart of a biometric recognition method provided in some other embodiments of this application. This method is executed by a computer device, which includes at least one of a terminal and a server. Taking the method being executed by a server as an example, as shown in Figure 8, the training process of the aforementioned physiological signal recognition model and skin attention recognition model during the training phase includes the following steps.

[0181] Step 810: Obtain biological images of the sample.

[0182] Biological images of samples can be images obtained from sample objects.

[0183] Biological sample images can also be image frames from biometric videos of sample objects. These images include the biological features of the sample object, such as: the sample object's face region, palm print region, fingerprint region, or iris region.

[0184] Step 820: Extract the sample physiological signal feature representation of the sample biological image through the physiological signal recognition model, and obtain the reference physiological signal feature representation corresponding to the sample biological image.

[0185] Sample physiological signal feature representation is the feature representation of biological pulse signals exhibited in sample biological images predicted by the physiological signal recognition model; it can also be called predicted physiological signal feature representation.

[0186] Reference physiological signal feature representation is data used to express biological pulse signals in sample biological images, such as real (or near-real) biological pulse signal data. It can also be called reference fusion feature representation and is used to supervise the training of physiological signal recognition models. In some embodiments, a teacher model is obtained, and the physiological signal feature representation of the sample biological image is extracted through the teacher model as the reference physiological signal feature representation. That is, pseudo-labels are generated through the teacher model.

[0187] The teacher model is a pre-trained machine learning model used to predict the physiological signal feature representation of sample objects in the sample video.

[0188] In some embodiments, the teacher model described above can be implemented using neural networks such as convolutional neural networks, feedforward neural networks, residual networks, and converters, without specific limitations.

[0189] In some embodiments, the teacher model is implemented as an rPPG-Estimate Teacher Model, the output of which is an rPPG signal, which is used as a pseudo-label for training the physiological signal recognition model.

[0190] By guiding the physiological signal recognition model through a teacher prediction model to learn key information that combines with image features during the generation of physiological signal feature representations, the training effect of the physiological signal recognition model can be improved, thereby enhancing the accuracy and efficiency of liveness detection.

[0191] The teacher model is a model trained using a physiological signal dataset.

[0192] To illustrate, the physiological signal dataset is a large dataset collected in advance, which includes a large amount of physiological signal data labeled with physiological tags; supervised training is performed on the pre-built model using the physiological signal dataset until the model loss converges to obtain the teacher model.

[0193] Step 830: Based on the physiological signal feature representation of the sample, a sample skin mask image corresponding to the sample biological image is generated through a skin attention recognition model.

[0194] In some embodiments, the skin attention recognition model can be used to analyze the physiological signal feature representation of the sample to obtain the sample skin mask image corresponding to the sample biological image. The sample skin mask image is predicted by the skin attention recognition model based on the physiological signal feature representation of the sample, and can also be called the predicted skin mask image.

[0195] The sample skin mask image is used to divide the sample biological image into biological and non-biological regions, such as face regions and non-face regions. Pixels belonging to biological regions in the sample skin mask image are assigned the first pixel value, and pixels outside the biological regions are assigned the second pixel value.

[0196] In some embodiments, the skin attention recognition model analyzes the physiological signal feature representation of the sample (e.g., analyzing the intensity of the biological pulse signal corresponding to different pixels in the physiological signal feature representation of the sample) to obtain the attention weights corresponding to each pixel in the sample biological image, thereby obtaining the sample skin mask image corresponding to the sample biological image. In some embodiments, the sample skin mask image is determined based on the attention weights corresponding to each pixel, for example: pixels with attention weights reaching a first weight threshold are regarded as pixels within the biofeature region and set as the first pixel value, while pixels with attention weights less than the first weight threshold are regarded as pixels outside the biofeature region and set as the second pixel value.

[0197] Step 840: Train the physiological signal recognition model.

[0198] In some embodiments, step 840 may include: obtaining a sample physiological signal feature representation adjusted based on the skin attention mechanism, and training a physiological signal recognition model based on a first loss between the adjusted sample physiological signal feature representation and a reference physiological signal feature representation.

[0199] The physiological signal feature representation of the sample can be a physiological signal feature representation obtained based on sampling from a sample skin mask image. In some embodiments, the m pixels with the highest attention weights in the sample skin mask image can be obtained as target pixels, where m is a positive integer. In some embodiments, obtaining the sample physiological signal feature representation adjusted based on the skin attention mechanism can include: weighting the sample physiological signal feature representation based on the attention weights of the m pixels. Then, the loss between the attention-weighted sample physiological signal feature representation of the m pixels and the reference physiological signal feature representation can be calculated as a first loss, and the physiological signal recognition model can be trained based on the calculated first loss.

[0200] In some embodiments, obtaining the sample physiological signal feature representation adjusted based on the skin attention mechanism may also include: extracting the sample skin feature representation corresponding to the sample physiological signal feature representation through a skin attention recognition model; fusing the sample physiological signal feature representation and the sample skin feature representation of m pixels based on attention weights to obtain the adjusted sample physiological signal feature representation (also referred to as the predicted physiological signal feature representation). Then, the loss between the fused feature representation (i.e., the adjusted sample physiological signal feature representation) and the reference physiological signal feature representation can be calculated as the first loss. In some embodiments, the reference physiological signal feature representation may be obtained after a fusion operation, for example, fusing the sample physiological signal feature representation and the reference physiological signal feature representation of m pixels based on attention weights to obtain a fused reference physiological signal feature representation, which is used to calculate the first loss with the adjusted sample physiological signal feature representation.

[0201] In some embodiments, step 840 may include: training a physiological signal recognition model based on a second loss between the sample skin attention feature representation and the reference skin attention feature representation.

[0202] Specifically, the sample skin attention feature representation corresponding to the sample physiological signal feature representation is obtained. The sample skin attention feature representation is used to characterize the active region of biological pulse signal in the sample biological image. In some embodiments, the sample skin attention feature representation is obtained based on skin attention analysis of the sample physiological signal feature representation. A reference skin attention feature representation is obtained, which is data used to characterize the active region of biological pulse signal in the sample biological image, such as the distribution of biological pulse signal intensity. A second loss between the sample skin attention feature representation and the reference skin attention feature representation is calculated to train the physiological signal recognition model.

[0203] In some embodiments, step 840 may include: jointly supervising the training of a physiological signal recognition model based on the first loss and the second loss described above. Illustratively, this refers to the sample skin attention feature representation F output by the skin attention recognition model. s and reference skin attention feature representation F gt The second loss is calculated using the mean squared error, and the loss function is shown in Formula 1 below:

[0204] Formula 1:

[0205] Where M represents the sample skin attention feature representation F s and reference skin attention feature representation F gt The number of elements in The F represents the attention features of the sample skin. s The j-th element in F represents the reference skin attention feature. gt The j-th element in L. MSE This indicates the second loss.

[0206] Schematic illustration of the fusion feature representation (i.e., the adjusted sample physiological signal feature representation) R pre (That is, the predicted value of the rPPG signal feature representation) and the reference physiological signal feature representation R gt (That is, the reference value for the characteristic representation of the rPPG signal) The first loss is calculated using the Pearson correlation coefficient, and the loss function is shown in Formula 2 below:

[0207] Formula 2:

[0208] Where K represents the number of sampling points in the fused feature representation. and They are K R pre and K R gt The mean.

[0209] The fusion feature representation R pre The feature representation of the k-th sampling point in the data. The reference physiological signal characteristics represent R. gt The feature representation of the k-th sampling point in the dataset.

[0210] In some embodiments, combining Formula 1 and Formula 2 above, the overall loss function is obtained as shown in Formula 3 below:

[0211] Formula 3: L total =αL MSE +βL PCC

[0212] Here, α and β are hyperparameters used to balance the importance of the two loss functions. They can be flexibly adjusted according to needs (e.g., α and β are both 1).

[0213] The physiological signal recognition model is trained with the goal of reducing the overall loss function.

[0214] Step 850: Obtain the reference skin mask image.

[0215] A reference skin mask image is an image used to distinguish pixels within and outside biometric regions.

[0216] Given the high uncertainty in the distribution of physiological feature signals (such as rPPG signals) in biometric regions (such as the face region), supervision can be performed only on non-biometric regions (such as non-facial regions).

[0217] To illustrate, taking a face recognition scenario as an example, the reference skin mask image is a mask image extracted by face detection software. To illustrate, the face detection software extracts the reference skin mask image through the following steps:

[0218] Step 1: Facial localization: The key points of the facial bounding box are detected by the face detector. The key points are the vertices of the facial bounding box.

[0219] Schematic, Figure 9 is a schematic diagram of a facial image provided in some embodiments of this application. As shown in Figure 9, key points of the facial bounding box 900 are detected by a face detector, including four vertex key points A, B, C, and D. In some embodiments, the facial bounding box is a box of a preset shape, such as a rectangular box with a preset ratio, or a square box.

[0220] Step 2: Region of Interest (ROI) Extraction: Multiply the side length of the face bounding box by a preset ratio to obtain the side length of the ROI. Extract the ROI by taking the midpoint of the diagonal of the face bounding box as the center of the ROI.

[0221] As an illustration, with a preset ratio of 1.2, the side length of the face bounding box is multiplied by 1.2 to obtain the side length of the ROI, as shown in Figure 9. The side length of the face bounding box 900 is multiplied by 1.2 to obtain the side length of the ROI. The ROI region is extracted according to the side length of the ROI, with the midpoint of the AD line segment as the center, and used as the input of the recognition model.

[0222] Step 3: Map the facial ROI onto a square of a preset size.

[0223] Indicatively, the facial ROI is mapped onto a square with dimensions of 128.

[0224] Step 4: Data Augmentation: Preprocess the facial ROI area, including noise reduction, contrast enhancement, and other preprocessing operations.

[0225] Step 5: Binarization of the facial region. In some embodiments, the 81-point facial marker detector in the machine learning algorithm library (Dlib) is used to extract the facial region, including the forehead, and the facial region is converted into a binary facial mask by binarization as a reference skin mask image.

[0226] Step 860: Train the skin attention recognition model based on the third loss between the sample skin mask image and the reference skin mask image.

[0227] As an illustration, the loss function for the third loss is shown in Formula 4 below:

[0228] Formula 4:

[0229] Where N represents the number of pixels in the non-facial region. This represents the i-th pixel value of the non-facial region binary mask (referencing the skin mask image) generated by Dlib. This represents the i-th pixel value of the non-facial region binary mask predicted by the skin attention recognition model.

[0230] The skin attention recognition model is trained based on the third loss obtained from Formula 4.

[0231] It is worth noting that the form of the loss function described above is only an illustrative example. The loss function can also be implemented as at least one of the following: cross-entropy loss function, binary cross-entropy loss function, mean squared error loss function, logarithmic loss function, minimum absolute deviation loss function, etc., without limitation here.

[0232] Figure 10 is a schematic diagram of the loss calculation process provided in some embodiments of this application. As shown in Figure 10, for the sample image 1010, a reference skin mask image 1030 is first extracted by the face marker detector 1020, and then the sample physiological signal feature representation 1040 is extracted by the physiological signal recognition model. The sample skin attention feature representation 1050 is extracted by the skin attention recognition model, and the sample skin mask image 1060 (also called the predicted skin mask image) is predicted. The mask loss 1011 is calculated based on the reference skin mask image 1030 and the sample skin mask image 1060, and the skin attention recognition model is trained by the mask loss.

[0233] In addition, based on the predicted sample skin mask image 1060, the sample skin attention feature representation 1050 of the m target pixels with the highest skin attention is obtained, and the sample skin attention feature representation 1050 of the m target pixels and the sample physiological signal feature representation 1040 of the m target pixels are fused to obtain the fused feature representation. The reference physiological signal feature representation 1070 is also obtained, and the first loss between the fused feature representation and the reference physiological signal feature representation is obtained. The second loss between the sample skin attention feature representation 1050 and the reference skin attention feature representation is obtained, and the sum of the first loss and the second loss is used as the two-level loss 1080 to train the physiological signal recognition model.

[0234] In summary, the method provided in this application, when performing liveness detection on biometric images, on the one hand, extracts physiological signal feature representations of the biometric images, wherein the physiological signal feature representations characterize the bio-pulse signals contained in the biometric images in the form of feature vectors; on the other hand, performs skin attention analysis on the physiological signal feature representations, so that liveness detection focuses on the active areas of bio-pulse signals in the biometric images, emphasizing the analysis of the spatial distribution of bio-pulse signals, improving the accuracy of liveness detection from the image dimension, reducing the dependence on the analysis of bio-pulse signals in the temporal dimension, thereby improving the efficiency of liveness detection of biometrics.

[0235] The method provided in this application improves the accuracy of skin attention analysis by training a skin attention recognition model through the difference between the predicted mask image and a pre-provided reference mask image.

[0236] The method provided in this application improves the accuracy of physiological signal recognition model in extracting physiological signal feature representation by training the physiological signal recognition model through the difference between sample skin feature representation and reference skin feature representation, thereby improving the accuracy of liveness recognition.

[0237] The method provided in this application has the following effect in the field of palmprint recognition, as shown in Table 1.

[0238] Table 1

[0239] The evaluation metric corresponding to the values ​​in the table is the Area Under the Curve (AUC). A higher AUC value indicates better performance of the method.

[0240] The biometric identification method provided in this application can be applied to scenarios such as face recognition, palm print recognition, and fingerprint recognition. The face recognition scenario will be used as an example for illustration.

[0241] Figure 11 is a flowchart of a biometric identification method provided in some embodiments of this application, wherein the biometric feature is implemented as a facial image acquired from the subject. The method is executed by a computer device, which includes at least one of a terminal and a server. Taking the method executed by a server as an example, as shown in Figure 11, the training process of the above-mentioned physiological signal recognition model and skin attention recognition model during the training phase includes the following steps.

[0242] Step 1110: Obtain sample face images.

[0243] A sample face image is a facial image captured from a sample object.

[0244] A sample face image is a frame of facial image from a video captured of the sample object. The sample face image includes the facial region of the sample object.

[0245] Step 1120: Extract the facial pulse wave signal of the sample face image through the physiological signal recognition model, and obtain the reference pulse wave signal corresponding to the sample face image.

[0246] Facial pulse wave signal, or facial pulse wave signal feature representation, is a feature representation of the biological pulse signal exhibited in a sample facial image predicted by a physiological signal recognition model. In some embodiments, the facial pulse wave signal is an rPPG signal obtained by analyzing a sample facial image.

[0247] The reference pulse wave signal, or reference pulse wave signal characteristic representation, is data used to express the biological pulse signal in a sample face image. The reference pulse signal is an rPPG signal used to express the biological pulse signal in a sample face image.

[0248] In some embodiments, a teacher model is obtained, and the pulse wave signal of the sample face image is extracted as a reference pulse wave signal through the teacher model. That is, pseudo-labels are generated through the teacher model.

[0249] The teacher model is a pre-trained machine learning model used to predict the facial pulse wave signal of the sample object in the sample video.

[0250] In some embodiments, the teacher model is implemented as an rPPG-Estimate Teacher Model, the output of which is an rPPG signal, which is used as a pseudo-label for training the physiological signal recognition model.

[0251] The teacher model is a model trained using a physiological signal dataset.

[0252] To illustrate, the physiological signal dataset is a large dataset collected in advance, which includes a large amount of physiological signal data labeled with physiological tags; supervised training is performed on the pre-built model using the physiological signal dataset until the model loss converges to obtain the teacher model.

[0253] Step 1130: Analyze the facial pulse wave signal using the skin attention recognition model to obtain the sample face mask image of the sample face image.

[0254] In the sample face mask image (also known as the predicted face mask image), pixels belonging to the biometric region are set as the first pixel value, and pixels outside the biometric region are set as the second pixel value.

[0255] In some embodiments, facial pulse wave signals are analyzed using a skin attention recognition model to obtain the attention weights corresponding to each pixel in the sample face image, thereby obtaining a sample face mask image corresponding to the sample face image. In some embodiments, the sample face mask image is determined based on the attention weights corresponding to each pixel, such as: pixels with attention weights reaching a first weight threshold are taken as pixels within the face region and a first pixel value is taken; pixels with attention weights less than the first weight threshold are taken as pixels outside the face region and a second pixel value is taken.

[0256] Step 1140: Train the physiological signal recognition model.

[0257] In some embodiments, the facial pulse wave signal can be adjusted based on an attention mechanism to obtain an adjusted facial pulse wave signal, and a physiological signal recognition model can be trained based on a first loss between the adjusted facial pulse wave signal and a reference pulse wave signal.

[0258] The facial pulse wave signal is a physiological signal feature representation obtained based on sampled facial mask images. In some embodiments, the m pixels with the highest attention weights in the sample facial mask image can be obtained as target pixels, where m is a positive integer. The facial pulse wave signal is adjusted based on the attention weights of the m pixels, and the loss between the attention-weighted adjusted facial pulse wave signal and the reference pulse wave signal is used as a first loss. The physiological signal recognition model is then trained based on the calculated first loss. Alternatively, a sample skin feature representation corresponding to the facial pulse wave signal can be extracted using a skin attention recognition model. The m-pixel facial pulse wave signal and the sample skin feature representation are fused based on attention weights, and the resulting fused feature representation is used as the adjusted facial pulse wave signal. The loss between the fused feature representation and the reference pulse wave signal is calculated as the first loss. In some embodiments, the reference pulse signal can be obtained after the following fusion operation: for example, the m-pixel facial pulse wave signal and the original reference pulse wave signal can be fused based on attention weights to obtain a fused feature representation, which is used as the reference pulse wave signal to calculate the first loss.

[0259] In some embodiments, facial pulse wave signals can be analyzed based on skin attention to obtain sample skin attention feature representations and obtain reference skin attention feature representations. The reference skin attention feature representations are pre-acquired data used to characterize the skin attention distribution area in the sample facial image. The physiological signal recognition model is trained based on the second loss between the sample skin attention feature representations and the reference skin attention feature representations.

[0260] In some embodiments, the physiological signal recognition model can also be trained under joint supervision based on the first loss and the second loss described above.

[0261] Step 1150: Obtain the reference face mask image.

[0262] The reference face mask image is a pre-acquired image used to distinguish pixels within and outside the face region.

[0263] The reference face mask image is illustrated here; it is a mask image extracted by face detection software. The face detection software extracts the reference face mask image through the following steps:

[0264] Step 1: Face Localization: Key points of the facial bounding box are detected using a face detector. These key points are the vertices of the bounding box. Step 2: Region of Interest (ROI) Extraction: The side length of the facial bounding box is multiplied by a preset ratio to obtain the side length of the ROI. The ROI is then extracted, with the midpoint of the diagonal of the facial bounding box as its center. Step 3: Mapping the facial ROI onto a square of a preset size. Step 4: Data Augmentation: Preprocessing of the facial ROI region, including noise reduction and contrast enhancement. Step 5: Binarization of the facial region.

[0265] Step 1160: Train the skin attention recognition model based on the third loss between the sample face mask image and the reference face mask image.

[0266] This application does not limit the training order of the skin attention model and the physiological signal recognition model. For example, the skin attention recognition model can be trained first, then its parameters can be fixed before training the physiological signal recognition model. Alternatively, the training can be performed in other orders. After obtaining the physiological signal recognition model and the skin attention recognition model, they are applied to the liveness detection branch of a face recognition scenario. That is, after the physiological signal recognition model and the skin attention recognition model identify that the subject is alive, the face image of the subject is then used for identity verification.

[0267] Figure 12 is a structural block diagram of a biometric identification device provided in some embodiments of this application. As shown in Figure 12, the device includes:

[0268] The acquisition module 1210 is used to acquire a biometric image, wherein the biometric image is an image acquired from the object being acquired;

[0269] The generation module 1220 is used to generate a physiological signal feature representation of the bio-feature image, wherein the physiological signal feature representation is used to characterize the biological pulse signal contained in the bio-feature image in the form of a feature vector;

[0270] The recognition module 1230 is used to generate a skin attention feature representation corresponding to the bio-feature image based on the intensity of the bio-pulse signal in the physiological signal feature representation, wherein the skin attention feature representation is used to characterize the active area of ​​the bio-pulse signal in the bio-feature image;

[0271] The recognition module 1230 is further configured to fuse the physiological signal feature representation and the skin attention feature representation to obtain a fused feature representation, and obtain a liveness recognition result of the biometric image based on the fused feature representation. The liveness recognition result is used to indicate whether the collected object in the biometric image is a live body.

[0272] In some embodiments, as shown in FIG13, the identification module 1230 includes:

[0273] Fusion unit 1231 is used to fuse the physiological signal feature representation and the skin attention feature representation to obtain the fused feature representation;

[0274] The identification unit 1232 is used to perform liveness identification based on the fused feature representation to obtain the liveness identification result of the biometric image.

[0275] It should be noted that the biometric identification device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the biometric identification device provided in the above embodiments belongs to the same concept as the biometric identification method embodiments, and its specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0276] This application also provides a computer device including a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the biometric identification method provided in the above-described method embodiments. It should be noted that the computer device may be the one shown in Figure 14 below.

[0277] Please refer to Figure 14, which shows a schematic diagram of the structure of a computer device provided in some embodiments of this application. Specifically, the computer device 1400 includes a Central Processing Unit (CPU) 1401, a system memory 1404 including Random Access Memory (RAM) 1402 and Read-Only Memory (ROM) 1403, and a system bus 1405 connecting the system memory 1404 and the CPU 1401. The computer device 1400 also includes a basic input / output system (I / O system) 1406 to facilitate the transfer of information between various devices within the computer, and a mass storage device 1407 for storing the operating system 1413, application programs 1414, and other program modules 1415.

[0278] The basic input / output system 1406 includes a display 1408 for displaying information and an input device 1409 for user input, such as a mouse or keyboard. Both the display 1408 and the input device 1409 are connected to the central processing unit 1401 via an input / output controller 1410 connected to the system bus 1405. The basic input / output system 1406 may also include the input / output controller 1410 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1410 also provides output to a display screen, printer, or other types of output devices.

[0279] The mass storage device 1407 is connected to the central processing unit 1401 via a mass storage controller (not shown) connected to the system bus 1405. The mass storage device 1407 and its associated computer-readable media provide non-volatile storage for the computer device 1400. That is, the mass storage device 1407 may include computer-readable media (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0280] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1404 and mass storage device 1407 described above can be collectively referred to as memory.

[0281] The memory stores one or more programs, which are configured to be executed by one or more central processing units 1401. The one or more programs contain instructions for implementing the above-described biometric identification method or machine translation model-based translation method. The central processing unit 1401 executes the one or more programs to implement the biometric identification method or machine translation model-based translation method provided in the above-described method embodiments.

[0282] According to various embodiments of this application, the computer device 1400 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1400 can be connected to a network 1412 via a network interface unit 1411 connected to the system bus 1405, or the network interface unit 1411 can be used to connect to other types of networks or remote computer systems (not shown).

[0283] The memory also includes one or more programs stored in the memory, and the one or more programs include steps executed by a computer device in the biometric identification method provided in the embodiments of this application.

[0284] This application also provides a computer device, which includes a memory and a processor. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads the at least one instruction, at least one program, code set, or instruction set to implement the above-mentioned biometric identification method.

[0285] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-described biometric identification method.

[0286] This application also provides a computer program product that, when run on a computer, causes the computer to execute the biometric identification method provided in the above-described method embodiments.

[0287] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into the terminal. The computer-readable storage medium stores at least one instruction, at least one program segment, a code set, or an instruction set. The at least one instruction, the at least one program segment, the code set, or the instruction set is loaded and executed by the processor to implement the above-described biometric identification method or machine translation-based translation method. In some embodiments, the computer-readable storage medium may include: a read-only memory, random access memory, a solid-state drive (SSD), or an optical disc, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0288] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. The above descriptions are merely optional embodiments of this application and are not intended to limit the application. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A biometric identification method, executed by an electronic device, comprising: Acquire biometric images, wherein the biometric images are images acquired from the subject being collected; Generate a physiological signal feature representation of the bio-image, wherein the physiological signal feature representation is used to characterize the biological pulse signal contained in the bio-image in the form of a feature vector; Based on the intensity of the biopulse signal in the physiological signal feature representation, a skin attention feature representation corresponding to the biofeature image is generated, and the skin attention feature representation is used to characterize the active area of ​​the biopulse signal in the biofeature image; The physiological signal feature representation and the skin attention feature representation are fused to obtain a fused feature representation. Based on the fused feature representation, the liveness recognition result of the biometric image is obtained. The liveness recognition result is used to indicate whether the acquired object in the biometric image is a live body.

2. The method according to claim 1, wherein, The skin attention feature representation includes: The skin attention weights of the target pixels in the biometric image, or The biometric image contains skin attention weights for target pixels and skin feature representations for those target pixels; the skin feature representations include at least one of texture feature representations and semantic feature representations for the target pixels. The target pixel is the pixel in the biometric image whose skin attention weight meets the preset weight requirement.

3. The method according to claim 1 or 2, wherein, The physiological signal feature representation includes: physiological signal feature representation extracted from each pixel in the bio-feature image, or physiological signal feature representation extracted from sampling points obtained from each pixel in the bio-feature image.

4. The method according to any one of claims 1 to 3, wherein, The step of generating a skin attention feature representation corresponding to the bio-feature image based on the intensity of the bio-pulse signal in the physiological signal feature representation includes: Based on the intensity of the biopulse signal in the physiological signal feature representation, skin attention weights are generated for each pixel in the biofeature image. A skin mask image corresponding to the biofeature image is generated according to the skin attention weights. In the skin mask image, pixels belonging to the biofeature region have a first pixel value, and pixels outside the biofeature region have a second pixel value. Based on the skin mask image, determine the biometric regions in the biometric image; Generate the skin attention feature representation corresponding to the biometric region.

5. The method of claim 4, wherein, The generation of the skin attention feature representation corresponding to the biometric region includes: Obtain the skin attention weights corresponding to the pixels in the biometric region, and the skin attention weights are used to express the importance of the pixels in the biometric region; Obtain target pixels whose skin attention weights meet preset weight requirements, and extract the skin attention feature representation corresponding to the target pixels.

6. The method of claim 5, wherein, The acquisition of target pixels whose skin attention weights meet preset weight requirements includes: The n pixels with the highest skin attention weights in the biometric region are selected as the target pixels, where n is a positive integer; or... The target pixel is obtained as the pixel whose skin attention weight in the biometric region reaches a preset weight threshold.

7. The method according to any one of claims 1 to 6, wherein, The skin attention feature representation includes: the skin attention weights of target pixels in the biometric image; The fused feature representation obtained by fusing the physiological signal feature representation and the skin attention feature representation includes: The feature vectors of target pixels in the physiological signal feature representation are weighted according to the skin attention weight to obtain the fused feature representation.

8. The method according to any one of claims 1 to 6, wherein, The skin attention feature representation includes: the skin attention weight of the target pixel in the biometric image, and the skin feature representation of the target pixel; The fusion of the physiological signal feature representation and the skin attention feature representation to obtain the fused feature representation includes: Using the skin attention weight as the weight of the skin feature representation, the skin feature representation and the physiological signal feature representation are weighted and fused to obtain the fused feature representation.

9. The method according to any one of claims 1 to 8, wherein, The physiological signal feature representation used to generate the biometric image includes: The biometric image is input into a pre-trained physiological signal recognition model, and the physiological signal feature representation is generated by the physiological signal recognition model.

10. The method according to claim 9, characterized in that, The biometric image is the first biometric image in the biometric video; The step of generating the physiological signal feature representation through the physiological signal recognition model includes: The physiological signal recognition model identifies the changes in biological pulse signals in at least two image frames of the biofeature video, thereby obtaining the physiological signal feature representation, wherein the at least two image frames include the first biofeature image.

11. The method according to claim 9, wherein, The physiological signal recognition model is a pre-trained model, and the training process of the physiological signal recognition model includes: Acquire biological images of samples, wherein the biological images of samples are images obtained from sample objects; The physiological signal recognition model generates a sample physiological signal feature representation of the sample biological image, and obtains a reference physiological signal feature representation corresponding to the sample biological image. Obtain a physiological signal feature representation of the sample adjusted based on the skin attention mechanism; Calculate the first loss between the adjusted sample physiological signal feature representation and the reference physiological signal feature representation; Obtain the sample skin attention feature representation corresponding to the sample physiological signal feature representation, and the sample skin attention feature representation is used to characterize the active area of ​​the biological pulse signal in the sample biological image; Obtain the reference skin attention feature representation; Calculate the second loss between the sample skin attention feature representation and the reference skin attention feature representation; The physiological signal recognition model is trained based on the first loss and the second loss.

12. The method according to claim 11, wherein, The acquisition of the sample physiological signal feature representation adjusted based on the skin attention mechanism includes: The sample physiological signal feature representation is adjusted based on attention weights to obtain the adjusted sample physiological signal feature representation; or The skin attention recognition model extracts the sample skin feature representation corresponding to the sample physiological signal feature representation. Based on the attention weight, the sample physiological signal feature representation and the sample skin feature representation are weighted and fused to obtain the adjusted physiological signal feature representation.

13. The method according to any one of claims 1 to 12, wherein, The step of generating a skin attention feature representation corresponding to the bio-feature image based on the intensity of the bio-pulse signal in the physiological signal feature representation includes: The physiological signal feature representation generated by the physiological signal recognition model is input into the skin attention recognition model, and the skin attention feature representation is generated based on the skin attention mechanism through the skin attention recognition model.

14. The method according to claim 13, wherein, The skin attention recognition model is a pre-trained model, and the training process includes: Acquire biological images of samples, wherein the biological images of samples are images obtained from sample objects; The physiological signal feature representation of the sample biological image is extracted using the physiological signal recognition model. Based on the physiological signal feature representation of the sample, a sample skin mask image corresponding to the sample biological image is generated, wherein pixels in the sample skin mask image belonging to the biological feature region have a first pixel value, and pixels outside the biological feature region have a second pixel value. Obtain a reference skin mask image, which is an image used to distinguish pixels within and outside the biometric region; The skin attention recognition model is trained based on a third loss between the non-biological feature regions of the sample skin mask image and the non-biological feature regions of the reference skin mask image.

15. The method according to any one of claims 1 to 14, wherein, The biometric image is the first biometric image frame in the biometric video; The method further includes: At least one second biometric image frame is acquired from the biometric image. Based on the liveness detection results corresponding to the first biometric image frame and at least one second biometric image frame, the liveness detection result of the acquired object is obtained. The liveness detection result is used to express whether the acquired object is a live body when the biometric acquisition is completed.

16. The method according to any one of claims 1 to 14, wherein, The biometric image is the first biometric image frame in the biometric video; The method further includes: Acquire multiple biometric image frames from the biometric video, either continuously or at predetermined time intervals, wherein the multiple biometric image frames include the first biometric image frame; Based on the fusion feature representations corresponding to the multiple biometric image frames, the liveness recognition result of the collected object is obtained. The liveness recognition result is used to indicate whether the collected object is alive when the biometric collection is completed.

17. A biometric identification device, comprising: The acquisition module is used to acquire biometric images, which are images acquired from the target object. A generation module is used to generate a physiological signal feature representation of the bio-feature image, wherein the physiological signal feature representation is used to characterize the biological pulse signal contained in the bio-feature image in the form of a feature vector; The recognition module is used to generate a skin attention feature representation corresponding to the bio-feature image based on the intensity of the bio-pulse signal in the physiological signal feature representation, wherein the skin attention feature representation is used to characterize the active area of ​​the bio-pulse signal in the bio-feature image; The recognition module is further configured to fuse the physiological signal feature representation and the skin attention feature representation to obtain a fused feature representation, and to obtain a liveness recognition result of the biometric image based on the fused feature representation. The liveness recognition result is used to indicate whether the collected object in the biometric image is a live body.

18. A computer device comprising a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the biometric identification method as claimed in any one of claims 1 to 16.

19. A computer-readable storage medium storing at least one instruction, the at least one instruction being loaded and executed by a processor to implement the biometric identification method as claimed in any one of claims 1 to 16.

20. A computer program product comprising a computer program or instructions that, when executed by a processor, implement the biometric identification method as described in any one of claims 1 to 16.

Images