Image-based living body identification method, training method and system

Abstract

The invention provides a living body recognition method of a living body based on an image, a training method and system of a living body recognition classifier, electronic equipment, a computer readable storage medium and a computer program product. The recognition method comprises the following steps: generating a plurality of region-of-interest images based on the image; performing feature extraction on the plurality of region-of-interest images to obtain a feature map; the feature map is input into a trained living body recognition classifier, a living body recognition result of the to-be-detected living body is obtained, and the multiple region-of-interest images all comprise the same part of the to-be-detected living body; and the sizes of the same part in the plurality of region-of-interest images are different. According to the method, a plurality of region-of-interest images extracted from the same image are utilized to identify whether a to-be-detected living body is a living body, the plurality of region-of-interest images comprise the same part of the to-be-detected living body, and the sizes of the same part in the plurality of region-of-interest images are different. The diversity of data and the accuracy of living body detection are improved.

Description

Technical Field

[0001] This disclosure relates to the field of information processing technology, specifically to image-based methods for identifying live organisms, training methods for live organism classifiers, systems, electronic devices, computer-readable storage media, and computer program products. Background Technology

[0002] With the rapid development of computer technology and advancements in big data and deep learning technologies within the field of artificial intelligence, breakthroughs have been achieved in biometric recognition technology based on local images of organisms. For example, an increasing number of security products utilize facial recognition technology. Furthermore, the application of facial recognition technology in surveillance cameras, turnstiles, and access control systems can bring significant convenience to people's lives.

[0003] Therefore, how to effectively improve the accuracy of liveness detection of organisms has become one of the research hotspots in the field of information processing technology. Summary of the Invention

[0004] This disclosure presents an image-based method for identifying live organisms, a training method for a live organism classifier, a system, an electronic device, a computer-readable storage medium, and a computer program product, which improves the diversity and utilization of local image data of organisms and enhances the accuracy of live organism detection.

[0005] In a first aspect, this disclosure proposes an image-based method for liveness detection of organisms, comprising: generating multiple regions of interest (ROI) images based on the image; extracting features from the multiple ROI images to obtain feature maps; and inputting the feature maps into a trained liveness detection classifier to obtain a liveness detection result of the organism to be tested, wherein the multiple ROI images all include the same part of the organism to be tested; and the same part has different sizes in the multiple ROI images.

[0006] In one embodiment of this disclosure, before feature extraction from multiple region-of-interest (ROI) images, the method further includes scaling the multiple ROC images so that the multiple ROC images have the same size.

[0007] In one embodiment of this disclosure, before feature extraction from multiple region-of-interest (ROI) images, the method further includes adjusting the resolution of the multiple RIO images to make the resolution of the multiple RIO images the same.

[0008] In one embodiment of this disclosure, generating multiple regions of interest (ROI) images based on an image includes: detecting the image to obtain key points of the parts and bounding boxes of the parts; and generating multiple ROI images based on the bounding boxes and multiple key points, wherein the parts include at least one of the face, the back of the hand, and the palm of the hand.

[0009] In one embodiment of this disclosure, the pixel corresponding to the image center of multiple regions of interest images is the same.

[0010] In one embodiment of this disclosure, the pixel corresponding to the image center is one of the pixels belonging to the part in the region of interest image.

[0011] In one embodiment of this disclosure, before feature extraction from multiple regions of interest (ROI) images, the method further includes: preprocessing the ROI images based on multiple key points, wherein the preprocessing includes at least one of geometric correction processing, optical improvement processing, and smoothing processing of the ROI images.

[0012] In one embodiment of this disclosure, feature extraction of multiple region-of-interest (ROI) images to obtain a feature map includes: fusing the multiple RIO images; and extracting features from the fused RIO images to obtain a feature map.

[0013] In one embodiment of this disclosure, feature extraction of multiple region-of-interest images to obtain a feature map further includes: extracting features from the multiple region-of-interest images respectively to obtain multiple sub-feature maps; and fusing the multiple sub-feature maps to obtain a feature map.

[0014] In one embodiment of this disclosure, feature extraction of multiple regions of interest (ROI) images to obtain feature maps further includes: extracting features from the multiple ROI images respectively to obtain multiple sub-feature maps; and inputting the feature maps into a trained liveness detection classifier to obtain a liveness detection result of the organism to be tested includes: inputting the multiple sub-feature maps into the multiple trained liveness detection classifiers respectively, and determining the liveness detection result of the organism to be tested based on the output results of the multiple trained liveness detection classifiers.

[0015] In one embodiment of this disclosure, multiple trained liveness detection classifiers have different structures.

[0016] In one embodiment of this disclosure, feature extraction of multiple regions of interest (ROI) images to obtain a feature map includes: performing grayscale compression processing on the ROI images to obtain a grayscale square image; and obtaining a feature map based on the grayscale square image.

[0017] Secondly, this disclosure provides a method for training a liveness detection classifier, comprising: generating multiple training region of interest images based on images in a training set, wherein the multiple training region of interest images all include the same part of the same organism, and the same part has different sizes in the multiple training region of interest images; and training a liveness detection classifier based on the multiple training region of interest images.

[0018] In one embodiment of this disclosure, training a liveness detection classifier based on multiple training region-of-interest (ROI) images includes: fusing multiple training ROI images from the same image in the training set; extracting features from the fused training ROI images to obtain a training feature map; and training the liveness detection classifier based on the training feature map.

[0019] In one embodiment of this disclosure, training a liveness detection classifier based on multiple training region-of-interest (ROI) images further includes: extracting features from multiple training ROI images from the same image in the training set to obtain multiple training sub-feature maps; fusing the multiple training sub-feature maps to obtain a training feature map; and training the liveness detection classifier based on the training feature map.

[0020] In one embodiment of this disclosure, the pixel corresponding to the image center of multiple training region of interest images is the same.

[0021] In one embodiment of this disclosure, the pixel corresponding to the image center is one of the pixels belonging to the part in the region of interest image used for training.

[0022] In one embodiment of this disclosure, training a liveness detection classifier based on multiple training region-of-interest images further includes: extracting features from multiple training region-of-interest images from the same image in the training set to obtain multiple training sub-feature maps; and training multiple sub-liveness detection classifiers based on the multiple training sub-feature maps, wherein the multiple sub-liveness detection classifiers are cascaded to form a liveness detection classifier.

[0023] Thirdly, this disclosure proposes an image-based liveness detection system for organisms, comprising: a region of interest (ROI) generation module configured to generate multiple ROI images based on an image; a feature extraction module configured to extract features from the multiple ROI images to obtain a feature map; and a liveness detection module configured to input the feature map into a trained liveness detection classifier to obtain a liveness detection result for the organism to be tested, wherein the multiple ROI images all include the same part of the organism to be tested; and the same part has different sizes in the multiple ROI images.

[0024] In one embodiment of this disclosure, the region of interest generation module includes a scaling unit configured to scale multiple region of interest images so that the multiple region of interest images have the same size.

[0025] In one embodiment of this disclosure, the region of interest generation module further includes a resolution adjustment unit configured to adjust the resolution of multiple region of interest images so that the multiple region of interest images have the same resolution.

[0026] In one embodiment of this disclosure, the region of interest generation module further includes: a localization unit configured to detect the image and obtain key points of the part and a localization box of the part; and a generation unit configured to generate multiple region of interest images based on the localization box and multiple key points, wherein the part includes at least one of the face, the back of the hand, and the palm of the hand.

[0027] In one embodiment of this disclosure, the pixel corresponding to the image center of multiple regions of interest images is the same.

[0028] In one embodiment of this disclosure, the pixel corresponding to the image center is one of the pixels belonging to the part in the region of interest image.

[0029] In one embodiment of this disclosure, the liveness detection system further includes a preprocessing module, wherein the preprocessing module is configured to preprocess the region of interest image based on multiple key points, wherein the preprocessing includes at least one of geometric correction processing, optical improvement processing, and smoothing processing of the region of interest image.

[0030] In one embodiment of this disclosure, the feature extraction module includes: an initial fusion unit configured to fuse multiple regions of interest (ROI) images; and a subsequent extraction unit configured to extract features from the fused ROI images to obtain a feature map.

[0031] In one embodiment of this disclosure, the feature extraction module includes: an initial extraction unit configured to extract features from multiple regions of interest images to obtain multiple sub-feature maps; and a subsequent fusion unit configured to fuse the multiple sub-feature maps to obtain a feature map.

[0032] In one embodiment of this disclosure, the feature extraction module includes a sub-extraction unit configured to extract features from multiple regions of interest images to obtain multiple sub-feature maps; and the liveness detection module includes a sub-detection module configured to input the multiple sub-feature maps into multiple trained liveness detection classifiers and output the results, wherein the liveness detection result of the organism to be tested is determined based on the output results of the multiple trained liveness detection classifiers.

[0033] In one embodiment of this disclosure, multiple trained liveness detection classifiers have different structures.

[0034] In one embodiment of this disclosure, the feature extraction module further includes a grayscale processing unit configured to perform grayscale compression processing on the region of interest image to obtain a grayscale square, wherein a feature map is obtained based on the grayscale square.

[0035] Fourthly, this disclosure provides a training system for a liveness detection classifier, comprising: a data generation module configured to generate multiple training region-of-interest (ROI) images based on images in a training set, wherein the multiple RIO images all include the same part of the same organism, and the same part has different sizes in the multiple RIO images; and a training module configured to train a liveness detection classifier based on the multiple RIO images.

[0036] In one embodiment of this disclosure, the training module includes: a first sub-training module configured to fuse multiple training region of interest images from the same image in the training set; a second sub-training module configured to extract features from the fused training region of interest images to obtain a training feature map; and a third sub-training module configured to train a liveness detection classifier based on the training feature map.

[0037] In one embodiment of this disclosure, the training module further includes: a fourth sub-training module configured to extract features from multiple training region of interest images from the same image in the training set to obtain multiple training sub-feature maps; a fifth sub-training module configured to fuse the multiple training sub-feature maps to obtain a training feature map; and a sixth sub-training module configured to train a liveness detection classifier based on the training feature map.

[0038] In one embodiment of this disclosure, the pixel corresponding to the image center of multiple training region of interest images is the same.

[0039] In one embodiment of this disclosure, the pixel corresponding to the image center is one of the pixels belonging to the part in the region of interest image used for training.

[0040] In one embodiment of this disclosure, the training module further includes: a seventh sub-training module configured to extract features from multiple training region-of-interest images from the same image in the training set to obtain multiple training sub-feature maps; and an eighth sub-training module configured to train multiple sub-liveness recognition classifiers based on the multiple training sub-feature maps, wherein the multiple sub-liveness recognition classifiers are cascaded into a liveness recognition classifier.

[0041] Fifthly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement a liveness detection method as described in any implementation of the first aspect or a training method as described in any implementation of the second aspect.

[0042] In a sixth aspect, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement a liveness detection method as described in any implementation of the first aspect or a training method as described in any implementation of the second aspect.

[0043] In a seventh aspect, embodiments of this disclosure provide a computer program product including a computer program, which, when executed by a processor, can implement the liveness detection method as described in any implementation of the first aspect or the training method as described in any implementation of the second aspect.

[0044] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0045] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0046] Figure 1 This is an exemplary system architecture to which this disclosure can be applied;

[0047] Figure 2 A flowchart illustrating an image-based method for liveness detection of a living organism provided for embodiments of this disclosure;

[0048] Figure 3 A flowchart for generating multiple regions of interest images based on an image, provided for embodiments of this disclosure;

[0049] Figure 4 Images provided for one embodiment of this disclosure;

[0050] Figure 5 A schematic diagram of key points and location frames of a part provided for one embodiment of this disclosure;

[0051] Figure 6A A first region of interest image provided for one embodiment of this disclosure;

[0052] Figure 6B A second region of interest image provided for one embodiment of this disclosure;

[0053] Figure 6C A first region of interest image provided for another embodiment of this disclosure;

[0054] Figure 6D Images provided for another embodiment of this disclosure;

[0055] Figure 6E Images of other regions of interest provided for another embodiment of this disclosure;

[0056] Figure 7 Images provided for another embodiment of this disclosure;

[0057] Figure 8 A schematic diagram of key points and location frames of a part provided for another embodiment of this disclosure;

[0058] Figure 9A A third region of interest image provided for another embodiment of this disclosure;

[0059] Figure 9B A fourth region of interest image provided for one embodiment of this disclosure;

[0060] Figure 10 A flowchart illustrating feature extraction from multiple regions of interest (ROIs) images to obtain a feature map, as provided in one embodiment of this disclosure;

[0061] Figure 11 A schematic flowchart of a method for identifying a living organism provided in one embodiment of this disclosure;

[0062] Figure 12 A flowchart for obtaining a feature map by feature extraction from multiple regions of interest images, provided as another embodiment of this disclosure;

[0063] Figure 13 A schematic flowchart of a method for identifying a living organism provided in another embodiment of this disclosure;

[0064] Figure 14 A schematic flowchart of a method for identifying a living organism provided in yet another embodiment of this disclosure;

[0065] Figure 15A A resized first region of interest image provided as an embodiment of this disclosure;

[0066] Figure 15B A resized second region of interest image provided as an embodiment of this disclosure;

[0067] Figure 16AA first region of interest image after determining the image center, provided as another embodiment of this disclosure;

[0068] Figure 16B A second region of interest image after determining the image center, provided as another embodiment of this disclosure;

[0069] Figure 17A A resized first region of interest image provided for another embodiment of this disclosure;

[0070] Figure 17B A resized second region of interest image provided for another embodiment of this disclosure;

[0071] Figure 18 A flowchart illustrating a training method for a liveness detection classifier provided for embodiments of this disclosure;

[0072] Figure 19 A schematic diagram of an image-based biological liveness detection method system provided for embodiments of this disclosure;

[0073] Figure 20 A schematic diagram of a training system for a liveness detection classifier provided for embodiments of this disclosure; and

[0074] Figure 21 A block diagram of an electronic device suitable for training a liveness detection method or liveness detection classifier based on an image, as provided in embodiments of this disclosure. Detailed Implementation

[0075] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0076] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0077] Figure 1 An exemplary system architecture 100 is shown, in which the image-based biological liveness detection method, liveness detection classifier training method, apparatus, electronic device and computer-readable storage medium of the present disclosure can be applied.

[0078] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0079] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications for enabling information communication between the terminal devices 101, 102, and 103 and server 105 can be installed, such as image-based biometric recognition applications, biometric classifier training applications, and instant messaging applications.

[0080] Terminal devices 101, 102, 103 and server 105 can be hardware or software. For example, Figure 1 The illustration shows scenarios where terminal devices 101, 102, and 103 are door locks, ATMs, and mobile phones. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to biometric devices used in security, information security, payment, and authentication fields, such as palm-scanning devices and facial recognition devices. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. When server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here.

[0081] Server 105 can provide various services through its built-in applications. For example, when running an image-based liveness detection application, server 105 can achieve the following effects: generate multiple regions of interest (ROI) images based on the image; extract features from the multiple ROI images to obtain feature maps; and input the feature maps into a trained liveness detection classifier to obtain the liveness detection result of the organism under test. Among these features, the multiple ROI images all include the same part of the organism under test, and the same part has different sizes in the multiple ROI images.

[0082] Since determining the state information of physical nodes requires significant computing resources and strong computational capabilities, the image-based biometric liveness detection method or liveness detection classifier training method provided in the subsequent embodiments of this disclosure is generally executed by a server 105 with strong computing power and abundant computing resources. Correspondingly, the image-based biometric liveness detection system or liveness detection classifier training system is also generally located in the server 105. However, it should also be noted that when the terminal devices 101, 102, and 103 also have sufficient computing power and resources, the terminal devices 101, 102, and 103 can also complete the aforementioned calculations performed by the server 105 through the image-based biometric liveness detection application or liveness detection classifier training application installed on them, and thus output the same results as the server 105. Especially when multiple terminal devices with different computing capabilities exist simultaneously, if the image-based biometrics liveness detection application or the liveness detection classifier training application determines that the terminal device has strong computing power and sufficient remaining computing resources, the terminal device can perform the aforementioned calculations, thereby appropriately reducing the computing pressure on server 105. Correspondingly, the image-based biometrics liveness detection device or the liveness detection classifier training device can also be located in terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may also exclude server 105 and network 104.

[0083] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0084] Please refer to Figure 2 , Figure 2 A flowchart of an image-based liveness detection method for embodiments of this disclosure is provided, wherein process 200 includes the following steps:

[0085] Step 201: Generate multiple regions of interest (ROI) images based on the image, wherein the multiple ROI images all include the same part of the organism under test, and the same part has different sizes in the multiple ROI images.

[0086] Step 202: Extract features from multiple regions of interest images to obtain feature maps.

[0087] Step 203: Input the feature map into the trained liveness detection classifier to obtain the liveness detection result of the organism to be tested.

[0088] The following will combine Figures 3-17B The steps of the above-described image-based biological liveness detection method 200 are described in detail.

[0089] Step 201

[0090] Figure 3 A flowchart for generating multiple regions of interest images based on an image, provided for embodiments of this disclosure.

[0091] like Figure 3 As shown, in some embodiments of this disclosure, step 201 may include, for example:

[0092] Step 2011: Detect the image and obtain the key points and bounding boxes of the parts, where the parts include at least one of the face, the back of the hand, and the palm of the hand.

[0093] Step 2012: Generate multiple region of interest images based on the bounding box and multiple key points.

[0094] Figure 4 Image 10 provided for one embodiment of this disclosure. Figure 5 This is a schematic diagram of a key point 11 and a location frame 12 provided for one embodiment of the present disclosure. Figure 6A The first region of interest (ROI) image ROI_1 is provided for one embodiment of this disclosure. Figure 6B The second region of interest image ROI_2 is provided for one embodiment of this disclosure.

[0095] like Figures 3-6B As shown, in one embodiment of this disclosure, image 10 can be obtained by acquiring data from the organism under test using a camera such as an IR camera (Infrared Camera), an RGB camera (Red-Green-Blue Camera), or a depth camera. Image 10 may include a portion of the organism under test (which can be understood as a part of the organism under test), such as at least one of the face, the back of the hand, and the palm of the hand. Furthermore, image 10 may also include only a portion of a portion of the organism under test, such as palm veins, palm prints, etc. Taking face liveness detection as an example, by detecting an image including a face, face bounding boxes and key points can be obtained, and multiple region-of-interest images can be generated based on the face bounding boxes and key points.

[0096] refer to Figure 4Image 10 includes not only a portion of the organism being tested or a local portion of the organism, but also other parts of the organism or the background outside the organism. Directly extracting features from Image 10, or generating only a region of interest from Image 10 and extracting features from that region of interest, may reduce the accuracy of liveness detection. Taking face liveness detection as an example, Image 10 includes the face, arms, hands, part of the torso, and background parts outside the body (e.g., parts of a table or chair, ceiling, and walls).

[0097] When the proportion of the face in image 10 is relatively large, if feature extraction is performed directly based on image 10, or if only a region of interest is generated for image 10 and feature extraction is performed based on that region of interest, the liveness detection result focuses more on facial features and loses background information. This makes it easier to be fooled by high-quality printed photos, thus reducing the accuracy of liveness detection. Furthermore, when the proportion of the face is relatively small, if feature extraction is performed directly based on image 10, or if only a region of interest is generated for image 10 and feature extraction is performed based on that region of interest, the liveness detection result cannot obtain detailed facial features. This makes it easier to be fooled by 3D head models or even paper face masks, which also reduces the accuracy of liveness detection.

[0098] To at least address the aforementioned problems and achieve a simple and cost-effective operation, in at least one embodiment of this disclosure, multiple region-of-interest (ROI) images are generated based on the same image. Each ROI image includes the same part of the organism under test, and the size of the same part differs across the multiple ROI images. This approach takes into account both the local details of the organism under test and the surrounding background information, improving the diversity and utilization of local image data of the organism and enhancing the accuracy of liveness detection.

[0099] Specifically, refer to Figure 4 and Figure 5 Taking face liveness detection as an example, image 10 includes the face, other body parts besides the face, and the background outside the human body. Optionally, by detecting image 10, multiple key points 11 of the face and the face localization box 12 can be obtained, wherein the multiple key points 11 may include a first key point 1, a second key point 2, a third key point 3, ... key point 68, etc.

[0100] Alternatively, the bounding box 12 can be determined based on multiple key points 11. For example, a subset of the key points 11 can be selected to form a minimum rectangular bounding box, and this minimum rectangular bounding box can be defined as the bounding box 12. Then, based on the bounding box 12 and the multiple key points 11, one of multiple region-of-interest images can be generated, for example... Figure 6AThe first region of interest (ROI) image is shown as ROI_1.

[0101] Alternatively, a subset or all of the keypoints 11 can be selected to form a minimum rectangular bounding box, which is then defined as the location box 12. Afterward, based on the location box 12 and the multiple keypoints 11, another of the multiple regions of interest images can be generated, for example... Figure 6B The second region of interest image ROI_2 is shown.

[0102] Optionally, the bounding box used to determine the region of interest (ROI) image does not have to be unique. Different ROI images from the same image may have the same or different bounding boxes.

[0103] refer to Figure 5 The positioning frame 12 may have a first dimension W in the x-direction and a second dimension H in the y-direction, wherein the x-direction and y-direction are perpendicular to each other. The two vertices on the same diagonal of the rectangle containing the positioning frame 12 are vertex A and vertex B, where the coordinates of vertex A can be set to (X_min, Y_min), and the coordinates of vertex B can be set to (X_max, Y_max). When the positioning frame 12 includes multiple keypoints 11, the X_min coordinate of vertex A can be the minimum x-coordinate among the multiple keypoints 11, and the Y_min coordinate of vertex A can be the minimum y-coordinate among the multiple keypoints 11; similarly, the X_max coordinate of vertex B can be the maximum x-coordinate among the multiple keypoints 11, and the Y_max coordinate of vertex B can be the maximum y-coordinate among the multiple keypoints 11.

[0104] The first dimension W of the bounding box 12 can be the difference between X_max in the coordinates of vertex B and X_min in the coordinates of vertex A, and the second dimension H of the bounding box 12 can be the difference between Y_max in the coordinates of vertex B and Y_min in the coordinates of vertex A. In this way, the size of the bounding box 12 and its position in the image 10 can be determined based on multiple keypoints 11 (e.g., ...). Figure 4 (As shown).

[0105] refer to Figure 6A and Figure 6B The first region of interest (ROI) image 1 and the second region of interest (ROI) image 2 are both derived from image 10 and both include the face to be detected. The face is of different size in the first region of interest (ROI) image 1 and the second region of interest (ROI) image 2. This can take into account the local detailed features of the organism under test as well as the background information around the organism under test, improve the diversity and utilization of local image data of the organism, and improve the accuracy of liveness detection of the organism.

[0106] Figure 6CThe first region of interest (ROI) image ROI_1 is provided for another embodiment of this disclosure. Figure 6D Image 10 provided for another embodiment of this disclosure. Figure 6E Other regions of interest (ROIs) images ROI_1-2 provided for another embodiment of this disclosure.

[0107] As an alternative, refer to Figures 4-5 , Figure 6C-6E Furthermore, after generating a first region of interest (ROI) image ROI_1 based on the location bounding box 12 and multiple key points 11, the image center of the first ROI image ROI_1 can be determined. Based on the image center of the first ROI image ROI_1, other location bounding boxes 13 are generated by expanding the edges of the location bounding box 12. Based on the other location bounding boxes 13 and the position 14 of the image center of the first ROI image ROI_1 in the image 10, at least one other region of interest image associated with the first ROI image ROI_1 can be generated, such as other region of interest images ROI_1-2, etc. Optionally, in the first ROI image ROI_1 and at least one other region of interest image ROI_1-2 generated based on this embodiment, the size of the face in the first ROI image ROI_1 accounts for the largest proportion of its own region of interest image.

[0108] Specifically, a first region of interest (ROI) image ROI_1 can be generated based on the bounding box 12 and multiple key points 11, and the image center of the first ROI image ROI_1 can be determined, for example... Figure 6C The image center of the first region of interest (ROI_1) is shown (the white dot shown in the figure). It should be noted that, for ease of observation, Figure 6C The size of the image center is enlarged; in fact, the size of the image center is the size of the pixel located at the image center. In other words, the image center of the first region of interest (ROI)_1 is a single pixel. Optionally, the pixel corresponding to the image center of the first ROI_1 can be one of the pixels in the first ROI_1 that belongs to the face. For example, the pixel corresponding to the image center of the first ROI_1 could be the pixel corresponding to the center point of the face. This can improve the accuracy of liveness detection.

[0109] like Figure 6C-6EAs shown, after determining the image center of the first region of interest (ROI) image ROI_1, other positioning boxes 13 are generated by expanding the edges of the positioning box 12 based on the image center of the first region of interest (ROI) image ROI_1. Based on the other positioning boxes 13 and the position 14 of the image center of the first region of interest (ROI) image ROI_1 in the image 10, at least one other region of interest (ROI) image ROI_1-2 associated with the first region of interest (ROI) image ROI_1 is generated.

[0110] Therefore, the pixels corresponding to the image centers of other regions of interest (ROI_1-2) (such as...) Figure 6E The white dot shown is the same pixel as the center of the first region of interest (ROI) image ROI_1. Alternatively, this embodiment generates multiple ROI images with the same image center, which is one of multiple pixels belonging to a part of the organism under test within the ROI image. For example, the image centers of multiple ROI images are the same, corresponding to the center point of the part of the organism under test. This ensures that the part of the organism under test (e.g., face, back of hand, or palm) is always located in the core region of the multiple ROI images. Furthermore, the background outside the part of the organism under test can vary in different ROI images.

[0111] Therefore, while taking into account the local detailed features of the organism under test as well as the background information around the organism, it improves the diversity and utilization of local image data of the organism and improves the accuracy of liveness detection of the organism.

[0112] As an option, a scaling factor, scalel, can be defined, and its specific value can be adjusted according to the actual situation. Then, based on the image center of the first region of interest (ROI) image ROI_1, the scaling factor scalel, and the size of the bounding box 12 (or the size of the first ROI image ROI_1), the sizes of the other bounding boxes 13 and their positions in the image 10 are determined. In other words, based on the other bounding boxes 13 and the position 14 of the image center of the first ROI image ROI_1 in the image 10, other regions of interest (ROIs) ROI_1-2 are generated. For example, if the first ROI image ROI_1 is rectangular, with a size W1 in the x-direction and a size H1 in the y-direction (W1 > H1), then the size of the bounding box 12 in the xy-plane is the same as that of the first ROI image ROI_1, with a size of W1 in the x-direction and H1 in the y-direction.

[0113] The other region of interest (ROI) image ROI_1-2 obtained under this implementation can also be rectangular, with a dimension of W1-2 = W1 × scalel in the x-direction and a dimension of H1-2 = H1 × scalel in the y-direction. It should be noted that... Figure 6E Only one other region of interest image ROI_1-2 is shown. However, those skilled in the art will understand that multiple other region of interest images ROI_1-2 can be generated according to the method provided in this application and actual needs. This application does not limit the number of other region of interest images ROI_1-2, and the expansion coefficients for different other region of interest images ROI_1-2 can be different.

[0114] Optionally, the expansion coefficients for the x-direction and y-direction expansion of the bounding box 12 can be different. For example, by selecting different values ​​for the expansion coefficients scalel1 and scalel2 in the x-direction and y-direction expansion, other bounding boxes 13 with different shapes and magnification can be obtained. For example, other bounding boxes 13 can be magnified rectangular bounding boxes or magnified square bounding boxes. Therefore, other regions of interest (ROIs) images ROI_1-2 can be rectangular or square, with a size of W1-2 = W1 × scalel1 in the x-direction and a size of H1-2 = H1 × scalel2 in the y-direction.

[0115] Figure 7 Image 20 provided for another embodiment of this disclosure. Figure 8 A schematic diagram of a key point 21 and a location frame 22 provided for another embodiment of this disclosure. Figure 9A The third region of interest (ROI) image ROI_3 is provided for another embodiment of this disclosure. Figure 9B The fourth region of interest (ROI) image ROI_4 is provided for one embodiment of this disclosure.

[0116] like Figures 7-9B As shown, in another embodiment of this disclosure, image 20 can be obtained by acquiring data from the organism under test using a camera such as an IR camera, an RGB camera, or a depth camera. Image 20 may include the palm of the hand and the background outside the palm.

[0117] Taking palmprint liveness detection as an example, by detecting images including the palm, palmprint localization boxes and key points can be obtained, and multiple region of interest images can be generated based on the palmprint localization boxes and key points.

[0118] Optionally, such as Figures 7-8As shown, by detecting the image 20, multiple key points 21 of the palm print and the positioning box 22 of the palm print can be obtained. The multiple key points 21 may include the first key point 1, the second key point 2, the third key point 3, ... key point 28, etc.

[0119] Alternatively, the bounding box 22 can be determined based on multiple keypoints 21. For example, it can be the smallest rectangle enclosed by a subset or all of the keypoints 21, and this smallest rectangle can be defined as the bounding box 22. Then, multiple region-of-interest images can be generated based on the bounding box 22 and the multiple keypoints 21, for example... Figure 9A The third region of interest image ROI_3 shown and Figure 9B The fourth region of interest (ROI) image, ROI_4, is shown. Optionally, the bounding boxes used to determine the ROI image may not be unique. Different ROI images from the same image may have the same or different bounding boxes.

[0120] The size of the bit frame 22 and its position in the image 20 can be determined by multiple key points 21 (e.g., Figure 7 (As shown). Reference Figure 9A and Figure 9B The third region of interest (ROI) image 3 and the fourth region of interest (ROI) image 4 are both derived from image 20 and both include the palm print to be detected. The palm print has a different size in the third region of interest (ROI) image 3 and the fourth region of interest (ROI) image 4. This can take into account the local detailed features of the organism under test as well as the background information around the organism under test, improve the diversity and utilization of the local image data of the organism, and improve the accuracy of the liveness detection of the organism.

[0121] Furthermore, after generating the third region of interest (ROI) image ROI_3 based on the bounding box 22 and multiple key points 21, the image center of the third region of interest (ROI) image ROI_3 can be determined; based on the image center of the third region of interest (ROI) image ROI_3, other bounding boxes can be generated by expanding the edges of the bounding box 22; based on the positions of the other bounding boxes and the image center of the third region of interest (ROI) image ROI_3 in image 20, other region of interest images associated with the third region of interest (ROI) image ROI_3 can be generated. The specific process of generating other region of interest images is similar to... Figures 4-5 , Figures 6C-6E The corresponding textual descriptions are identical, and will not be repeated here.

[0122] Step 202

[0123] Figure 10 This is a flowchart illustrating feature extraction from multiple regions of interest images to obtain a feature map, as provided in one embodiment of this disclosure. Figure 11 This is a schematic flowchart of a method for identifying the liveness of an organism provided in one embodiment of the present disclosure.

[0124] like Figure 10 and Figure 11 As shown, in one embodiment of this disclosure, step 202 may include, for example:

[0125] Step 2021: Fuse multiple region of interest images.

[0126] Step 2022: Extract features from the fused region of interest image to obtain a feature map.

[0127] Specifically, in a scheme that obtains the liveness recognition result of the organism under test through a single-model trained liveness recognition classifier, multiple region-of-interest (ROI) images can be fused first, and then feature maps can be obtained by extracting features from the fused RIO images. By effectively merging complementary, redundant, or unique information from each source image (e.g., multiple RIO images), a comprehensive image that contains all the key details and has enhanced properties can be generated.

[0128] For example, the first region of interest (ROI) image ROI_1 and the second region of interest (ROI) image ROI_2 from image 10 are fused, both of which include the face to be detected, but the face has different sizes in the first ROI image ROI_1 and the second ROI image ROI_2. Then, feature extraction can be performed on the fused ROI image, such as the fused ROI, to obtain a feature map.

[0129] Alternatively, a weight value can be assigned to each region of interest image, and the fused region of interest image can be determined based on the weight values ​​of different region of interest images and the selected fusion algorithm. The setting of the weight value of the region of interest image and the selection of the fusion algorithm can be determined based on the specific biological liveness detection task and at least one of the following: color saturation, detail preservation, edge transition, and sharpness of the acquired image.

[0130] Optionally, the fusion algorithm may include at least one of spatial domain fusion algorithms and frequency domain fusion algorithms, wherein the spatial fusion algorithm may include pyramid fusion algorithm, Poisson fusion algorithm, IHS (Intensity-Hue-Saturation Fusion) fusion algorithm, etc., and the frequency domain fusion algorithm may include DCT (Discrete Cosine Transformation) fusion algorithm and wavelet fusion algorithm, etc.

[0131] After obtaining the fused region of interest image, features can be extracted from the fused region of interest image to obtain a feature map.

[0132] For example, a feature extraction network is invoked to obtain a feature map of the fused region of interest (ROI) image. The feature extraction network may include multiple residual convolutional modules and dilated convolutional modules. The residual convolutional modules can extract live hidden layer features from the fused ROI image to obtain a hidden layer representation representing the liveness information of the organism. Furthermore, the aforementioned live hidden layer features can be obtained by reducing or increasing the feature dimension. The dilated convolutional modules can perform processing such as concatenation on the live hidden layer features obtained from the residual convolutional modules to obtain a feature map representing the liveness information of the organism. It should be noted that this disclosure does not limit the specific method of fusing multiple ROI images or the specific method of feature extraction from the fused ROI image.

[0133] Figure 12 This is a flowchart illustrating feature extraction of multiple regions of interest images to obtain a feature map, as provided in another embodiment of this disclosure. Figure 13 This is a schematic flowchart of a method for identifying the liveness of an organism provided in another embodiment of the present disclosure.

[0134] like Figure 12 and Figure 13 As shown, in another embodiment of this disclosure, step 202 may further include, for example:

[0135] Step 2023: Extract features from multiple regions of interest images to obtain multiple sub-feature maps.

[0136] Step 2024: Merge multiple sub-feature maps to obtain a feature map.

[0137] Specifically, in the scheme of obtaining the liveness recognition result of the organism to be tested through a single-model trained liveness recognition classifier, features can be extracted from multiple region-of-interest images separately to obtain multiple sub-feature maps. Then, the multiple sub-feature maps are fused to obtain a feature map. By extracting features from each source image (e.g., multiple region-of-interest images) separately and then fusing the extracted sub-feature maps to obtain the feature map, the dimensionality and representational ability of the feature map can be enriched, and the accuracy of feature map extraction can be improved.

[0138] For example, sub-feature extraction is performed on the first region of interest (ROI) image 1 and the second region of interest (ROI) image 2 from image 10, where both images include the face to be detected, but the face has different sizes in the first and second ROI images. Then, the multiple sub-feature maps obtained through sub-feature extraction can be fused to obtain a feature map.

[0139] Alternatively, a feature extraction network can be used to obtain sub-feature maps, and a feature fusion network can be used to obtain feature maps.

[0140] For example, each feature extraction network may include multiple residual convolutional modules and dilated convolutional modules. The residual convolutional modules can extract sub-liveness hidden layer features from one of multiple regions of interest (ROI) images to obtain a sub-hidden layer representation characterizing the liveness information of the organism. Furthermore, the aforementioned sub-liveness hidden layer features can be obtained by reducing or increasing the feature dimension during extraction. The dilated convolutional modules can perform processing such as concatenation on the sub-liveness hidden layer features obtained from the residual convolutional modules to obtain a sub-feature map characterizing the liveness information of the organism. The feature fusion network can be configured based on one of orthogonal fusion, convolutional fusion, or adaptive feature fusion. It should be noted that this disclosure does not limit the specific methods for obtaining multiple sub-feature maps or the specific methods for fusing multiple sub-feature maps to obtain a feature map.

[0141] Figure 14 This is a schematic flowchart of a method for identifying the liveness of an organism provided in yet another embodiment of this disclosure.

[0142] In addition, refer to Figure 12 and Figure 14 In another embodiment of this disclosure, step 202 may further include only step 2023, which involves extracting features from multiple regions of interest images to obtain multiple sub-feature maps. Then, the liveness detection result of the organism to be tested can be obtained by cascading the recognition results output by multiple trained liveness detection classifiers.

[0143] For example, sub-feature extraction can be performed on the first region of interest (ROI) image ROI_1, the second region of interest (ROI_2), and the nth region of interest (ROI_n) (n ≥ 3 positive integers) from the same image, etc. Each of these ROI images can include the same part of the organism to be detected, but the size of this same part differs in the multiple ROI images. Then, the multiple sub-feature maps can be input into multiple trained liveness detection classifiers, performing liveness detection classification 1, liveness detection classification 2, and liveness detection classification n steps, outputting liveness detection result 1, liveness detection result 2, and liveness detection result n, etc. The final liveness detection result can be obtained by cascading multiple liveness detection results 1, liveness detection result 2, and liveness detection result n, etc.

[0144] Optionally, depending on the specific task of detecting live organisms, any of the above-described embodiments can be selected to obtain feature maps. This enables the effective use of multiple regions of interest images and, on this basis, improves the accuracy of live organism recognition.

[0145] Furthermore, step 202 may also include performing grayscale compression processing on the region of interest image to obtain a grayscale square, and obtaining a feature map based on the grayscale square. In other words, by performing grayscale compression processing on multiple region of interest images, the computational cost of the biological liveness detection method can be reduced, and the efficiency of the biological liveness detection method can be improved. After obtaining multiple grayscale squares, a feature map can be obtained based on the method for obtaining a feature map described in any of the above embodiments.

[0146] In some embodiments of this disclosure, before performing step 202, the region of interest image may be preprocessed based on multiple key points, wherein the preprocessing may include at least one of geometric correction processing, optical improvement processing, and smoothing processing of the region of interest image.

[0147] For example, the region of interest (ROI) image can be corrected by performing geometric methods such as rotation, horizontal line calibration, and vertical line calibration. Figure 4 For example, since the face in the acquired image 10 is tilted, geometric methods such as rotation, horizontal line calibration, and vertical line calibration can be used to process the acquired image or region of interest image during the face liveness detection process to optimize the subsequent feature extraction process.

[0148] In addition, optical enhancements such as brightness, contrast, and sharpness can be applied to the region of interest image to improve the imaging effect of the organism under test and increase the accuracy of liveness identification.

[0149] In addition, smoothing processes such as noise suppression can be applied to the region of interest image to make the region of interest smoother, sharper, and enhance its boundaries while preserving as much detail as possible.

[0150] Optionally, in some embodiments of this disclosure, the acquired image may also be preprocessed, wherein the preprocessing may include at least one of geometric correction, optical improvement, and smoothing. This can also optimize the subsequent feature extraction process, enhance the imaging effect of the organism under test in the image, or, while preserving as much detail as possible in the acquired image, make the acquired image smoother, sharper, and enhance its boundaries.

[0151] Figure 15A The resized first region of interest (ROI) image ROI_1' is provided for one embodiment of this disclosure. Figure 15B The resized second region of interest image ROI_2' is provided for one embodiment of this disclosure. Figure 16AThe first region of interest (ROI) image ROI_1 is provided for another embodiment of this disclosure after determining the image center. Figure 16B The second region of interest (ROI) image ROI_2 is provided for another embodiment of this disclosure after determining the image center. Figure 17A "A resized first region of interest image ROI_1 provided for another embodiment of this disclosure". Figure 17B "A resized second region of interest image ROI_2 provided for another embodiment of this disclosure".

[0152] To facilitate feature extraction from multiple region-of-interest (ROI) images and reduce the impact of factors such as image resolution and variable distance between the camera and the face on face liveness detection, thereby improving the robustness of the detection method, multiple RIO images can be scaled up to make them the same size before executing step S202.

[0153] Specifically, refer to Figure 6A and Figure 15A , Figure 6B and Figure 15B In one embodiment of this disclosure, since the region of interest (ROI) image can be obtained through a bounding box, and the bounding box is rectangular, the ROI image is also rectangular. For example, the first ROI image ROI_1 is rectangular, with a dimension W1 in the x-direction and a dimension H1 in the y-direction, where W1 > H1. The second ROI image ROI_2 is square, with a dimension W2 in the x-direction and a dimension H2 in the y-direction, where W2 = H2.

[0154] Therefore, the first region of interest (ROI) image ROI_1 can be expanded. If W1 > H1, the size of ROI_1 in the y-direction is increased, expanding it into a square shape to form an adjusted ROI_1'. The adjusted ROI_1' has a size of W1' in the x-direction and H1' in the y-direction, where W1' = H1'. In other words, the ROI image generated from the image can be adjusted by determining the dimensions of the shorter and longer sides of its rectangular shape, increasing the size of the shorter side, and expanding the ROI image to a square shape.

[0155] Furthermore, taking the second region of interest (ROI) image ROI_2 as an example, in the step of adjusting the ROI image generated from the image, if the ROI image is determined to be square, no adjustment is required. Alternatively, after performing this step, the adjusted ROI_2' obtained will have a size W2' in the x-direction, which is equal to the size W2 of the ROI_2 image in the x-direction; and a size H2' in the y-direction, which is equal to the size H2 of the ROI_2 image in the y-direction.

[0156] After generating multiple regions of interest (ROI) images based on the image into square shapes, the adjusted ROI images can be further scaled to make the scaled ROI images the same size.

[0157] In addition, refer to Figure 6A , Figure 16A and Figure 17A , Figure 6B , Figure 16B and Figure 17B In another embodiment of this disclosure, the image center of the region of interest image generated based on the image can be determined first, for example... Figure 16A The image center of the first region of interest (ROI_1) is shown (as the white dot shown in the figure); Figure 16B The image center of the second region of interest (ROI_2) is shown (as the white dot in the figure). It should be noted that, for ease of observation, Figure 16A and Figure 16B The size of the image center has been increased; in fact, the size of the image center is the size of the pixel located at the center of the image.

[0158] After determining the image center of the region of interest (ROI), an expansion coefficient, scalel, can be defined. The specific value of scalel can be adjusted according to the actual situation; for example, scalel can be set to 2. Then, based on the image center (X_center, Y_center) of each ROI, the expansion coefficient, and the size of each ROI, the vertex coordinates of the adjusted ROI are determined.

[0159] For example, the coordinates of the two vertices on the same diagonal of the adjusted region of interest image can be represented as (X_center-Scalel×w, Y_center-Scalel×h) and (X_center+Scalel×w, Y_center+Scalel×h), respectively, where h and w are the dimensions of the region of interest image before adjustment in the x and y directions, respectively.

[0160] Through the above adjustments, the adjusted region of interest (ROI) image can be square in shape, and multiple adjusted ROI images have the same size. For example, the side lengths of the adjusted first ROI image ROI_1” are H1” and W1”, and the side lengths of the adjusted second ROI image ROI_2” are H2” = W2”, where H1” = W1” = H2” = W2”.

[0161] Furthermore, in some embodiments of this disclosure, the resolution of multiple region-of-interest (ROI) images can be adjusted before performing step S202 to ensure that the resolutions of the multiple RIO images are identical. This eliminates the effects caused by resolution inconsistencies, enhances the imaging effect of the organism under test in the image, and improves the accuracy of liveness detection of the organism.

[0162] Step 203

[0163] refer to Figure 11 , Figure 13 as well as Figure 14 After obtaining the feature map, it can be input into a trained liveness detection classifier to obtain the liveness detection result of the organism to be tested. In the embodiments of this disclosure, various feature fusion methods and model training schemes can be used to achieve effective utilization of data and improve the accuracy of the model, thereby improving the accuracy of liveness detection of organisms.

[0164] As an alternative, unlike most models which train a single model with a single input, one implementation of this disclosure can provide a multi-receptive-field input method to input a liveness detection classifier model with feature maps extracted from fused region of interest images.

[0165] As an alternative, one embodiment of this disclosure provides a multi-receptive-field input method, inputting a feature map formed by fusing sub-feature maps into a liveness detection classifier model, wherein the sub-features are obtained by extracting features from multiple regions of interest images respectively.

[0166] As an alternative, one embodiment of this disclosure may provide multiple liveness detection classifier models to analyze different regions of interest images, and obtain the liveness detection result of the organism to be tested based on the cascade of the recognition results output by multiple liveness detection classifier models.

[0167] For example, if the output of all liveness identification classifier models involved in the liveness identification process of the organism to be tested is "true", it means that each liveness identification classifier model has obtained the identification result that the organism to be tested is a live organism. Based on the cascading of these identification results, the identification result that the organism to be tested is a live organism can be obtained.

[0168] Furthermore, among all the liveness recognition classifier models participating in the liveness identification process of the organism under test, a certain number of liveness recognition classifier models output "true," indicating that a portion of the liveness recognition classifier models have obtained the recognition result that the organism under test is alive. Based on the cascading of these recognition results, the recognition result that the organism under test is alive can be obtained. Optionally, the number of a portion of the liveness recognition classifier models may exceed or be equal to half of the total number of all liveness recognition classifier models.

[0169] In addition, the average value of the output of all liveness identification classifier models involved in the liveness identification process of the organism under test is "true". Based on the cascading of these identification results, the identification result that the organism under test is a live organism can be obtained.

[0170] Optionally, all liveness recognition classifier models involved in the liveness recognition process of the organism under test can have different structures. In other words, the liveness recognition classifier models belong to different classification model structures to increase the accuracy of the recognition results.

[0171] Depending on the specific task of detecting live organisms, any of the above implementation methods can be selected to obtain feature maps, and a suitable classification model structure can be chosen. This enables the effective use of multiple regions of interest images and, on this basis, improves the accuracy of live organism recognition.

[0172] Please refer to Figure 18 , Figure 18 A flowchart of a training method for a liveness detection classifier provided for embodiments of this disclosure is provided, wherein process 300 includes the following steps:

[0173] Step 301: Based on the images in the training set, generate multiple training region of interest images, wherein the multiple training region of interest images all include the same part of the same organism, and the same part has different sizes in the multiple training region of interest images.

[0174] Step 302: Train a liveness detection classifier based on multiple training region-of-interest images.

[0175] Multiple regions of interest (ROI) images are generated from the same image in the training set. These ROI images all include the same part of the organism, but the size of the same part varies across the multiple ROI images. This approach takes into account both the local details of the organism and the surrounding background information. Using these multiple ROI images to train a liveness detection classifier improves the diversity and utilization of local image data of the organism, and ultimately increases the accuracy of liveness detection.

[0176] Optionally, the pixel corresponding to the image center of the multiple training regions of interest (ROI) images can be the same. Furthermore, the pixel corresponding to the image center can be one of the pixels in the training ROI images that belong to a living organism. Taking face liveness detection as an example, the pixel corresponding to the image center of the multiple training ROI images can be one of the pixels belonging to a face in the multiple training ROI images. For example, the pixel corresponding to the image center of the training ROI image is the pixel corresponding to the center point of the face. This can improve the accuracy of living organism recognition.

[0177] Alternatively, multiple training region-of-interest (ROI) images from the same image in the training set can be fused, and features can be extracted from the fused RIO images to obtain training feature maps. Based on these training feature maps, a liveness detection classifier can be trained.

[0178] By first fusing multiple regions of interest (ROI) images from the same image in the training set, and then extracting features from the fused ROI images to obtain feature maps, a single-model liveness detection classifier can be trained. This effectively combines complementary, redundant, or unique information from various source images (e.g., multiple ROI images) to generate a comprehensive image that contains all key details and has enhanced properties, which is beneficial for improving the judgment accuracy of the liveness detection classifier model.

[0179] Alternatively, features can be extracted from multiple training region-of-interest images from the same image in the training set to obtain multiple training sub-feature maps. These sub-feature maps can then be fused to obtain a single training feature map. Based on these training feature maps, a liveness detection classifier can be trained.

[0180] By first extracting features from multiple regions of interest images from the same image in the training set, multiple sub-feature maps are obtained. Then, by fusing these sub-feature maps to obtain a feature map, a single-model liveness detection classifier can be trained. This enriches the dimensionality and representational ability of the feature map, improves the accuracy of feature map extraction, and thus improves the judgment accuracy of the liveness detection classifier model.

[0181] As another option, features can be extracted from multiple training region-of-interest images from the same image in the training set to obtain multiple training sub-feature maps. Based on these training sub-feature maps, multiple sub-liveness recognition classifiers can be trained. These multiple sub-liveness recognition classifiers are cascaded into a single liveness recognition classifier. In other words, multiple liveness recognition classifiers can be trained. By inputting the feature map to each trained liveness recognition classifier, multiple liveness recognition classifier outputs can be obtained. Based on the cascading of these recognition results, the result indicating whether the organism to be tested is alive can be obtained.

[0182] Multiple liveness detection classifier models are trained using multiple regions of interest images from the same image in the training set. Based on the output of these trained liveness detection classifier models, the liveness detection result of the organism under test can be obtained, which can improve the accuracy of liveness detection.

[0183] Please refer to Figure 19 , Figure 19 This diagram illustrates an image-based liveness detection system for organisms according to an embodiment of the present disclosure. The image-based liveness detection system 400 includes: a region of interest (ROI) generation module 410, a feature extraction module 420, and a liveness detection module 430. The ROI generation module 410 is configured to generate multiple ROI images based on an image. The feature extraction module 420 is configured to extract features from the multiple ROI images to obtain feature maps. The liveness detection module 430 is configured to input the feature maps into a trained liveness detection classifier to obtain the liveness detection result of the organism to be tested. The multiple ROI images all include the same body part of the organism to be tested; and the same body part has different sizes in the multiple ROI images.

[0184] In at least one embodiment of this disclosure, multiple region-of-interest (ROI) images are generated based on the same image, wherein the multiple RIO images all include the same part of the organism under test, and the size of the same part differs in the multiple RIO images. This can take into account both the local detailed features of the organism under test and the background information around the organism, improving the diversity and utilization of the local image data of the organism, and improving the accuracy of liveness detection of the organism.

[0185] In one embodiment of this disclosure, to facilitate feature extraction from multiple region-of-interest (ROI) images and reduce the impact of factors such as image resolution and variable distance between the camera and the face on face liveness detection, thereby improving the robustness of the detection system, a scaling unit can be set in the RIO generation module to scale the multiple RIO images so that they have the same size. Specifically, the RIO generation module 410 may include a scaling unit (not shown), which is configured to scale the multiple RIO images so that they have the same size.

[0186] In one embodiment of this disclosure, the region of interest generation module 410 may further include a resolution adjustment unit (not shown), which is configured to adjust the resolution of multiple region of interest images to make the multiple region of interest images have the same resolution. This can eliminate the influence caused by resolution inconsistency, enhance the imaging effect of the organism under test in the image, and improve the accuracy of liveness detection of the organism.

[0187] In one embodiment of this disclosure, the region of interest (ROI) generation module 410 further includes a localization unit (not shown), configured to detect the image and obtain key points of the body part and a localization bounding box of the body part; and a generation unit, configured to generate multiple ROI images based on the localization bounding boxes and multiple key points, wherein the body part includes at least one of the face, the back of the hand, and the palm of the hand. Optionally, after generating a first ROI image based on the localization bounding boxes and multiple key points, the image center of the first ROI image is determined; based on the image center of the first ROI image, other localization bounding boxes are generated by expanding the edges of the localization bounding boxes; based on the positions of the other localization bounding boxes and the image center of the first ROI image in the image, at least one other ROI image associated with the first ROI image is generated. Optionally, in the first ROI image and at least one other ROI image generated based on this embodiment, the size of the body part of the organism to be detected in the first ROI image accounts for the largest proportion of its own ROI image.

[0188] In this implementation, the pixel corresponding to the center of multiple regions of interest (ROI) images is the same. Furthermore, the pixel corresponding to the image center is one of the pixels within the ROI image that belongs to a part of the organism being tested. For example, the pixel corresponding to the image center is the pixel corresponding to the center point of a part of the organism being tested. This improves the accuracy of liveness detection.

[0189] In addition, the liveness detection system 400 also includes a preprocessing module (not shown), which can be configured to preprocess the region of interest image based on multiple key points, wherein the preprocessing includes at least one of geometric correction processing, optical improvement processing and smoothing processing of the region of interest image.

[0190] For example, the region of interest (ROI) image can be corrected by performing geometric methods such as rotation, horizontal line calibration, and vertical line calibration. Figure 4 For example, since the face in the acquired image 10 is tilted, the preprocessing module can be used in the face liveness detection process to process the acquired image or region of interest image by geometric methods such as rotation, horizontal line calibration, and vertical line calibration, so as to optimize the subsequent feature extraction process.

[0191] In addition, the preprocessing module can be used to perform optical enhancements on the region of interest image, such as brightness, contrast, and sharpness, thereby enhancing the imaging effect of the organism under test in the image and improving the accuracy of liveness identification of the organism.

[0192] In addition, a preprocessing module can be used to perform smoothing processes on the region of interest image, such as noise suppression, so as to make the region of interest image smoother, sharper, and with enhanced boundaries while preserving as much detail as possible.

[0193] In one embodiment of this disclosure, the feature extraction module 420 may include an initial fusion unit (not shown) and a subsequent extraction unit (not shown). The initial fusion unit is configured to fuse multiple regions of interest images; the subsequent extraction unit is configured to extract features from the fused regions of interest images to obtain a feature map.

[0194] In a scheme that obtains the liveness recognition result of a test organism using a single-model trained liveness recognition classifier, an initial fusion unit can be used to fuse multiple region-of-interest (ROI) images. Then, a subsequent extraction unit can be used to extract features from the fused RIO images to obtain a feature map. By effectively merging complementary, redundant, or unique information from the source images (e.g., multiple RIO images), a comprehensive image containing all key details and enhanced properties can be generated.

[0195] In one embodiment of this disclosure, the feature extraction module 420 may include an initial extraction unit (not shown) and a subsequent fusion unit (not shown). The initial extraction unit is configured to extract features from multiple regions of interest images to obtain multiple sub-feature maps, and the subsequent fusion unit is configured to fuse the multiple sub-feature maps to obtain a feature map.

[0196] In a scheme that obtains the liveness recognition result of a target organism using a single-model trained liveness recognition classifier, an initial extraction unit can be used to extract features from multiple regions of interest (ROIs) images separately, obtaining multiple sub-feature maps. Then, a subsequent fusion unit fuses these sub-feature maps to obtain a final feature map. By extracting features from each source image (e.g., multiple ROI images) separately and then fusing the extracted sub-feature maps, the feature map's dimensionality and representational power can be enriched, improving the accuracy of feature map extraction.

[0197] In one embodiment of this disclosure, the feature extraction module 420 may include a sub-extraction unit (not shown), which is configured to extract features from multiple regions of interest images to obtain multiple sub-feature maps.

[0198] Furthermore, the liveness detection module 430 may include a sub-detection module (not shown), configured to input multiple sub-feature maps into multiple trained liveness detection classifiers respectively, and output the results. The liveness detection result of the organism to be tested is determined based on the output results of the multiple trained liveness detection classifiers. Using the output results of multiple sub-detection modules to obtain the liveness detection result of the organism to be tested can improve the accuracy of liveness detection. Optionally, the multiple trained liveness detection classifiers have different structures.

[0199] In one embodiment of this disclosure, the feature extraction module 420 may further include a grayscale processing unit (not shown), which is configured to perform grayscale compression processing on the region of interest image to obtain a grayscale square, wherein a feature map is obtained based on the grayscale square. By performing grayscale compression processing on multiple regions of interest images, the computational cost of the biological liveness recognition system is reduced, and the efficiency of the biological liveness recognition system is improved.

[0200] Please refer to Figure 20 , Figure 20 This diagram illustrates a training system for a liveness detection classifier according to an embodiment of the present disclosure. The training system 500 includes a data generation module 510 and a training module 520. The data generation module 510 is configured to generate multiple training region-of-interest (ROI) images based on images in a training set. Each of the multiple RIO images includes the same body part of the same organism, and the size of the same body part differs across the multiple RIO images. The training module 520 is configured to train the liveness detection classifier based on the multiple RIO images.

[0201] In one embodiment of this disclosure, the training module 520 may include: a first sub-training module (not shown), a second sub-training module (not shown), and a third sub-training module (not shown). The first sub-training module is configured to fuse multiple training region-of-interest (ROI) images from the same image in the training set. The second sub-training module is configured to extract features from the fused training RIO images to obtain training feature maps. The third sub-training module is configured to train a liveness detection classifier based on the training feature maps.

[0202] By first fusing multiple regions of interest (ROI) images from the same image in the training set, and then extracting features from the fused ROI images to obtain feature maps, a single-model liveness detection classifier can be trained. This effectively combines complementary, redundant, or unique information from various source images (e.g., multiple ROI images) to generate a comprehensive image that contains all key details and has enhanced properties, which is beneficial for improving the judgment accuracy of the liveness detection classifier model.

[0203] In one embodiment of this disclosure, the training module 520 further includes a fourth sub-training module (not shown), a fifth sub-training module (not shown), and a sixth sub-training module (not shown). The fourth sub-training module is configured to extract features from multiple training region-of-interest images from the same image in the training set, respectively, to obtain multiple training sub-feature maps. The fifth sub-training module is configured to fuse the multiple training sub-feature maps to obtain a training feature map. The sixth sub-training module is configured to train a liveness detection classifier based on the training feature map.

[0204] By first extracting features from multiple regions of interest images from the same image in the training set, multiple sub-feature maps are obtained. Then, by fusing these multiple sub-feature maps to obtain a feature map, a single-model liveness detection classifier can be trained. This enriches the dimensionality and representational ability of the feature map, improves the accuracy of feature map extraction, and thus improves the judgment accuracy of the liveness detection classifier model.

[0205] In one embodiment of this disclosure, the training module 520 may further include a seventh sub-training module (not shown) and an eighth sub-training module (not shown). The seventh sub-training module is configured to extract features from multiple training region-of-interest images from the same image in the training set, respectively, to obtain multiple training sub-feature maps. The eighth sub-training module is configured to train multiple sub-liveness recognition classifiers based on the multiple training sub-feature maps, wherein the multiple sub-liveness recognition classifiers are cascaded into a liveness recognition classifier.

[0206] Multiple liveness detection classifier models are trained using multiple regions of interest images from the same image in the training set. Based on the output of these trained liveness detection classifier models, the liveness detection result of the organism under test can be obtained, which can improve the accuracy of liveness detection.

[0207] Furthermore, the pixel corresponding to the image center of the multiple training regions of interest (ROI) images can be the same. For example, the pixel corresponding to the image center can be one of the pixels in the training ROI images that belongs to a living organism. Taking face liveness detection as an example, the pixel corresponding to the image center of the multiple training ROI images can be one of the pixels in the multiple training ROI images that belongs to a face. For example, the pixel corresponding to the image center of the training ROI image is the pixel corresponding to the center point of the face. This can improve the accuracy of living organism recognition.

[0208] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to implement the image-based biological liveness detection method or the liveness detection classifier training method described in any of the above embodiments.

[0209] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the image-based biological liveness detection method or the liveness detection classifier training method described in any of the above embodiments.

[0210] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the image-based biological liveness detection method or the liveness detection classifier training method described in any of the above embodiments.

[0211] Figure 21 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0212] like Figure 21 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0213] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0214] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as image-based biometrics methods for liveness detection or training methods for liveness detection classifiers. For example, in some embodiments, the image-based biometrics method for liveness detection or training methods for liveness detection classifiers can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the image-based biometrics method for liveness detection or training methods for liveness detection classifiers described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured by any other suitable means (e.g., by means of firmware) to perform an image-based method for liveness detection of organisms or a method for training a liveness detection classifier.

[0215] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0216] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0217] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0218] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0219] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0220] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0221] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0222] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for liveness detection of organisms based on images, characterized in that, include: Multiple region-of-interest (ROI) images are generated based on the image; Feature maps are obtained by extracting features from multiple regions of interest images; as well as The feature map is input into a trained liveness detection classifier to obtain the liveness detection result of the organism to be tested. Among them, multiple regions of interest images all include the same part of the organism under test; and The same region has different sizes in multiple images of the region of interest.

2. The liveness detection method according to claim 1, wherein, Before performing feature extraction on the multiple regions of interest images, the method further includes: By scaling multiple region-of-interest (ROI) images, the ROI images are made to have the same size.

3. The liveness detection method according to claim 1, wherein, Before performing feature extraction on the multiple regions of interest images, the method further includes: By adjusting the resolution of multiple region-of-interest (ROI) images, the resolution of the multiple ROC images becomes the same.

4. The liveness detection method according to claim 1, wherein, Generating multiple region-of-interest images based on the image includes: The image is inspected to obtain key points of the region and the bounding box of the region; and Based on the location bounding box and multiple key points, multiple regions of interest images are generated. The location includes at least one of the face, the back of the hand, and the palm of the hand.

5. A training method for a liveness detection classifier, characterized in that, include: Based on the images in the training set, multiple training region of interest images are generated, wherein the multiple training region of interest images all include the same part of the same organism, and the same part has different sizes in the multiple training region of interest images; as well as The liveness detection classifier is trained based on multiple training region-of-interest images.

6. A liveness detection system for biological organisms based on images, characterized in that, include: The region of interest generation module is configured to generate multiple region of interest images based on the image; The feature extraction module is configured to extract features from multiple regions of interest images to obtain feature maps; as well as The liveness detection module is configured to input the feature map into a trained liveness recognition classifier to obtain the liveness recognition result of the organism to be tested. Among them, multiple regions of interest images all include the same part of the organism under test; and The same region has different sizes in multiple images of the region of interest.

7. A training system for a liveness detection classifier, characterized in that, include: The data generation module is configured to generate multiple training region of interest images based on images in the training set, wherein the multiple training region of interest images all include the same part of the same organism, and the same part has different sizes in the multiple training region of interest images; as well as The training module is configured to train the liveness detection classifier based on multiple training region-of-interest images.

8. An electronic device, characterized in that, include: At least one processor; A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, causes the at least one processor to perform the liveness detection method according to any one of claims 1-4 or the training method according to claim 5.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the liveness detection method according to any one of claims 1-4 or the training method according to claim 5.

10. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, implements the liveness detection method according to any one of claims 1-4 or the training method according to claim 5.

Images