A multimodal biometric comparison method and system
Through the multimodal biometric comparison method, combined with red light and infrared spectral analysis, the security problems of existing identity verification technologies in forgery and fraud identification are solved, achieving higher authentication accuracy and security.
Patent Information
- Application Number
- CN202410890263.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-04
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-04
AI Technical Summary
Existing identity verification technologies are less secure when facing forgery and fraud, making it difficult to accurately identify identities, especially in entry and exit and border inspection scenarios.
By using multimodal biometric comparison method, by obtaining the fingerprint characteristics and face images of the four-linked fingers, and combining red light and infrared spectrum analysis, we can determine whether the four-linked fingers and face are real biometric features, using different human skins and The absorption and reflection characteristics of light by bone tissue are verified at multiple levels.
Improve the security of identity verification and the accuracy of fake fake detection, enhance the ability to identify fake biometrics, and ensure the reliability of identity verification.
Smart Images

Figure CN118627049B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of identity verification, and in particular to a multimodal biometric comparison method and system. Background Art
[0002] In the scenarios of entry and exit, customs, and border inspection, it is usually necessary to compare and identify the identity and biometric features of the person in order to check whether the current person is the real person whose identity is stored in the system. At present, the fingerprint features of four consecutive fingers and the face image are usually collected and compared with the fingerprint features of four consecutive fingers and the face image of the person stored in the system to determine whether the current person's identity is real and whether there is any forged biometric features. However, the technology of forging biometric features is getting more and more advanced. After the fingerprint features of four consecutive fingers and the face features are forged and falsified with high quality, it is not easy to accurately identify the identity and the security is low. Therefore, how to improve the security of identity authentication and improve the effect of detecting forgery and fraud has become a problem. Summary of the invention
[0003] In order to improve the security of identity authentication and enhance the effectiveness of detecting forgery and fraud, the present application provides a multimodal biometric feature comparison method and system.
[0004] In a first aspect, the present application provides a multimodal biometric feature comparison method, which adopts the following technical solution:
[0005] A multimodal biometric comparison method, comprising:
[0006] Obtain identity information, four-finger fingerprint features of the current person, and face image;
[0007] If the fingerprint feature is consistent with the preset fingerprint feature corresponding to the identity information, and the face image is consistent with the preset face image corresponding to the identity information, then control the red light emitting device to emit red light, so that the red light emitting device emits red light from below the four-linked finger, and emits infrared light to a designated area of the face of the current person;
[0008] Acquire the red light image of the fingers above the four-finger linkage and the infrared reflection spectrum of the designated area of the current face;
[0009] Based on the finger red light image, it is determined whether the four-finger sequence is the real four-finger sequence corresponding to the identity information, and based on the infrared reflection spectrum, it is determined whether the human face is the real human face corresponding to the identity information.
[0010] By adopting the above technical solution, the identity information of the current person, the fingerprint features of the four-finger joint and the face image are obtained, so that the corresponding preset fingerprint features and the preset face image are retrieved according to the identity information, the fingerprint features are compared with the preset fingerprint features, and the face image is compared with the preset face image, so as to achieve preliminary judgment of the biological features. After the comparison is consistent, the red light emitting device is controlled to emit visible red light to the bottom of the four-finger joint, and to emit infrared light to the designated area of the current person's face, so as to collect the finger red light image of the back of the four-finger joint and the infrared reflection spectrum reflected by the designated area. Since the skin, bones and other tissues at the four-finger joint of different people are different, , the absorption of visible red light is different, so the red light image of the finger presented on the back of the four-fingered finger is different, so that the four-fingered finger is judged whether it is forged or fake according to the red light image of the finger, that is, whether it is the real four-fingered finger of the person corresponding to the identity information, and the infrared reflection spectrum is used to judge whether the face is forged. Since the skin tissue of different people's faces is different, the absorption and reflection of infrared light are different, and thus the infrared spectrum is different. Therefore, the infrared reflection spectrum can be used to judge whether the face is forged and fake. By judging the four-fingered finger and the face, a multi-level judgment is made to see whether it is forged and fake, thereby improving security and improving the effect of detecting forgery and fake.
[0011] In another possible implementation, the determining, based on the finger red light image, whether the four-finger sequence is a real four-finger sequence corresponding to the identity information includes:
[0012] Acquire a preset finger red light image, where the preset finger red light image is a red light image above four consecutive fingers of the person corresponding to the identity information;
[0013] Determine a first histogram of red values within a four-finger range in the preset finger red light image, and a second histogram of red values within a four-finger range in the finger red light image;
[0014] Determining a first similarity between the first histogram and the second histogram;
[0015] Determine a first center point of each fingerprint of the four-finger fingerprint feature and a second center point of each fingerprint of the preset fingerprint feature;
[0016] Mapping and overlapping the finger red light image and the preset finger red light image based on the first center point and the second center point, and calculating the red value difference of the pixel at each position corresponding to the finger red light image;
[0017] Determine the variance of the red value difference of each finger, and determine the target pixel whose difference value is greater than a preset difference value;
[0018] Determine the distance between any target pixel and other target pixels in each finger, and calculate the average value of the distance;
[0019] determining a quality score for each finger based on the variance, the number of target pixels, and the average value;
[0020] It is determined whether the four-finger sequence is a real four-finger sequence corresponding to the identity information based on the quality score and the first similarity.
[0021] In another possible implementation, judging whether the four-digit finger is a real four-digit finger corresponding to the identity information based on the quality score and the first similarity includes:
[0022] If there is no target finger, or the number of target fingers reaches a preset number threshold, an average quality score is determined, and a first score is determined based on the average quality score, the first similarity, and the respective corresponding coefficients, and the target finger is a finger whose quality score does not reach the preset score value;
[0023] If the first score reaches a preset score threshold, determining that the four-finger sequence is a real four-finger sequence corresponding to the identity information;
[0024] If there are target fingers, and the number of target fingers reaches a preset number threshold, it is determined that the four-finger series is not the real four-finger series corresponding to the identity information.
[0025] In another possible implementation, if there are target fingers and the number of the target fingers reaches a preset number threshold, the method further includes:
[0026] Acquire image information of the full picture of the current person;
[0027] The motion trajectory of the current person is tracked based on the image information.
[0028] In another possible implementation manner, the obtaining of the infrared reflection spectrum of the specified area of the current face includes:
[0029] Randomly determine at least one preset instruction from a preset instruction library, and output the at least one preset instruction, so that the current person performs an action according to the at least one preset instruction;
[0030] Acquire multiple infrared reflection spectra of the current face during the action according to each preset instruction.
[0031] In another possible implementation, the infrared reflection spectrum is a plurality of infrared reflection spectra of the current face during an action according to each preset instruction, each preset instruction corresponds to a set of infrared reflection spectra, and judging whether the face belongs to a real face corresponding to the identity information based on the infrared reflection spectrum includes:
[0032] Calculating a second similarity between each infrared reflection spectrum in each group of infrared reflection spectrum and a preset infrared reflection spectrum;
[0033] Determine a difference between two adjacent second similarities, and generate a line graph corresponding to each preset instruction based on the difference;
[0034] Calculating a third similarity between each line graph and a plurality of preset line graphs in a corresponding preset database, each preset line graph is generated based on a plurality of infrared reflection spectra of a corresponding sample person's face during an action according to the preset instruction, and calculating a difference between the similarity with a preset infrared reflection spectrum of a specified area of the sample person's face, and the preset database is determined according to each preset instruction;
[0035] Calculating a fourth similarity between the infrared reflection spectrum of the designated area when the current person is not taking action and a preset infrared reflection spectrum;
[0036] Determine the number of third similarities in each group of infrared reflection spectra whose similarity reaches a first preset similarity threshold, and determine the proportion of the third similarities in the total number of the third similarities;
[0037] If there is at least one group of infrared reflection spectra whose proportion reaches the preset proportion, and the number of the at least one group of infrared reflection spectra accounts for the preset proportion, and the fourth similarity reaches the second preset similarity threshold, it is determined that the current face belongs to the real face corresponding to the identity information.
[0038] In a second aspect, the present application provides a multimodal biometric feature comparison system, which adopts the following technical solution:
[0039] A multimodal biometric comparison system, comprising:
[0040] The first data acquisition module is used to acquire identity information, fingerprint features of four consecutive fingers of the current person, and a face image;
[0041] a control module, configured to control a red light emitting device to emit red light when the fingerprint feature is consistent with a preset fingerprint feature corresponding to the identity information, and the face image is consistent with a preset face image corresponding to the identity information, so that the red light emitting device emits red light from below the four-linked finger and emits infrared light to a designated area of the face of the current person;
[0042] A second data acquisition module is used to acquire the red light image of the fingers above the four-finger linkage and the infrared reflection spectrum of the specified area of the current face;
[0043] A judgment module is used to judge whether the four-finger linkage is the real four-finger linkage corresponding to the identity information based on the finger red light image, and to judge whether the face is the real face corresponding to the identity information based on the infrared reflection spectrum.
[0044] By adopting the above technical solution, the first data acquisition module obtains the identity information of the current person, the fingerprint features of the four-finger connection and the face image, so as to retrieve the corresponding preset fingerprint features and the preset face image according to the identity information, compare the fingerprint features with the preset fingerprint features, and compare the face image with the preset face image, so as to achieve preliminary judgment of the biological features. After all the comparisons are consistent, the control module controls the red light emitting device to emit visible red light to the bottom of the four-finger connection, and emits infrared light to the designated area of the current person's face, so that the second data acquisition module can obtain the finger red light image of the back of the four-finger connection and the infrared reflection spectrum reflected by the designated area, because the skin at the four-finger connection of different people is consistent. , bones and other tissues are different, and the absorption of visible red light is different. Therefore, the red light image of the finger presented on the back of the four-linked finger is different. The judgment module judges whether the four-linked finger is forged or fake according to the red light image of the finger, that is, judges whether it is the real four-linked finger of the person corresponding to the identity information, and judges whether the face is forged according to the infrared reflection spectrum. Since the skin tissues of different people's faces are different, the absorption and reflection of infrared light are different, and thus the infrared spectra are different. Therefore, the infrared reflection spectrum can be used to judge whether the face is forged and fake. By judging the four-linked fingers and the face, multi-level judgment is made to see whether there is forgery and fake, thereby improving security and improving the effect of detecting forgery and fake.
[0045] In another possible implementation, when the judgment module judges whether the four-finger sequence is the real four-finger sequence corresponding to the identity information based on the finger red light image, it is specifically used to:
[0046] Acquire a preset finger red light image, where the preset finger red light image is a red light image above four consecutive fingers of the person corresponding to the identity information;
[0047] Determine a first histogram of red values within a four-finger range in the preset finger red light image, and a second histogram of red values within a four-finger range in the finger red light image;
[0048] Determining a first similarity between the first histogram and the second histogram;
[0049] Determine a first center point of each fingerprint of the four-finger fingerprint feature and a second center point of each fingerprint of the preset fingerprint feature;
[0050] Mapping and overlapping the finger red light image and the preset finger red light image based on the first center point and the second center point, and calculating the red value difference of the pixel at each position corresponding to the finger red light image;
[0051] Determine the variance of the red value difference of each finger, and determine the target pixel whose difference value is greater than a preset difference value;
[0052] Determine the distance between any target pixel and other target pixels in each finger, and calculate the average value of the distance;
[0053] determining a quality score for each finger based on the variance, the number of target pixels, and the average value;
[0054] It is determined whether the four-finger sequence is a real four-finger sequence corresponding to the identity information based on the quality score and the first similarity.
[0055] In another possible implementation, when the judgment module judges whether the four-digit finger is a real four-digit finger corresponding to the identity information based on the quality score and the first similarity, it is specifically configured to:
[0056] If there is no target finger, or the number of target fingers reaches a preset number threshold, an average quality score is determined, and a first score is determined based on the average quality score, the first similarity, and the respective corresponding coefficients, and the target finger is a finger whose quality score does not reach the preset score value;
[0057] If the first score reaches a preset score threshold, determining that the four-finger sequence is a real four-finger sequence corresponding to the identity information;
[0058] If there are target fingers, and the number of target fingers reaches a preset number threshold, it is determined that the four-finger series is not the real four-finger series corresponding to the identity information.
[0059] In another possible implementation, if there are target fingers and the number of the target fingers reaches a preset number threshold, the system further includes:
[0060] An image acquisition module, used to acquire image information of the full picture of the current person;
[0061] A tracking module is used to track the movement trajectory of the current person based on the image information.
[0062] In another possible implementation, when acquiring the infrared reflection spectrum of the specified area of the current face, the second data acquisition module is specifically used to:
[0063] Randomly determine at least one preset instruction from a preset instruction library, and output the at least one preset instruction, so that the current person performs an action according to the at least one preset instruction;
[0064] Acquire multiple infrared reflection spectra of the current face during the action according to each preset instruction.
[0065] In another possible implementation, the infrared reflection spectrum is a plurality of infrared reflection spectra of the current face during the action according to each preset instruction, each preset instruction corresponds to a set of infrared reflection spectra, and the judgment module is specifically used to judge whether the face belongs to the real face corresponding to the identity information based on the infrared reflection spectrum:
[0066] Calculating a second similarity between each infrared reflection spectrum in each group of infrared reflection spectrum and a preset infrared reflection spectrum;
[0067] Determine a difference between two adjacent second similarities, and generate a line graph corresponding to each preset instruction based on the difference;
[0068] Calculating a third similarity between each line graph and a plurality of preset line graphs in a corresponding preset database, each preset line graph is generated based on a plurality of infrared reflection spectra of a corresponding sample person's face during an action according to the preset instruction, and calculating a difference between the similarity with a preset infrared reflection spectrum of a specified area of the sample person's face, and the preset database is determined according to each preset instruction;
[0069] Calculating a fourth similarity between the infrared reflection spectrum of the designated area when the current person is not taking action and a preset infrared reflection spectrum;
[0070] Determine the number of third similarities in each group of infrared reflection spectra whose similarity reaches a first preset similarity threshold, and determine the proportion of the third similarities in the total number of the third similarities;
[0071] If there is at least one group of infrared reflection spectra whose proportion reaches the preset proportion, and the number of the at least one group of infrared reflection spectra accounts for the preset proportion, and the fourth similarity reaches the second preset similarity threshold, it is determined that the current face belongs to the real face corresponding to the identity information.
[0072] In a third aspect, the present application provides an electronic device, which adopts the following technical solution:
[0073] An electronic device, comprising:
[0074] at least one processor;
[0075] Memory;
[0076] At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and at least one is configured to: execute a multimodal biometric feature comparison method shown in any possible implementation of the first aspect.
[0077] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:
[0078] A computer-readable storage medium, when the computer program is executed in a computer, enables the computer to execute a multimodal biometric feature comparison method as described in any one of the first aspects.
[0079] In summary, the present application includes at least one of the following beneficial technical effects:
[0080] The identity information, fingerprint features of the four-finger joint and face image of the current person are obtained, so as to retrieve the corresponding preset fingerprint features and preset face image according to the identity information, compare the fingerprint features with the preset fingerprint features, and compare the face image with the preset face image, so as to realize the preliminary judgment of the biological features. After the comparison is consistent, the red light emitting device is controlled to emit visible red light to the bottom of the four-finger joint, and emit infrared light to the designated area of the current person's face, so as to collect the finger red light image of the back of the four-finger joint and the infrared reflection spectrum reflected by the designated area. Since the skin, bones and other tissues at the four-finger joint of different people are different, the visible red light is emitted to the bottom of the four-finger joint. The absorption of light is different, so the red light image of the finger presented on the back of the four-fingered finger is different, so as to judge whether the four-fingered finger is forged or fake according to the red light image of the finger, that is, to judge whether it is the real four-fingered finger of the person corresponding to the identity information, and to judge whether the face is forged according to the infrared reflection spectrum. Since the skin tissue of different people's faces is different, the absorption and reflection of infrared light are different, and thus the infrared spectrum is different. Therefore, according to the infrared reflection spectrum, it can be judged whether the face is forged and faked. By judging the four-fingered finger and the face, a multi-level judgment is made to judge whether there is forgery and fake, thereby improving security and improving the effect of detecting forgery and fake. BRIEF DESCRIPTION OF THE DRAWINGS
[0081] Figure 1 It is a flowchart of a multimodal biometric feature comparison method according to an embodiment of the present application.
[0082] Figure 2 It is a structural schematic diagram of a multimodal biometric feature comparison system according to an embodiment of the present application.
[0083] Figure 3 It is a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0084] The present application is further described in detail below in conjunction with the accompanying drawings.
[0085] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they are within the scope of the claims of this application.
[0086] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0087] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.
[0088] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.
[0089] The embodiment of the present application provides a multimodal biometric feature comparison method, which is performed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and the embodiment of the present application does not limit this. Figure 1 As shown, the method includes step S101, step S102, step S103 and step S104, wherein:
[0090] S101, obtaining identity information, fingerprint features of four consecutive fingers of the current person, and a face image.
[0091] For the embodiments of the present application, the electronic device is connected to the border inspection machine via wired or wireless connection, and is connected to the license inspection device via wired or wireless connection. The current person inputs identity information through the license inspection device, or inputs identity information through the border inspection machine, so that the electronic device obtains the identity information. The storage medium in the electronic device or the cloud server stores biometric features corresponding to the identity information, such as facial images, fingerprints and other biometric features. When the biometric features of the current person match the biometric features corresponding to the stored identity information, it can be determined that the identity of the current person is correct. The border inspection machine is also provided with a four-finger fingerprint collection device to collect the fingerprint features of the current person and enable the electronic device to obtain them. Similarly, the border inspection machine is also provided with an image collection device, such as a camera, to collect the facial image of the current person and enable the electronic device to obtain it.
[0092] S102, if the fingerprint feature is consistent with the preset fingerprint feature corresponding to the identity information, and the face image is consistent with the preset face image corresponding to the identity information, the red light emitting device is controlled to emit red light, so that the red light emitting device emits red light from under the four-linked fingers, and emits infrared light to the designated area of the current person's face.
[0093] For the embodiment of the present application, after the electronic device obtains the identity information, it retrieves the fingerprint features of the four consecutive fingers and the facial image of the real person corresponding to the identity information from the local storage medium or the cloud server, that is, the preset fingerprint features and the preset facial image. And the acquired fingerprint features of the four consecutive fingers of the current person are compared with the preset fingerprint features, and the facial image is compared with the preset facial image. Specifically, the electronic device compares the fingerprint features with the preset fingerprint features through a trained network model to determine whether the fingerprint features of the current four consecutive fingers are consistent with the preset fingerprint features of the identity information, and also uses the trained network model to determine whether the facial image of the current person is consistent with the preset facial image of the identity information. The network model can be a convolutional neural network model, a recurrent neural network model, or other types of network models, which are not limited here. The above-mentioned method of identity recognition through the fingerprint features of the four consecutive fingers and the facial image is a prior art and will not be repeated here.
[0094] If the fingerprint features and facial images are consistent with the preset fingerprint features and facial images corresponding to the identity information, it means that the preliminary verification has passed. Since fingerprints and faces are at risk of forgery, further verification is required. The border inspection machine is equipped with a red light emitting device, which is arranged below the fingerprint collection point and can emit visible red light and infrared light. The wavelengths of red light and infrared light can be set according to actual usage. The electronic device controls the red light emitting device to emit red light, which shines from under the current person's finger, cooperates with the camera to identify the designated area of the current person's face, and controls the red light emitting device to emit infrared light to illuminate the designated area. The red light emitting device emits red light and infrared light to obtain other optical data of the current person, so as to facilitate the subsequent verification of whether the identity of the current person is consistent with the identity of the person corresponding to the identity information based on the optical data.
[0095] S103, obtaining a red light image of the fingers above the four-finger daisy chain and an infrared reflection spectrum of a designated area of the current face.
[0096] For the embodiment of the present application, after the red light emitting device emits visible red light to the finger of the current person, the skin, blood, bones and other tissues of the four fingers absorb part of the red light, and because the skin, blood, bones and other tissues of the four fingers are different from person to person, the red light presented on the back of the finger is different, that is, after each person is irradiated with visible red light, the red light image presented on the back of the finger is specific, which can be used as a basis for determining whether the fingerprint is forged. Therefore, an image acquisition device, such as a camera, is provided on the border inspection machine, and the camera is provided above the fingerprint collection point to collect the red light image of the back of the finger, that is, the red light image of the finger.
[0097] Similarly, infrared light has a certain degree of penetration and can reflect the structure and tissue characteristics of the deep layer of the skin. The facial skin of different people has differences in tissue structure, moisture content, blood flow, etc. These differences can ensure that the characteristics of infrared reflected light are different. Therefore, by emitting infrared light to the designated area of the current person's face, and the device receiving infrared reflected light on the border inspection machine receives the reflected infrared light data and generates an infrared reflection spectrum, a deeper level of identity authentication is performed through the infrared reflection spectrum to detect facial forgery as much as possible.
[0098] S104, judging whether the four-finger sequence is the real four-finger sequence corresponding to the identity information based on the red light image of the fingers, and judging whether the face is the real face corresponding to the identity information based on the infrared reflection spectrum.
[0099] For the embodiments of the present application, since the skin, blood, bones and other tissues on the fingers of different people are different, after the visible red light is irradiated on the fingers, the image of the visible red light on the back of the fingers is collected, which facilitates the electronic device to analyze whether the currently verified four-finger connection is a real four-finger connection.
[0100] After the electronic device determines that the four-finger sequence is the real four-finger sequence of the person corresponding to the identity information, it further determines whether the face of the current person is the real face corresponding to the identity information through the infrared reflection spectrum, that is, through further red light images of the fingers and infrared reflection spectra of the specified area of the face, it determines whether there is forgery and fraud in the current identity authentication process, thereby improving accuracy and reducing the possibility of passing border inspection by forging identity features.
[0101] In a possible implementation of the embodiment of the present application, in step S104, judging whether the four-finger sequence is the real four-finger sequence corresponding to the identity information based on the red light image of the finger, specifically includes step S1041 (not shown in the figure), step S1042 (not shown in the figure), step S1043 (not shown in the figure), step S1044 (not shown in the figure), step S1045 (not shown in the figure), step S1046 (not shown in the figure), step S1048 (not shown in the figure) and step S1049 (not shown in the figure), wherein:
[0102] S1041, obtaining a preset finger red light image.
[0103] Among them, the preset finger red light image is the red light image above the four-linked fingers of the person corresponding to the identity information.
[0104] For the embodiment of the present application, when the person corresponding to the identity information initially records the fingerprint information of four fingers, the border inspection machine same as in the above embodiment can be used to record biometric features, such as the fingerprint features of four fingers and the face image. When recording the fingerprint features of four fingers, a red light emitting device emits red light to irradiate the bottom of the four fingers, and a camera device collects a red light image of the back of the four fingers, that is, a preset finger red light image. The preset finger red light image can be stored in a storage medium inside the electronic device or in a cloud server. The electronic device determines the definite relationship between the preset finger red light image and the identity information, thereby facilitating subsequent verification and comparison.
[0105] S1042, determining a first histogram of red values within a four-finger range in a preset finger red light image, and a second histogram of red values within a four-finger range in the finger red light image.
[0106] For the embodiment of the present application, after the electronic device determines the finger red light image, it can first pre-process the finger red light image. Due to the situation of nail growth and trimming or girls doing manicure, the electronic device can segment the finger red light image and remove the local image of the nail part. Specifically, the finger red light image is grayed out, and the gray gradient is used to remove the finger red light image of the nail part along the edge of the nail, and the remaining finger red light image is used for subsequent analysis, thereby reducing the error at the nail. For the preset finger red light image, the electronic device can also use the above method to remove the finger red light image at the nail, and use the remaining preset finger red light image for subsequent analysis.
[0107] Since both the preset finger red light image and the finger red light image are formed by irradiating visible red light as the backlight source, the electronic device can directly process and obtain the red value of each pixel in the two images, thereby determining the first histogram corresponding to the preset finger red light image and the second histogram corresponding to the finger red light image.
[0108] S1043: Determine a first similarity between the first histogram and the second histogram.
[0109] For the embodiment of the present application, the histogram is used to display the distribution of pixel intensity of the red channel in the image. The X-axis of the histogram represents the pixel value (ranging from 0 to 255), and the Y-axis represents the number of pixels with the pixel value in the image. Through the histogram of the red value, the brightness distribution of the red part of the image can be understood. The electronic device calculates the first similarity between the first histogram and the second histogram, thereby determining from the color distribution of the two images whether the four consecutive fingers of the current person are similar to the preset finger red light image after being irradiated with visible red light. If similar, it can also be used as a factor to prove that the current person has not forged or falsified. The first similarity is only a single factor in determining forgery, and it is necessary to combine other data for comprehensive determination. Specifically, the electronic device can calculate the similarity by calculating the Bhattacharyya distance, Euclidean distance of the two histograms, or by using the histogram intersection method.
[0110] S1044, determining a first center point of each fingerprint of the four-finger fingerprint feature and a second center point of each fingerprint of the preset fingerprint feature.
[0111] For the embodiment of the present application, the electronic device can first divide the fingerprint into regions, pre-process the fingerprint image into blocks, divide it into multiple regions, and then determine the search region moving path, and introduce an adaptive path selection scheme based on the statistical analysis method. By analyzing the characteristics of each region, the search region moving path for center point positioning is determined, and finally the center point detection is performed, and the fingerprint information in the current region is detected using a composite filter algorithm. The center point of each fingerprint feature is determined in the above manner.
[0112] S1045, mapping and overlapping the finger red light image and the preset finger red light image based on the first center point and the second center point, and calculating the red value difference of the pixel at each corresponding position.
[0113] For the embodiment of the present application, after determining the first center point and the second center point, since each person does not change much in the movement of the fingers when collecting the fingerprint features of four consecutive fingers due to personal habits and the constraints of the hand structure, the electronic device aligns and overlaps the finger red light image and the preset finger red light image according to the first center point and the second center point, thereby reducing the error of subsequent calculations. After alignment, the electronic device sequentially calculates the difference in the red value of the pixel corresponding to each position.
[0114] S1046, determining the variance of the red value difference of each finger, and determining the target pixel whose difference value is greater than the preset difference value.
[0115] For the embodiments of the present application. The red value difference is used to characterize the degree of pixel change at the same position. The smaller the difference, the closer it is, and the larger the difference, the less similar it is to the preset finger red light image. The electronic device calculates the variance of all differences. The variance is used as the floating property of the difference. The larger the floating property, the greater the difference from the preset red light image. The preset difference is used as a demarcation point to determine whether the pixels at the same position deviate too much. If the difference at a certain pixel is greater than the preset difference, it means that the pixel is suspicious and differs too much from the same position of the preset finger red light image, that is, the target pixel. The more target pixels there are, the more likely the current finger red light image is to be forged.
[0116] S1047, determining the distance between any target pixel and other target pixels in each finger, and calculating the average value of the distance.
[0117] For the embodiment of the present application, the electronic device can obtain the distance between any target pixel and other target pixels by calculating the Euclidean distance. The larger the distance, the more dispersed the target pixels are, and the greater the possibility of random errors. The smaller the distance, the more concentrated the target pixels are, and the larger the area that is not close to the preset finger red light image. For example, if there is forgery in the area representing the phalanges in the red light images of two fingers, the more target pixels there are, and the smaller the average distance, the higher the possibility of forgery. Therefore, the electronic device calculates the average value of the distance and uses the average value to represent the overall difference between each finger and the preset finger red light image.
[0118] S1048, determining a quality score for each finger based on the variance, the number of target pixels, and the average value.
[0119] For the embodiment of the present application, the larger the variance, the greater the pixel difference at the same position is, and the variance is inversely proportional to the quality of the finger; the larger the number of target pixels, the more places there are with the preset finger red light image, and the number of target fingers is inversely proportional to the quality of the finger; the larger the average value, the larger the average value, the more cases where the distance between the target pixels is larger, the greater the possibility of accidental error, the higher the quality, and the average value is proportional to the quality of the finger. The above-mentioned variance, the number of target pixels and the average value are all important factors affecting the consistency between the finger red light image of each finger and the same finger in the preset finger red light image, and the degree of influence is different. In order to facilitate the actual and more accurate quality score, after determining the variance and the number of target pixels, the inverse of the variance and the number of target pixels is taken for calculation. The larger the inverse of the variance and the number of target pixels, the higher the finger quality. Therefore, the electronic device can obtain a score by setting different coefficients for the above three weighted calculations, or set appropriate coefficients according to the proportional relationship between the above three and the quality to calculate the quality score, and the coefficients are set and adjusted according to the actual situation and needs.
[0120] S1049: Determine whether the four-digit finger is the real four-digit finger corresponding to the identity information based on the quality score and the first similarity.
[0121] For the embodiment of the present application, after determining the quality score of each finger, the first similarity is used to comprehensively judge whether the four-finger connection is the real four-finger connection corresponding to the identity information. The first similarity of the histogram representing the distribution of red values is combined with the quality score representing the degree of proximity of each finger to the same finger in the preset finger red light image to comprehensively judge the authenticity of the four-finger connection more accurately.
[0122] In other embodiments, the electronic device can also control the camera device on the border inspection machine to collect images of the back of the third joint of each finger of the four-jointed finger. Because body hair grows on the back of the third joint, and because of the differences in growth levels, genes and other factors of different people, the distribution of body hair at this position is special, and usually there is less human intervention in the growth of body hair at this location. The characteristics of the body hair at this location are relatively stable and can be identified as biological features. Therefore, the electronic device performs denoising and other processing on the image of the back of the third joint, and then performs grayscale conversion and other operations to obtain the number and distribution of sweat pores at the third joint of each finger. The electronic device stores a preset body hair image of the person corresponding to the identity information, and then determines the distribution and number of body hair sweat pores at the third joint of each finger of the benchmark according to the preset body hair image. Based on the distribution and number of sweat pores at the third joint of each finger of the current person, the proximity to each finger in the preset body hair image is calculated, and the variance, the number of target pixels and the average value in the above embodiment are combined, so as to further serve as a key factor in determining the quality of each finger, and further improve the security of the comparison of biometric features of border inspection entry and exit.
[0123] In a possible implementation of the embodiment of the present application, in step S1049, judging whether the four-finger sequence is the real four-finger sequence corresponding to the identity information based on the quality score and the first similarity specifically includes step S1 (not shown in the figure), step S2 (not shown in the figure), and step S3 (not shown in the figure), wherein:
[0124] S1, if there is no target finger, or the number of target fingers does not reach a preset number threshold, determine an average quality score, and determine a first score based on the average quality score, the first similarity, and the corresponding coefficients.
[0125] Among them, the target finger is the finger whose quality score does not reach the preset score.
[0126] For the embodiment of the present application, if the target finger does not exist, or the number of target fingers does not reach the preset number threshold, it means that the quality of the four fingers is high and close to the preset finger red light image, or it means that the existence of the target finger is an accidental error. Therefore, the electronic device calculates the average of the four quality scores to represent the overall quality score, and combines the first similarity, sets different coefficients for the first similarity and the average of the quality score to obtain a first score by weighted calculation. The first score represents the degree of closeness between the current person's four consecutive fingers and the preset finger red light image, that is, there is a possibility of forgery or forgery.
[0127] S2: If the first score reaches a preset score threshold, determining that the four-finger sequence is a real four-finger sequence corresponding to the identity information.
[0128] For the embodiment of the present application, if the first score reaches the preset score threshold, it means that the four-finger combo of the current person is close to the four-finger combo corresponding to the identity information, and it can be proved to be a real four-finger combo. If the first score threshold is not reached, it means that the first similarity is too low, that is, there is a suspicious situation in the distribution of the number of red values, and the four-finger combo may be forged or fake. The electronic device can output a prompt message to remind the relevant personnel.
[0129] S3: If there are target fingers and the number of target fingers reaches a preset number threshold, determine the real four-finger connection corresponding to the four-finger connection non-identity information.
[0130] For the embodiment of the present application, if there are target fingers and the number of target fingers reaches the preset number threshold, it means that there are more problematic fingers, and it can be directly determined that the four-finger combination of the current person is not the four-finger combination corresponding to the identity information. For example, if the preset number threshold is 2, it can be determined that the number of target fingers reaches half to determine whether there is forgery or fraud. It is more accurate to judge the quality of each finger of the four-finger combination and then determine whether there is forgery based on the number of fingers with poor quality.
[0131] In a possible implementation of the embodiment of the present application, if there are target fingers and the number of target fingers reaches a preset number threshold, step S3 further includes step S4 (not shown in the figure) and step S5 (not shown in the figure), wherein:
[0132] S4, obtaining the full-image image information of the current person.
[0133] In the embodiment of the present application, image information is collected by a camera device installed on a border inspection machine or a camera device in the surrounding environment and sent to an electronic device, so that the electronic device can obtain image information of the current person's full appearance.
[0134] S5, tracking the movement trajectory of the current person based on the image information.
[0135] For the embodiment of the present application, after the electronic device determines that there is forgery or fraud on the four fingers where the current person is located, the current person may flee the scene. Therefore, the electronic device performs target recognition on the surveillance video of the surrounding camera devices based on the image information of the border inspection machine, thereby tracking the current person, making it easier for relevant personnel to know the whereabouts of the current person.
[0136] In other embodiments, an ultraviolet stamp device may be further provided in the border inspection machine, which can print an ultraviolet stamp on the back of the four-fingered hand of the current person. When the electronic device determines that the four-fingered hand of the current person does not correspond to the real four-fingered hand of the identity information, the electronic device controls the ultraviolet stamp device to stamp an ultraviolet stamp on the back of the four-fingered hand of the current person as a mark. If the current person tries to escape and changes his appearance by changing clothes, or if the current person hides in a blind spot of monitoring, the electronic device will fail to track the person through various monitoring. Therefore, the relevant personnel can use ultraviolet lamps or ultraviolet imaging cameras to check the on-site personnel one by one to find the person.
[0137] In a possible implementation of the embodiment of the present application, the infrared reflection spectrum of the specified area of the current face is obtained in step S103, which specifically includes step S1031 (not shown in the figure) and step S1032 (not shown in the figure), wherein:
[0138] S1031, randomly determine at least one preset instruction from the preset instruction library, and output at least one preset instruction, so that the current person takes action according to the at least one preset instruction.
[0139] For the embodiment of the present application, the preset instruction library can be stored in a storage medium in the electronic device or in a cloud server. Each instruction in the preset instruction library represents an action, such as turning the head slightly to the left, turning the head slightly to the right, nodding slightly, etc. The electronic device can use the relevant existing random algorithm to determine at least one preset instruction. The electronic device outputs the voice information of at least one preset instruction through the speaker device on the border inspection machine to prompt the current person to perform the action in each instruction.
[0140] Specifically, after the electronic device determines that the four-finger sequence is the real four-finger sequence corresponding to the identity information, in order to further determine whether the face is forged, the number of preset instructions can be determined according to the difference between the first score and the preset score threshold. The larger the difference, the larger the first score, and the more likely it is that it is a real four-finger sequence. Therefore, a small number of preset instructions can be determined based on the preset coefficients and differences. Conversely, the smaller the difference, the more it just meets the preset score threshold, and it is judged to be a real four-finger sequence, but there is a certain possibility that it is a high-quality forgery and a misjudgment. Therefore, multiple preset instructions can be determined according to the preset coefficients and differences, so that the electronic device can obtain a large number of infrared reflection spectra of specified areas of the face, and perform more rigorous analysis on face judgment, thereby further improving security.
[0141] S1032, obtaining multiple infrared reflection spectra of the current face during the action according to each preset instruction.
[0142] For the embodiment of the present application, after the red light emitting device emits infrared light to the designated position of the current person's face, the infrared reflection spectrum during the current face movement can be obtained, and the infrared receiving device on the border inspection machine receives the reflected infrared light and generates an infrared reflection spectrum so that the electronic device can obtain it. By determining at least one preset instruction, it is convenient for the electronic device to obtain a suitable number of infrared reflection spectra, thereby improving the accuracy and security of subsequent analysis of whether the face is forged or fake.
[0143] In a possible implementation of the embodiment of the present application, the infrared reflection spectrum is a plurality of infrared reflection spectra of the current face in the process of following the preset instruction action. In step S104, based on the infrared reflection spectrum and the finger red light image, it is determined whether the four-finger continuous finger and the face belong to the real face corresponding to the identity information, specifically including step Sa (not shown in the figure), step Sb (not shown in the figure), step Sc (not shown in the figure), step Sd (not shown in the figure), step Se (not shown in the figure) and step Sf (not shown in the figure), wherein:
[0144] Sa, calculating a second similarity between each infrared reflection spectrum in each group of infrared reflection spectrum and a preset infrared reflection spectrum.
[0145] For the embodiment of the present application, in order to improve the accuracy of facial feature comparison, the electronic device border inspection machine can output voice prompt information through the speaker, such as "turn your head slightly", "nod slightly" and other prompt information. The border inspection machine collects multiple sets of infrared reflection spectra of the current person in the process of doing the above actions, and one preset instruction corresponds to a set of infrared reflection spectra. And the electronic device calculates the second similarity between each infrared reflection spectrum and the preset infrared reflection spectrum through the trained network model. As the current person makes an action, the infrared reflection spectrum of the specified area gradually changes, and the change is characterized by the second similarity.
[0146] Sb, determining the difference between two adjacent second similarities, and generating a line graph corresponding to each preset instruction based on the difference.
[0147] For the embodiment of the present application, the electronic device calculates the difference between two adjacent second similarities in each group of infrared reflection spectra and generates a line graph through a pre-stored coordinate system.
[0148] Sc, calculating a third similarity between each line graph and a plurality of preset line graphs in a corresponding preset database.
[0149] Among them, each preset line graph is generated based on multiple infrared reflection spectra of the corresponding sample person's face during the action according to the preset instruction, and the difference in similarity with the preset infrared reflection spectrum of the specified area of the sample person's face is calculated. The preset database is determined according to each preset instruction.
[0150] For the embodiment of the present application, when each person follows the instructions, the infrared reflection spectrum of the designated area of the face changes regularly, that is, the infrared reflection spectrum of the designated area when the sample person does not make any action is used as the preset infrared reflection spectrum, and the second similarity between the preset infrared spectrum and the infrared reflection spectrum of the designated area during the sample person's action is calculated in sequence, and the difference between two adjacent second similarities is determined, the difference is used to characterize the change of the second similarity, and a line graph is generated based on the difference.
[0151] The preset line graph in the preset database is a line graph formed based on the similarity between the infrared reflection spectrum of the sample personnel and the corresponding preset infrared reflection spectrum when the sample personnel perform actions according to the same instructions, as well as the difference between adjacent similarities. There may be slight differences in the line graphs of different sample personnel, but the similarity between the preset line graphs is high, which can ensure the accuracy of calculating the third similarity.
[0152] Different actions form different infrared reflection spectra, but when each person follows the instructions, the infrared reflection spectrum of the designated area of the face changes regularly. For example, person A does not forge or falsify. When he follows preset instructions a, preset instructions b and preset instructions c respectively, three sets of infrared reflection spectra are formed respectively. The electronic device retrieves the preset database corresponding to the above three preset instructions, and calculates the third similarities respectively. The three third similarities obtained are all high. If person B is a high-quality facial forgery or falsification, the third similarity calculated based on multiple infrared reflection spectra corresponding to the action of a preset instruction has a certain degree of randomness, that is, a high third similarity is obtained that is sufficient to be determined as a real face. Since person B is a forged or falsified face, when acting according to more than one preset instruction, the electronic device calculates the third similarity respectively. Since it is a forged or falsified face, there are some cases where the third similarity is too low, thereby improving the security of verification, that is, infrared reflection spectra of multiple actions are required for comprehensive verification.
[0153] Sd, calculates the fourth similarity between the infrared reflection spectrum of the specified area when the current person is not taking any action and the preset infrared reflection spectrum.
[0154] For the embodiment of the present application, the electronic device calculates the fourth similarity of the two infrared reflection spectra through the trained network model, which will not be repeated here. When not in action, the fourth similarity with the preset infrared reflection spectrum can be used as an important factor in determining whether it is a real face corresponding to the identity information.
[0155] Se, determine the number of third similarities whose similarity reaches a first preset similarity threshold in each group of infrared reflection spectra, and determine the proportion of the number to the total number of third similarities.
[0156] For the embodiment of the present application, reaching the first preset similarity threshold indicates that the face is closer to the preset line graph in the preset database. And the more the number is, the more the current face conforms to the regularity of the change of the infrared reflection spectrum during the action. Therefore, the electronic device determines the proportion of the number to the total number of all third similarities. The larger the proportion is, the more it conforms to the regularity. The electronic device determines to process each group of infrared reflection spectra in the same way as above.
[0157] Sf, if there is at least one group of infrared reflection spectra whose proportion reaches the preset proportion, and the number of at least one group of infrared reflection spectra accounts for the preset proportion, and the fourth similarity reaches the second preset similarity threshold, it is determined that the current face belongs to the real face corresponding to the identity information.
[0158] For the embodiment of the present application, the proportion reaching the preset proportion indicates the regularity of the spectrum changing with the action, which can to a certain extent illustrate the real face corresponding to the identity information, and the more the number of groups of infrared reflection spectra that reach the preset proportion is, the more it can illustrate that the current face is not forged or faked, and combined with the fourth similarity between the specified position and the preset infrared reflection spectrum when there is no action, if the fourth similarity reaches the second preset similarity threshold at the same time, it means that the current person has a high degree of adaptability to the real face corresponding to the identity information, and it can be determined that it is a real face, and the possibility of forgery or fakeness is low. Through the above method, the forgery of four fingers and face is eliminated in turn, so that the identity authentication and comparison of biometric features are more hierarchical and more secure.
[0159] The above embodiment introduces a multimodal biometric feature comparison method from the perspective of method flow, and the following embodiment introduces a multimodal biometric feature comparison system from the perspective of a virtual module or a virtual unit. For details, please refer to the following embodiment.
[0160] The present application embodiment provides a multi-modal biometric feature comparison system 20, such as Figure 2 As shown, the multimodal biometric feature comparison system 20 may specifically include:
[0161] The first data acquisition module 201 is used to acquire identity information, fingerprint features of four consecutive fingers of the current person, and a face image;
[0162] The control module 202 is used to control the red light emitting device to emit red light when the fingerprint feature is consistent with the preset fingerprint feature corresponding to the identity information, and the face image is consistent with the preset face image corresponding to the identity information, so that the red light emitting device emits red light from below the four-linked fingers and emits infrared light to a designated area of the face of the current person;
[0163] The second data acquisition module 203 is used to acquire the red light image of the finger above the four-finger tandem and the infrared reflection spectrum of the current designated area of the face;
[0164] The judgment module 204 is used to judge whether the four-finger linkage is the real four-finger linkage corresponding to the identity information based on the finger red light image, and judge whether the face is the real face corresponding to the identity information based on the infrared reflection spectrum.
[0165] The embodiment of the present application discloses a multimodal biometric feature comparison system 20, wherein the first data acquisition module 201 acquires the identity information of the current person, the fingerprint features of the four-finger digit, and the face image, so as to retrieve the corresponding preset fingerprint features and the preset face image according to the identity information, compare the fingerprint features with the preset fingerprint features, and compare the face image with the preset face image, so as to achieve a preliminary judgment of the biometric features. After all the comparisons are consistent, the control module 202 controls the red light emitting device to emit visible red light to the bottom of the four-finger digit, and emit infrared light to the designated area of the current person's face, so that the second data acquisition module 203 obtains the finger red light image of the back of the four-finger digit and the infrared reflection spectrum reflected by the designated area, Since the skin, bones and other tissues at the four-linked fingers of different people are different and absorb visible red light differently, the red light images of the fingers presented on the back of the four-linked fingers are different. Therefore, the judgment module 204 judges whether the four-linked fingers are forged or faked based on the red light images of the fingers, that is, judges whether they are the real four-linked fingers of the person corresponding to the identity information, and judges whether the face is forged based on the infrared reflection spectrum. Since the skin tissues on the faces of different people are different and absorb and reflect infrared light differently, the infrared spectra are different. Therefore, the infrared reflection spectrum can be used to judge whether the face is forged and faked. By judging the four-linked fingers and the face to make multi-level judgments on whether there is forgery and fakeness, security is improved and the effect of detecting forgery and fakeness is improved.
[0166] In a possible implementation of the embodiment of the present application, when the judgment module 204 judges whether the four-finger joint is the real four-finger joint corresponding to the identity information based on the finger red light image, it is specifically used to:
[0167] Obtaining a preset finger red light image, where the preset finger red light image is a red light image above four consecutive fingers of a person corresponding to the identity information;
[0168] Determine a first histogram of red values within a four-finger range in a preset finger red light image, and a second histogram of red values within a four-finger range in the finger red light image;
[0169] Determining a first similarity between the first histogram and the second histogram;
[0170] Determine a first center point of each fingerprint of the four-finger fingerprint feature, and a second center point of each fingerprint of the preset fingerprint feature;
[0171] Mapping and overlapping the finger red light image and the preset finger red light image based on the first center point and the second center point, and calculating the red value difference of the pixel at each corresponding position;
[0172] Determine the variance of the red value difference of each finger, and determine the target pixel whose difference value is greater than a preset difference value;
[0173] Determine the distance between any target pixel and other target pixels in each finger, and calculate the average value of the distance;
[0174] Determine the quality score of each finger based on the variance, the number of target pixels, and the mean;
[0175] It is determined whether the four-finger sequence is the real four-finger sequence corresponding to the identity information based on the quality score and the first similarity.
[0176] In a possible implementation of the embodiment of the present application, when the judgment module 204 judges whether the four-finger sequence is the real four-finger sequence corresponding to the identity information based on the quality score and the first similarity, it is specifically used to:
[0177] If there is no target finger, or the number of target fingers reaches a preset number threshold, an average quality score is determined, and a first score is determined based on the average quality score, the first similarity, and the respective corresponding coefficients, and the target finger is a finger whose quality score does not reach the preset score value;
[0178] If the first score reaches a preset score threshold, the four-finger sequence is determined to be the real four-finger sequence corresponding to the identity information;
[0179] If there are target fingers and the number of target fingers reaches a preset number threshold, the real four-finger connection corresponding to the four-finger non-identity information is determined.
[0180] In a possible implementation of the embodiment of the present application, if there are target fingers and the number of target fingers reaches a preset number threshold, the system 20 further includes:
[0181] An image acquisition module is used to obtain the image information of the current person's full appearance;
[0182] The tracking module is used to track the movement trajectory of the current person based on image information.
[0183] In a possible implementation of the embodiment of the present application, when the second data acquisition module 203 acquires the infrared reflection spectrum of the current specified area of the face, it is specifically used to:
[0184] Randomly determine at least one preset instruction from a preset instruction library, and output at least one preset instruction so that the current person performs an action according to the at least one preset instruction;
[0185] Obtain multiple infrared reflection spectra of the current face during the action according to each preset instruction.
[0186] In a possible implementation of the embodiment of the present application, the infrared reflection spectrum is a plurality of infrared reflection spectra of the current face in the process of following each preset instruction, each preset instruction corresponds to a set of infrared reflection spectra, and the judgment module 204 is specifically used to judge whether the face belongs to the real face corresponding to the identity information based on the infrared reflection spectrum:
[0187] Calculating a second similarity between each infrared reflection spectrum in each group of infrared reflection spectrum and a preset infrared reflection spectrum;
[0188] Determine a difference between two adjacent second similarities, and generate a line graph corresponding to each preset instruction based on the difference;
[0189] Calculating a third similarity between each line graph and a plurality of preset line graphs in a corresponding preset database, wherein each preset line graph is generated based on a plurality of infrared reflection spectra of a corresponding sample person's face during an action according to a preset instruction, and calculating a difference between the similarity with a preset infrared reflection spectrum of a specified area of the sample person's face, and the preset database is determined according to each preset instruction;
[0190] Calculating a fourth similarity between the infrared reflection spectrum of the designated area when the current person is not moving and the preset infrared reflection spectrum;
[0191] Determine the number of third similarities in each group of infrared reflection spectra whose similarity reaches a first preset similarity threshold, and determine the proportion of the number to the total number of third similarities;
[0192] If there is at least one group of infrared reflection spectra whose proportion reaches the preset proportion, and the number of at least one group of infrared reflection spectra accounts for the preset proportion, and the fourth similarity reaches the second preset similarity threshold, it is determined that the current face belongs to the real face corresponding to the identity information.
[0193] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the multimodal biometric feature comparison system 20 described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.
[0194] An electronic device is provided in an embodiment of the present application, such as Figure 3 As shown, Figure 3 The electronic device 30 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 30 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 30 does not constitute a limitation on the embodiments of the present application.
[0195] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0196] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but it does not mean that there is only one bus or only one type of bus.
[0197] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.
[0198] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above method embodiment.
[0199] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0200] The embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment. Compared with the related art, the embodiment of the present application obtains the identity information of the current person, the fingerprint features of the four-finger connection, and the face image, so as to retrieve the corresponding preset fingerprint features and the preset face image according to the identity information, compare the fingerprint features with the preset fingerprint features, and compare the face image with the preset face image, so as to achieve a preliminary judgment of the biometric features. After all the comparisons are consistent, the red light emitting device is controlled to emit visible red light to the bottom of the four-finger connection, and to emit infrared light to the specified area of the current person's face, so as to collect the finger red light image of the back of the four-finger connection and the infrared reflection spectrum reflected by the specified area. Due to the different skin, bones and other tissues at the four-finger connection of different people, the red light emitting device can emit visible red light to the bottom of the four-finger connection, and emit infrared light to the specified area of the current person's face, so as to collect the finger red light image of the back of the four-finger connection and the infrared reflection spectrum reflected by the specified area. The four fingers are different and absorb visible red light differently, so the red light images of the fingers presented on the back are different. Therefore, it is possible to judge whether the four fingers are forged or fake based on the red light images of the fingers, that is, to judge whether it is the real four fingers of the person corresponding to the identity information, and to judge whether the face is forged based on the infrared reflection spectrum. Since the skin tissues of different people's faces are different, the absorption and reflection of infrared light are different, and thus the infrared spectra are different. Therefore, it is possible to judge whether the face is forged or fake based on the infrared reflection spectrum. By judging the four fingers and the face to make multi-level judgments on whether there is forgery or fake, the security is improved and the effect of detecting forgery and fake is improved.
[0201] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0202] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A multimodal biometric feature comparison method, characterized in that: include: Obtain identity information, four-finger fingerprint features of the current person, and face image; If the fingerprint feature is consistent with the preset fingerprint feature corresponding to the identity information, and the face image is consistent with the preset face image corresponding to the identity information, then control the red light emitting device to emit red light, so that the red light emitting device emits red light from below the four-linked finger, and emits infrared light to a designated area of the face of the current person; Acquire the red light image of the fingers above the four-finger linkage and the infrared reflection spectrum of the designated area of the current face; Determining whether the four-finger sequence is the real four-finger sequence corresponding to the identity information based on the finger red light image, and determining whether the human face is the real human face corresponding to the identity information based on the infrared reflection spectrum; The step of obtaining the infrared reflection spectrum of the specified area of the current face includes: Randomly determine at least one preset instruction from a preset instruction library, and output the at least one preset instruction, so that the current person performs an action according to the at least one preset instruction; Acquire multiple infrared reflection spectra of the current face during the action according to each preset instruction; Wherein, judging whether the four-finger sequence is a real four-finger sequence corresponding to the identity information based on the finger red light image includes: Acquire a preset finger red light image, where the preset finger red light image is a red light image above four consecutive fingers of the person corresponding to the identity information; Determine a first histogram of red values within a four-finger range in the preset finger red light image, and a second histogram of red values within a four-finger range in the finger red light image; Determining a first similarity between the first histogram and the second histogram; Determine a first center point of each fingerprint of the four-finger fingerprint feature and a second center point of each fingerprint of the preset fingerprint feature; Mapping and overlapping the finger red light image and the preset finger red light image based on the first center point and the second center point, and calculating the red value difference of the pixel at each position corresponding to the finger red light image; Determine the variance of the red value difference of each finger, and determine the target pixel whose difference value is greater than a preset difference value; Determine the distance between any target pixel and other target pixels in each finger, and calculate the average value of the distance; determining a quality score for each finger based on the variance, the number of target pixels, and the average value; Determining whether the four-finger sequence is a real four-finger sequence corresponding to the identity information based on the quality score and the first similarity; The infrared reflection spectrum is a plurality of infrared reflection spectra of the current face during the action according to each preset instruction, each preset instruction corresponds to a set of infrared reflection spectra, and judging whether the face belongs to the real face corresponding to the identity information based on the infrared reflection spectrum includes: Calculating a second similarity between each infrared reflection spectrum in each group of infrared reflection spectrum and a preset infrared reflection spectrum; Determine a difference between two adjacent second similarities, and generate a line graph corresponding to each preset instruction based on the difference; Calculating a third similarity between each line graph and a plurality of preset line graphs in a corresponding preset database, each preset line graph is generated based on a plurality of infrared reflection spectra of a corresponding sample person's face during an action according to the preset instruction, and calculating a difference between the similarity with a preset infrared reflection spectrum of a specified area of the sample person's face, and the preset database is determined according to each preset instruction; Calculating a fourth similarity between the infrared reflection spectrum of the designated area when the current person is not taking action and a preset infrared reflection spectrum; Determine the number of third similarities in each group of infrared reflection spectra whose similarity reaches a first preset similarity threshold, and determine the proportion of the third similarities in the total number of the third similarities; If there is at least one group of infrared reflection spectra whose proportion reaches the preset proportion, and the number of the at least one group of infrared reflection spectra accounts for the preset proportion, and the fourth similarity reaches the second preset similarity threshold, it is determined that the current face belongs to the real face corresponding to the identity information.
2. A multimodal biometric feature comparison method according to claim 1, characterized in that: The determining, based on the quality score and the first similarity, whether the four-digit finger is a real four-digit finger corresponding to the identity information includes: If there is no target finger, or the number of target fingers reaches a preset number threshold, an average quality score is determined, and a first score is determined based on the average quality score, the first similarity, and the respective corresponding coefficients, and the target finger is a finger whose quality score does not reach the preset score value; If the first score reaches a preset score threshold, determining that the four-finger sequence is a real four-finger sequence corresponding to the identity information; If there are target fingers, and the number of target fingers reaches a preset number threshold, it is determined that the four-finger series is not the real four-finger series corresponding to the identity information.
3. A multimodal biometric feature comparison method according to claim 2, characterized in that: If there are target fingers, and the number of the target fingers reaches a preset number threshold, the method further includes: Acquire image information of the full picture of the current person; The motion trajectory of the current person is tracked based on the image information.
4. A multimodal biometric comparison system, characterized in that: include: The first data acquisition module is used to acquire identity information, fingerprint features of four consecutive fingers of the current person, and a face image; a control module, configured to control a red light emitting device to emit red light when the fingerprint feature is consistent with a preset fingerprint feature corresponding to the identity information, and the face image is consistent with a preset face image corresponding to the identity information, so that the red light emitting device emits red light from below the four-linked finger and emits infrared light to a designated area of the face of the current person; A second data acquisition module is used to acquire the red light image of the fingers above the four-finger linkage and the infrared reflection spectrum of the specified area of the current face; a judgment module, configured to judge whether the four-finger sequence is a real four-finger sequence corresponding to the identity information based on the finger red light image, and judge whether the face is a real face corresponding to the identity information based on the infrared reflection spectrum; When acquiring the infrared reflection spectrum of the specified area of the current face, the second data acquisition module is specifically used to: Randomly determine at least one preset instruction from a preset instruction library, and output the at least one preset instruction, so that the current person performs an action according to the at least one preset instruction; Acquire multiple infrared reflection spectra of the current face during the action according to each preset instruction; When the judging module judges whether the four-finger sequence is the real four-finger sequence corresponding to the identity information based on the finger red light image, it is specifically used to: Acquire a preset finger red light image, where the preset finger red light image is a red light image above four consecutive fingers of the person corresponding to the identity information; Determine a first histogram of red values within a four-finger range in the preset finger red light image, and a second histogram of red values within a four-finger range in the finger red light image; Determining a first similarity between the first histogram and the second histogram; Determine a first center point of each fingerprint of the four-finger fingerprint feature and a second center point of each fingerprint of the preset fingerprint feature; Mapping and overlapping the finger red light image and the preset finger red light image based on the first center point and the second center point, and calculating the red value difference of the pixel at each position corresponding to the finger red light image; Determine the variance of the red value difference of each finger, and determine the target pixel whose difference value is greater than a preset difference value; Determine the distance between any target pixel and other target pixels in each finger, and calculate the average value of the distance; determining a quality score for each finger based on the variance, the number of target pixels, and the average value; Determining whether the four-finger sequence is a real four-finger sequence corresponding to the identity information based on the quality score and the first similarity; The infrared reflection spectrum is a plurality of infrared reflection spectra of the current face during the action according to each preset instruction, each preset instruction corresponds to a set of infrared reflection spectra, and the judgment module is specifically used to judge whether the face belongs to the real face corresponding to the identity information based on the infrared reflection spectrum: Calculating a second similarity between each infrared reflection spectrum in each group of infrared reflection spectrum and a preset infrared reflection spectrum; Determine a difference between two adjacent second similarities, and generate a line graph corresponding to each preset instruction based on the difference; Calculating a third similarity between each line graph and a plurality of preset line graphs in a corresponding preset database, each preset line graph is generated based on a plurality of infrared reflection spectra of a corresponding sample person's face during an action according to the preset instruction, and calculating a difference between the similarity with a preset infrared reflection spectrum of a specified area of the sample person's face, and the preset database is determined according to each preset instruction; Calculating a fourth similarity between the infrared reflection spectrum of the designated area when the current person is not taking action and a preset infrared reflection spectrum; Determine the number of third similarities in each group of infrared reflection spectra whose similarity reaches a first preset similarity threshold, and determine the proportion of the third similarities in the total number of the third similarities; If there is at least one group of infrared reflection spectra whose proportion reaches the preset proportion, and the number of the at least one group of infrared reflection spectra accounts for the preset proportion, and the fourth similarity reaches the second preset similarity threshold, it is determined that the current face belongs to the real face corresponding to the identity information.
5. An electronic device, characterized in that: It includes: at least one processor; Memory; At least one application, wherein the at least one application is stored in the memory and is configured to be executed by the at least one processor, and the at least one application is used to execute a multimodal biometric feature comparison method according to any one of claims 1 to 3.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute a multimodal biometric feature comparison method as described in any one of claims 1 to 3.
