Biometric multimodal fusion recognition method, device, storage medium and equipment

Abstract

The present invention discloses a biometric multimodal fusion recognition method, device, storage medium and equipment, belonging to the field of biometric recognition. In multimodal recognition, the comparison threshold of the latter modality is adjusted according to the quality score of the biometric image of the latter modality; when the comparison score of the latter modality is not less than the adjusted comparison threshold of the latter modality, and the quality score of the biometric image of the latter modality is not less than the decision threshold of the latter modality, it is judged that the recognition is passed or the recognition of the subsequent modality is performed; otherwise, the comparison scores of all the recognized modalities are fused to obtain a fusion score; according to the fusion score, it is judged that the recognition is passed or the recognition of the subsequent modality is performed. The present invention dynamically adjusts the comparison threshold according to the quality score of the biometric image, and uses the quality score of the biometric image to assist in decision-making on the recognition result, and combines the comparison score fusion to gradually increase the probability of identity authentication, thereby improving the security and reliability of biometric identity recognition.

Description

Technical Field

[0001] The present invention relates to the field of biometrics, and in particular to a method, device, storage medium and equipment for multimodal fusion recognition of biometric features. Background Art

[0002] Biometric technology closely combines computers with high-tech means such as optics, acoustics, biosensors and biostatistics principles, and uses the inherent physiological characteristics of the human body such as fingerprints, faces, irises, finger veins, etc. to identify personal identity.

[0003] With the increasing requirements for accuracy and reliability of social security and identity authentication, the limitations of single biometric recognition in terms of accuracy and reliability have become increasingly prominent and can no longer meet the needs of product and technology development. Currently, multimodal biometric recognition is considered to be a research direction with the most potential and advantages. Multimodal biometric recognition can effectively utilize the independence between different modalities to further improve the recognition rate, so that the accuracy of the fused result can be higher than that of any single modality.

[0004] One method of the biometric multimodal fusion method in the prior art is as follows: wherein P1, P2, and P3 are several biometric images, R1, R2, and R3 are biometric templates corresponding to P1, P2, and P3 respectively. During the recognition process, P1 and R1 are compared to obtain a comparison score S1. If the comparison score S1 is greater than a threshold value T1, the comparison threshold corresponding to P2 is adjusted to obtain an adjusted comparison threshold value T2, for example, the score of the original comparison threshold is lowered. Conversely, if the comparison score S1 is less than the threshold value T1, the score of the original comparison threshold value T2 is increased.

[0005] After adjusting the comparison threshold of P2 according to the comparison score of P1 to obtain the comparison threshold T2, P2 is compared with R2 to obtain the comparison score S2. The original comparison threshold T3 of P3 is adjusted according to the comparison score S2 to obtain the comparison threshold T3 corresponding to P3 after adjustment. For example, if the comparison score S2 is greater than the comparison threshold T2, the original comparison threshold T3 corresponding to P3 is adjusted, for example, the score of the original comparison threshold T3 is lowered. Conversely, if the comparison score S2 is less than the threshold T2, the score of the original comparison threshold T3 is increased, and then P3 is compared with R3 to obtain the comparison score S3. If S3 is greater than the comparison threshold T3 obtained after adjustment, it is considered that the comparison passes and the identity recognition is successful. Otherwise, it is considered that the comparison fails and the identity recognition fails.

[0006] That is, the existing technology adjusts the comparison threshold of the subsequent biometric comparison according to the comparison result of the previous biometric. Although this method makes the comparison of the latter biometric easier and can improve the comparison pass rate, the method of determining the comparison threshold of the latter biometric by the comparison result of the previous biometric is uncertain, and the extent of increasing or decreasing the comparison threshold is difficult to grasp, resulting in poor security and reliability of biometric identity recognition. Summary of the invention

[0007] In order to solve the technical problem of uncertainty in the adjustment of the comparison threshold in the multimodal biometric recognition in the prior art, the present invention provides a biometric multimodal fusion recognition method, device, storage medium and equipment, which dynamically adjust the comparison threshold according to the quality score of the biometric image, and use the quality score of the biometric image to assist in the decision-making of the recognition result, and at the same time combine the comparison score fusion to gradually increase the probability of identity authentication and improve the security and reliability of biometric identity recognition.

[0008] The present invention provides the following technical solutions:

[0009] In a first aspect, the present invention provides a biometric multimodal fusion recognition method, the method comprising:

[0010] Recognize the biometric image of each modality in turn;

[0011] When the comparison score of the current modality is not less than the minimum threshold of the current modality, adjusting the comparison threshold of the next modality according to the quality score of the biometric image of the next modality;

[0012] When the comparison score of the latter modality is not less than the minimum threshold of the latter modality and the adjusted comparison threshold of the latter modality, and the quality score of the biometric image of the latter modality is not less than the decision threshold of the latter modality, it is judged that the recognition is passed or the recognition of the subsequent modality is carried out;

[0013] When the comparison score of the latter modality is not less than the minimum threshold of the latter modality, and the comparison score of the latter modality is less than the adjusted comparison threshold of the latter modality or the quality score of the biometric image of the latter modality is less than the decision threshold of the latter modality, the comparison scores of all the identified modalities are fused to obtain a fusion score;

[0014] When the fusion score is not less than the corresponding fusion score threshold, it is judged that the recognition is passed or the recognition of the subsequent modality is performed.

[0015] Furthermore, for any modality, the lowest threshold is lower than the threshold at which the modality is normally recognized.

[0016] Furthermore, for any modality, the higher the quality score of the biometric image of the modality, the higher the comparison threshold of the modality is adjusted.

[0017] Furthermore, for any modality, the comparison threshold of the modality is adjusted as follows:

[0018] The quality score of the biometric image of the modality is divided into three intervals [0, Qmin), [Qmin, Qref), [Qref, Qmax]; wherein Qmin, Qref and Qmax are the minimum quality score, reference quality score and maximum quality score of the biometric image of the modality respectively;

[0019] If the quality score of the biometric image of the modality is within [0, Qmin), the biometric image of the modality is discarded;

[0020] If the quality score of the biometric image of the modality is within [Qmin, Qref), the comparison threshold of the modality remains unchanged;

[0021] If the quality score of the biometric image of the modality is within [Qref, Qmax], the comparison threshold of the modality increases with the increase of the quality score of the biometric image of the modality; its calculation formula is:

[0022] T=Tmin+(Tmax-Tmin)*f((Q-Qref) / (QMax-Qref))

[0023] Where Q∈[Qmin,Qmax]

[0024]

[0025] Furthermore, the step of sequentially identifying the biometric image of each modality includes:

[0026] Identify the biometric image of the first modality with the biometric template of the first modality to obtain a comparison score of the first modality, and determine whether the comparison score of the first modality is not less than a minimum threshold of the first modality. If so, proceed to the next step, otherwise determine that the recognition fails;

[0027] Calculating a quality score of the biometric image of the second modality, and adjusting a comparison threshold of the second modality according to the quality score of the biometric image of the second modality, wherein the higher the quality score of the biometric image of the second modality, the higher the comparison threshold of the second modality;

[0028] Identify the biometric image of the second modality with the biometric template of the second modality to obtain a comparison score of the second modality, and determine whether the comparison score of the second modality is not less than a minimum threshold of the second modality. If so, proceed to the next step, otherwise determine that the recognition fails;

[0029] Determine whether the comparison score of the second modality is not less than the adjusted comparison threshold of the second modality and the quality score of the biometric image of the second modality is not less than the decision threshold of the second modality. If both conditions are met, the recognition is judged to be successful, otherwise, proceed to the next step.

[0030] The comparison score of the first modality and the comparison score of the second modality are combined to obtain a first fusion score, and it is determined whether the first fusion score is less than a first fusion score threshold, and if so, it is determined that the recognition fails;

[0031] If the first fusion score is not less than the first fusion score threshold and there is no unrecognized modality, it is determined that the recognition is successful; otherwise, recognition of subsequent modalities is performed.

[0032] Furthermore, the step of sequentially identifying the biometric image of each modality includes:

[0033] Identify the biometric image of the first modality with the biometric template of the first modality to obtain a comparison score of the first modality, and determine whether the comparison score of the first modality is not less than a minimum threshold of the first modality. If so, proceed to the next step, otherwise determine that the recognition fails;

[0034] Calculating a quality score of the biometric image of the second modality, and adjusting a comparison threshold of the second modality according to the quality score of the biometric image of the second modality, wherein the higher the quality score of the biometric image of the second modality, the higher the comparison threshold of the second modality;

[0035] Identify the biometric image of the second modality with the biometric template of the second modality to obtain a comparison score of the second modality, and determine whether the comparison score of the second modality is not less than a minimum threshold of the second modality. If so, proceed to the next step, otherwise determine that the recognition fails;

[0036] Determine whether the comparison score of the second modality is not less than the adjusted comparison threshold of the second modality and the quality score of the biometric image of the second modality is not less than the decision threshold of the second modality. If both conditions are met, the recognition is judged to be successful, otherwise, proceed to the next step.

[0037] The comparison score of the first modality and the comparison score of the second modality are combined to obtain a first fusion score, and it is determined whether the first fusion score is not less than a first fusion score threshold. If so, the next step is executed, otherwise, it is determined that the recognition fails;

[0038] Calculating a quality score of the biometric image of the third modality, and adjusting a comparison threshold of the third modality according to the quality score of the biometric image of the third modality, wherein the higher the quality score of the biometric image of the third modality, the higher the comparison threshold of the third modality;

[0039] Identify the biometric image of the third modality with the biometric template of the third modality to obtain a comparison score of the third modality, and determine whether the comparison score of the third modality is not less than a minimum threshold of the third modality. If so, proceed to the next step, otherwise determine that the recognition fails.

[0040] Determine whether the comparison score of the third modality is not less than the adjusted comparison threshold of the third modality and the quality score of the biometric image of the third modality is not less than the decision threshold of the third modality. If both conditions are met, the recognition is judged to be passed, otherwise, the next step is executed;

[0041] The comparison score of the first modality, the comparison score of the second modality and the comparison score of the third modality are integrated to obtain a second fusion score, and it is determined whether the second fusion score is less than a second fusion score threshold. If so, it is determined that the recognition fails.

[0042] If the second fusion score is not less than the second fusion score threshold and there is no unrecognized modality, it is determined that the recognition is successful; otherwise, recognition of subsequent modalities is performed.

[0043] In a second aspect, the present invention provides a biometric multimodal fusion recognition device, the device comprising:

[0044] A multimodal recognition module, used to recognize the biometric image of each modality in turn;

[0045] A comparison threshold adjustment module, used to adjust the comparison threshold of the next modality according to the quality score of the biometric image of the next modality when the comparison score of the current modality is not less than the minimum threshold of the current modality;

[0046] The first judgment module is used to judge that the recognition is passed or to proceed to the recognition of the subsequent modality when the comparison score of the subsequent modality is not less than the minimum threshold of the subsequent modality and the adjusted comparison threshold of the subsequent modality, and the quality score of the biometric image of the subsequent modality is not less than the decision threshold of the subsequent modality;

[0047] The second judgment module is used to fuse the comparison scores of all the identified modalities to obtain a fusion score when the comparison score of the latter modality is not less than the minimum threshold of the latter modality, and the comparison score of the latter modality is less than the adjusted comparison threshold of the latter modality or the quality score of the biometric image of the latter modality is less than the decision threshold of the latter modality;

[0048] The fusion module is used to judge whether the recognition is passed or to perform recognition of subsequent modalities when the fusion score is not less than the corresponding fusion score threshold.

[0049] Furthermore, for any modality, the lowest threshold is lower than the threshold at which the modality is normally recognized.

[0050] Furthermore, for any modality, the higher the quality score of the biometric image of the modality, the higher the comparison threshold of the modality is adjusted.

[0051] Furthermore, for any modality, the comparison threshold of the modality is adjusted as follows:

[0052] The quality score of the biometric image of the modality is divided into three intervals [0, Qmin), [Qmin, Qref), [Qref, Qmax]; wherein Qmin, Qref and Qmax are the minimum quality score, reference quality score and maximum quality score of the biometric image of the modality respectively;

[0053] If the quality score of the biometric image of the modality is within [0, Qmin), the biometric image of the modality is discarded;

[0054] If the quality score of the biometric image of the modality is within [Qmin, Qref), the comparison threshold of the modality remains unchanged;

[0055] If the quality score of the biometric image of the modality is within [Qref, Qmax], the comparison threshold of the modality increases with the increase of the quality score of the biometric image of the modality; its calculation formula is:

[0056] T=Tmin+(Tmax-Tmin)*f((Q-Qref) / (QMax-Qref))

[0057] Where Q∈[Qmin,Qmax]

[0058]

[0059] Furthermore, the device comprises:

[0060] The first recognition unit is used to recognize the biometric image of the first modality with the biometric template of the first modality, obtain the comparison score of the first modality, and determine whether the comparison score of the first modality is not less than the minimum threshold of the first modality. If so, execute the first adjustment unit, otherwise determine that the recognition fails.

[0061] The first adjustment unit is used to calculate the quality score of the biometric image of the second modality, and adjust the comparison threshold of the second modality according to the quality score of the biometric image of the second modality, wherein the higher the quality score of the biometric image of the second modality, the higher the comparison threshold of the second modality.

[0062] The second recognition unit is used to recognize the biometric image of the second modality with the biometric template of the second modality, obtain the comparison score of the second modality, and determine whether the comparison score of the second modality is not less than the minimum threshold of the second modality. If so, execute the first judgment unit, otherwise determine that the recognition fails.

[0063] A first judgment unit is used to judge whether the comparison score of the second modality is not less than the adjusted comparison threshold of the second modality and the quality score of the biometric image of the second modality is not less than the decision threshold of the second modality. If both conditions are met, it is judged that the recognition is passed, otherwise the first fusion unit is executed;

[0064] The first fusion unit is used to fuse the comparison score of the first modality and the comparison score of the second modality to obtain a first fusion score, and determine whether the first fusion score is less than a first fusion score threshold. If so, determine that the recognition fails.

[0065] The second judgment unit is configured to judge that the recognition is successful if the first fusion score is not less than a first fusion score threshold and there is no unrecognized modality, and to perform recognition of a subsequent modality otherwise.

[0066] Furthermore, the device comprises:

[0067] The third recognition unit is used to recognize the biometric image of the first modality with the biometric template of the first modality, obtain the comparison score of the first modality, and determine whether the comparison score of the first modality is not less than the minimum threshold of the first modality. If so, execute the second adjustment unit, otherwise determine that the recognition fails.

[0068] The second adjustment unit is used to calculate the quality score of the biometric image of the second modality, and adjust the comparison threshold of the second modality according to the quality score of the biometric image of the second modality, wherein the higher the quality score of the biometric image of the second modality, the higher the comparison threshold of the second modality.

[0069] a fourth recognition unit, configured to recognize the biometric image of the second modality and the biometric template of the second modality, obtain a comparison score of the second modality, and determine whether the comparison score of the second modality is not less than a minimum threshold of the second modality, and if so, execute the third judgment unit, otherwise, determine that the recognition fails;

[0070] The third judgment unit is used to judge whether the comparison score of the second modality is not less than the adjusted comparison threshold of the second modality and the quality score of the biometric image of the second modality is not less than the decision threshold of the second modality. If both conditions are met, the recognition is judged to be successful, otherwise the second fusion unit is executed.

[0071] The second fusion unit is used to fuse the comparison score of the first modality and the comparison score of the second modality to obtain a first fusion score, and determine whether the first fusion score is not less than a first fusion score threshold. If so, the third adjustment unit is executed, otherwise it is determined that the recognition fails.

[0072] The third adjustment unit calculates a quality score of the biometric image of the third modality, and adjusts a comparison threshold of the third modality according to the quality score of the biometric image of the third modality, wherein the higher the quality score of the biometric image of the third modality, the higher the comparison threshold of the third modality.

[0073] The fifth recognition unit is used to recognize the biometric image of the third modality with the biometric template of the third modality, obtain the comparison score of the third modality, and determine whether the comparison score of the third modality is not less than the minimum threshold of the third modality. If so, execute the fourth judgment unit, otherwise determine that the recognition fails.

[0074] The fourth judgment unit is used to judge whether the comparison score of the third modality is not less than the adjusted comparison threshold of the third modality and the quality score of the biometric image of the third modality is not less than the decision threshold of the third modality. If both conditions are met, the recognition is judged to be passed, otherwise the third fusion unit is executed.

[0075] a third fusion unit, configured to fuse the comparison score of the first modality, the comparison score of the second modality, and the comparison score of the third modality to obtain a second fusion score, and determine whether the second fusion score is less than a second fusion score threshold, and if so, determine that the recognition fails;

[0076] The fifth judgment unit is configured to judge that the recognition is successful if the second fusion score is not less than a second fusion score threshold and there is no unrecognized modality, and to perform recognition of a subsequent modality otherwise.

[0077] In a third aspect, the present invention provides a computer-readable storage medium for biometric multimodal fusion recognition, comprising a memory for storing processor-executable instructions, which, when executed by the processor, implements the steps of the biometric multimodal fusion recognition method described in the first aspect.

[0078] In a fourth aspect, the present invention provides a device for biometric multimodal fusion recognition, comprising at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the biometric multimodal fusion recognition method described in the first aspect are implemented.

[0079] The present invention has the following beneficial effects:

[0080] The present invention firstly dynamically adjusts the comparison threshold according to the quality score of the biometric image, and also uses the quality score of the biometric image to assist in decision-making on the recognition result. At the same time, combined with the comparison score fusion, the probability of identity authentication is gradually increased, thereby improving the security and reliability of biometric identity recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is a flow chart of the biometric multimodal fusion recognition method of the present invention;

[0082] Figure 2 for Figure 1 A flowchart of a specific example of the biometric multimodal fusion recognition method;

[0083] Figure 3 for Figure 1 A flowchart of another specific example of the biometric multimodal fusion recognition method;

[0084] Figure 4 It is a schematic diagram of the biometric multi-modal fusion recognition device of the present invention;

[0085] Figure 5 for Figure 4 A schematic diagram of a specific example of the biometric multimodal fusion recognition device;

[0086] Figure 6 for Figure 4 A schematic diagram of another specific example of the biometric multimodal fusion recognition device. DETAILED DESCRIPTION

[0087] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0088] Embodiment 1:

[0089] The embodiment of the present invention provides a biometric multi-modal fusion recognition method, such as Figure 1 As shown, the method includes:

[0090] S100: Identify the biometric image of each modality in turn.

[0091] This step is used to obtain biometric images of multiple modalities and recognize the biometric images of each modality in chronological order, that is, multimodal recognition.

[0092] S110: When the comparison score of the current modality is not less than the minimum threshold of the current modality, adjusting the comparison threshold of the next modality according to the quality score of the biometric image of the next modality.

[0093] The minimum threshold of a certain modality is a score lower than the threshold score set when the modality is normally recognized. For example, under normal circumstances, a score greater than or equal to 70 points is considered to be recognized. In this embodiment, the minimum threshold is set to be less than 70 points. For example, the minimum threshold is set to 60 points. If the score is lower than the minimum threshold, it is directly judged as recognition failure. This can prevent real users from misjudgment due to factors such as environment, image effect, and shaking.

[0094] The present invention calculates the quality score of the biometric image of the next modality when the comparison score of the current modality is not less than the minimum threshold of the modality, and adjusts the comparison threshold of the next modality according to the quality score of the biometric image of the next modality.

[0095] For any modality, the higher the quality score of the biometric image of the modality, the higher the comparison threshold of the modality is adjusted.

[0096] The present invention adaptively adjusts the comparison threshold according to the quality score of the biometric image of the subsequent modality, and adjusts the amplitude of the comparison threshold accordingly according to the quality score, thereby gradually increasing the probability of identity authentication in the process of multimodal biometric recognition and improving the security and reliability of biometric identity recognition.

[0097] S120: When the comparison score of the subsequent modality is not less than the minimum threshold of the subsequent modality and the adjusted comparison threshold of the subsequent modality, and the quality score of the biometric image of the subsequent modality is not less than the decision threshold of the subsequent modality, it is determined that the recognition is passed or the recognition of the subsequent modality is performed.

[0098] S130: When the comparison score of the latter modality is not less than the minimum threshold of the latter modality, and the comparison score of the latter modality is less than the adjusted comparison threshold of the latter modality or the quality score of the biometric image of the latter modality is less than the decision threshold of the latter modality, the comparison scores of all the identified modalities are fused to obtain a fusion score.

[0099] S140: When the fusion score is not less than the corresponding fusion score threshold, determine that the recognition is passed or proceed to recognition of a subsequent modality.

[0100] In this step, if the fusion score is not less than the corresponding fusion score threshold and there is no unrecognized modality, the recognition is judged to be successful. If there is still an unrecognized modality, the recognition of the subsequent modality is performed.

[0101] The present invention firstly dynamically adjusts the comparison threshold according to the quality score of the biometric image, and also uses the quality score of the biometric image to assist in decision-making on the recognition result. At the same time, combined with the comparison score fusion, the probability of identity authentication is gradually increased, thereby improving the security and reliability of biometric identity recognition.

[0102] In the present invention, for any modality, the comparison threshold of the modality is adjusted in the following manner:

[0103] The quality score of the biometric image of this modality is divided into three intervals [0, Qmin), [Qmin, Qref), [Qref, Qmax]; wherein Qmin, Qref and Qmax are respectively the minimum quality score, reference quality score and maximum quality score of the biometric image of this modality.

[0104] The minimum quality score Qmin corresponds to the lowest threshold value Tmin, and the maximum quality score Qmax corresponds to the highest threshold value Tmax.

[0105] If the quality score of the biometric image of the modality is within [0, Qmin), it is considered that the quality of P2 / P3 is not up to standard, and the biometric image of the modality can be discarded.

[0106] If the quality score of the biometric image of the modality is within [Qmin, Qref), it is considered that the quality of the biometric image is in an uncertain state, and the comparison threshold of the modality remains unchanged, that is, the comparison threshold of the modality is not adjusted.

[0107] If the quality score Q of the biometric image of the modality is within [Qref, Qmax], the comparison threshold T of the modality increases with the increase of the quality score Q of the biometric image of the modality.

[0108] When the quality score is between [Qref, Qmax], the quality of the biometric image is considered to meet the requirements. As the size of the biometric image increases, the comparison threshold also increases;

[0109] The calculation formula is:

[0110] T=Tmin+(Tmax-Tmin)*f((Q-Qref) / (QMax-Qref))

[0111] Where Q∈[Qmin,Qmax]

[0112]

[0113] The present invention does not limit the types of biometric images of multiple modalities, which may include face images, fingerprint images, iris images and / or finger vein images, etc.

[0114] Based on a specific example, Figure 2 As shown, the aforementioned method includes:

[0115] S10: Identify the biometric image P1 of the first modality with the biometric template R1 of the first modality to obtain a comparison score S1 of the first modality, and determine whether the comparison score S1 of the first modality is not less than the minimum threshold T1min of the first modality. If so, execute S20, otherwise determine that the recognition fails.

[0116] P1 is a biometric image representing the identity of a human individual, which may include one of the face, fingerprint, iris, and finger vein. These biometric images can be collected by a dedicated image acquisition device such as a high-definition camera or a finger vein acquisition head. R1 is a registration template of the user's biometric pre-stored in the template library.

[0117] In this step, the acquired biometric image P1 of the first modality is compared for similarity with the biometric registration template R1 of the user pre-stored in the template library to obtain a comparison score S1 of the first modality.

[0118] The minimum threshold value T1min of the first mode is lower than the threshold score set when P1 is normally recognized. For example, under normal circumstances, if S1 is greater than or equal to 70 points, it is considered that the recognition is successful. In this embodiment, T1min is less than 70 points. For example, T1min is set to 60 points. This can prevent misjudgment of real users due to factors such as environment, image effect, and shaking.

[0119] When the comparison score S1 of the user's first modality is less than 60 points, it is determined that the biometric image matching of the current user fails and the recognition fails, otherwise, step S20 is continued.

[0120] S20: Calculate the quality score Q2 of the biometric image P2 of the second modality, and adjust the comparison threshold T2 of the second modality according to the quality score Q2 of the biometric image of the second modality, wherein the higher the quality score Q2 of the biometric image of the second modality, the higher the comparison threshold T2 of the second modality, and conversely, if Q2 is lower, T2 is lower.

[0121] In this step, when S1 is greater than or equal to T1min, it is necessary to further analyze the quality score Q2 of P2 through the image quality algorithm, and adjust the comparison threshold T2 accordingly according to the quality score Q2.

[0122] S30: Identify the biometric image P2 of the second modality with the biometric template R2 of the second modality to obtain a comparison score S2 of the second modality, and determine whether the comparison score S2 of the second modality is not less than the minimum threshold T2min of the second modality. If so, execute S40, otherwise determine that the recognition fails.

[0123] In this step, after the comparison of the biometric image P1 of the first modality is completed and the comparison threshold T2 of the biometric is adjusted according to the quality score Q2 of the biometric image P2 of the second modality, the biometric image P2 of the second modality is compared with the biometric registration template R2 of the user pre-stored in the template library for similarity to obtain a comparison score S2.

[0124] The minimum threshold value T2min of the second mode is lower than the threshold score set when P2 is normally recognized. For example, under normal circumstances, if S2 is greater than or equal to 80 points, it is considered that the recognition is passed. In this embodiment, T2min is less than 80 points, for example, T2min is 70 points. This setting can prevent real users from being misjudged due to factors such as environment, image effect, and shaking.

[0125] When the comparison score S2 of the second modality of the user is less than 70 points, it is determined that the biometric image matching of the current user fails, otherwise, step S40 is continued.

[0126] It should be noted that T1min and T2min can be the same or different. For example, when P1 is a face image and P2 is an iris image, T1min can be set to 60 points. Since the iris comparison accuracy is higher, T2min can be greater than T1min, for example, T2min is set to 70 points.

[0127] S40: Determine whether the comparison score S2 of the second modality is not less than the adjusted comparison threshold T2 of the second modality and the quality score Q2 of the biometric image of the second modality is not less than the decision threshold Q of the second modality. If both conditions are met, determine that the recognition is passed, otherwise execute S50.

[0128] In this step, when the comparison score S2 of the second modality is greater than or equal to the minimum threshold T2min of the second modality, it is necessary to further determine whether the comparison score S2 of the second modality is greater than or equal to the adjusted second comparison threshold T2 to prevent misidentification. If it is greater than or equal to, it is further determined whether the quality score Q2 of P2 is greater than or equal to the decision threshold of the second modality, which is the set quality score threshold. If it is greater than or equal to, the recognition is determined to be passed.

[0129] The purpose of judging whether the quality score Q2 of P2 is greater than or equal to Q is to judge the reliability of the second comparison threshold T2 adjusted according to Q2. If it is greater than or equal to Q, the adjusted T2 is reliable and the recognition is considered to be successful.

[0130] S50: Fusing the comparison score S1 of the first modality and the comparison score S2 of the second modality to obtain a first fusion score S12, and determining whether the first fusion score is less than a first fusion score threshold T12. If so, determining that the recognition fails.

[0131] In this step, the comparison score S1 corresponding to P1 and the comparison score S2 corresponding to P2 are fused to obtain a first fusion score threshold T12.

[0132] During fusion, S1 and S2 can be used to construct a feature vector, which is then input into a classifier to obtain a fusion score. The classifier can be a decision tree classifier, a neural network, a support vector machine, a K nearest neighbor classifier, a random forest classifier, and the like.

[0133] S60: If the first fusion score S12 is not less than the first fusion score threshold T12, and there is no unrecognized modality, it is determined that the recognition is successful, otherwise the recognition of the subsequent modality is performed.

[0134] Based on another specific example, Figure 3 As shown, the aforementioned method includes:

[0135] S10': Identify the biometric image P1 of the first modality with the biometric template R1 of the first modality to obtain a comparison score S1 of the first modality, and determine whether the comparison score S1 of the first modality is not less than the minimum threshold T1min of the first modality. If so, execute S20', otherwise determine that the recognition fails.

[0136] S20': Calculate the quality score Q2 of the biometric image P2 of the second modality, and adjust the comparison threshold T2 of the second modality according to the quality score Q2 of the biometric image of the second modality, wherein the higher the quality score Q2 of the biometric image of the second modality, the higher the comparison threshold T2 of the second modality.

[0137] S30': Identify the biometric image P2 of the second modality with the biometric template R2 of the second modality to obtain a comparison score S2 of the second modality, and determine whether the comparison score S2 of the second modality is not less than the minimum threshold T2min of the second modality. If so, execute S40', otherwise determine that the recognition fails.

[0138] S40': Determine whether the comparison score S2 of the second modality is not less than the adjusted comparison threshold T2 of the second modality and the quality score Q2 of the biometric image of the second modality is not less than the decision threshold QT2 of the second modality. If both conditions are met, determine that the recognition is passed, otherwise execute S50'.

[0139] S50': Fusing the comparison score S1 of the first modality and the comparison score S2 of the second modality to obtain a first fusion score S12, and determining whether the first fusion score is not less than a first fusion score threshold T12, if so, executing S60', otherwise determining that the recognition fails.

[0140] S60': Calculate the quality score Q3 of the biometric image P3 of the third modality, and adjust the comparison threshold T3 of the third modality according to the quality score Q3 of the biometric image of the third modality, wherein the higher the quality score Q3 of the biometric image of the third modality, the higher the comparison threshold T3 of the third modality.

[0141] S70': Identify the biometric image P3 of the third modality with the biometric template R3 of the third modality to obtain the comparison score S3 of the third modality, and determine whether the comparison score S3 of the third modality is not less than the minimum threshold T3min of the third modality. If so, execute S80', otherwise determine that the recognition fails.

[0142] S80': Determine whether the comparison score S3 of the third modality is not less than the adjusted comparison threshold T3 of the third modality and the quality score Q3 of the third modality biometric image is not less than the decision threshold QT3 of the third modality. If both conditions are met, determine that the recognition is passed, otherwise execute S90'.

[0143] S90': Fuse the comparison score S1 of the first modality, the comparison score S2 of the second modality and the comparison score S3 of the third modality to obtain a second fusion score S123, and determine whether the second fusion score S123 is less than a second fusion score threshold T123. If so, determine that the recognition fails.

[0144] S100': If the second fusion score S123 is not less than the second fusion score threshold T123, and there is no unrecognized modality, it is determined that the recognition is successful, otherwise the recognition of the subsequent modality is performed.

[0145] Embodiment 2:

[0146] The embodiment of the present invention provides a biometric multi-modal fusion recognition device, the device comprising:

[0147] The multimodal recognition module 100 is used to recognize the biometric image of each modality in turn.

[0148] The comparison threshold adjustment module 110 is used to adjust the comparison threshold of the next modality according to the quality score of the biometric image of the next modality when the comparison score of the current modality is not less than the minimum threshold of the current modality.

[0149] The first judgment module 120 is used to judge whether the recognition is passed or to proceed to the recognition of the subsequent modality when the comparison score of the subsequent modality is not less than the minimum threshold of the subsequent modality and the adjusted comparison threshold of the subsequent modality, and the quality score of the biometric image of the subsequent modality is not less than the decision threshold of the subsequent modality.

[0150] The second judgment module 130 is used to fuse the comparison scores of all the identified modalities to obtain a fusion score when the comparison score of the latter modality is not less than the minimum threshold of the latter modality, and the comparison score of the latter modality is less than the adjusted comparison threshold of the latter modality or the quality score of the biometric image of the latter modality is less than the decision threshold of the latter modality.

[0151] The fusion module 140 is used to determine whether the recognition is passed or to perform recognition of a subsequent modality when the fusion score is not less than the corresponding fusion score threshold.

[0152] The present invention firstly dynamically adjusts the comparison threshold according to the quality score of the biometric image, and also uses the quality score of the biometric image to assist in decision-making on the recognition result. At the same time, combined with the comparison score fusion, the probability of identity authentication is gradually increased, thereby improving the security and reliability of biometric identity recognition.

[0153] For any modality, the lowest threshold is lower than the threshold when the modality is normally recognized.

[0154] Moreover, for any modality, the higher the quality score of the biometric image of the modality, the higher the comparison threshold of the modality is adjusted.

[0155] For any modality, the comparison threshold of the modality is adjusted as follows:

[0156] The quality score of the biometric image of this modality is divided into three intervals [0, Qmin), [Qmin, Qref), [Qref, Qmax]; wherein Qmin, Qref and Qmax are respectively the minimum quality score, reference quality score and maximum quality score of the biometric image of this modality.

[0157] If the quality score of the biometric image of the modality is within [0, Qmin), the biometric image of the modality is discarded.

[0158] If the quality score of the biometric image of the modality is within [Qmin, Qref), the comparison threshold of the modality remains unchanged.

[0159] If the quality score of the biometric image of the modality is within [Qref, Qmax], the comparison threshold of the modality increases as the quality score of the biometric image of the modality increases.

[0160] Based on a specific example, Figure 5 As shown, the aforementioned device includes:

[0161] The first recognition unit 10 is used to recognize the biometric image of the first modality and the biometric template of the first modality to obtain a comparison score of the first modality, and determine whether the comparison score of the first modality is not less than a minimum threshold of the first modality. If so, the first adjustment unit 20 is executed, otherwise it is determined that the recognition fails.

[0162] The first adjustment unit 20 is used to calculate the quality score of the biometric image of the second modality, and adjust the comparison threshold of the second modality according to the quality score of the biometric image of the second modality, wherein the higher the quality score of the biometric image of the second modality, the higher the comparison threshold of the second modality.

[0163] The second recognition unit 30 is used to recognize the biometric image of the second modality and the biometric template of the second modality to obtain a comparison score of the second modality, and determine whether the comparison score of the second modality is not less than a minimum threshold of the second modality. If so, the first judgment unit 40 is executed, otherwise it is determined that the recognition fails.

[0164] The first judgment unit 40 is used to judge whether the comparison score of the second modality is not less than the adjusted comparison threshold of the second modality and the quality score of the biometric image of the second modality is not less than the decision threshold of the second modality. If both conditions are met, it is judged that the recognition is passed, otherwise the first fusion unit 50 is executed;

[0165] The first fusion unit 50 is used to fuse the comparison score of the first modality and the comparison score of the second modality to obtain a first fusion score, and determine whether the first fusion score is less than a first fusion score threshold. If so, determine that the recognition fails.

[0166] The second judgment unit 60 is configured to judge that the recognition is successful if the first fusion score is not less than a first fusion score threshold and there is no unrecognized modality, and to perform recognition of subsequent modalities otherwise.

[0167] Based on another specific example, Figure 6 As shown, the aforementioned device includes:

[0168] The third recognition unit 10' is used to recognize the biometric image of the first modality with the biometric template of the first modality, obtain the comparison score of the first modality, and determine whether the comparison score of the first modality is not less than the minimum threshold of the first modality. If so, execute the second adjustment unit 20', otherwise determine that the recognition fails.

[0169] The second adjustment unit 20' is used to calculate the quality score of the biometric image of the second modality, and adjust the comparison threshold of the second modality according to the quality score of the biometric image of the second modality, wherein the higher the quality score of the biometric image of the second modality, the higher the comparison threshold of the second modality.

[0170] The fourth recognition unit 30' is used to recognize the biometric image of the second modality and the biometric template of the second modality to obtain a comparison score of the second modality, and determine whether the comparison score of the second modality is not less than the minimum threshold of the second modality. If so, the third judgment unit 40' is executed, otherwise it is determined that the recognition fails;

[0171] The third judgment unit 40' is used to judge whether the comparison score of the second modality is not less than the adjusted comparison threshold of the second modality and the quality score of the biometric image of the second modality is not less than the decision threshold of the second modality. If both conditions are met, it is judged that the recognition is passed, otherwise the second fusion unit 50' is executed.

[0172] The second fusion unit 50' is used to fuse the comparison score of the first modality and the comparison score of the second modality to obtain a first fusion score, and determine whether the first fusion score is not less than a first fusion score threshold. If so, the third adjustment unit 60' is executed, otherwise it is determined that the recognition fails.

[0173] The third adjustment unit 60' calculates the quality score of the biometric image of the third modality, and adjusts the comparison threshold of the third modality according to the quality score of the biometric image of the third modality, wherein the higher the quality score of the biometric image of the third modality, the higher the comparison threshold of the third modality.

[0174] The fifth recognition unit 70' is used to recognize the biometric image of the third modality and the biometric template of the third modality to obtain a comparison score of the third modality, and determine whether the comparison score of the third modality is not less than the minimum threshold of the third modality. If so, the fourth judgment unit 80' is executed, otherwise it is determined that the recognition fails.

[0175] The fourth judgment unit 80' is used to judge whether the comparison score of the third modality is not less than the adjusted comparison threshold of the third modality and the quality score of the biometric image of the third modality is not less than the decision threshold of the third modality. If both conditions are met, the recognition is judged to be passed, otherwise the third fusion unit 90' is executed.

[0176] The third fusion unit 90' is used to fuse the comparison score of the first modality, the comparison score of the second modality and the comparison score of the third modality to obtain a second fusion score, and determine whether the second fusion score is less than a second fusion score threshold, and if so, determine that the recognition fails;

[0177] The fifth judging unit 100' is configured to judge that the recognition is successful if the second fusion score is not less than a second fusion score threshold and there is no unrecognized modality, and to perform recognition of subsequent modalities otherwise.

[0178] The present invention is not limited to the types of biometric images of multiple modalities, which may include face images, fingerprint images, iris images and / or finger vein images.

[0179] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned method embodiment 1. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment 1. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment 1, and will not be repeated here.

[0180] Embodiment 3:

[0181] The method described in the above-mentioned embodiment 1 provided in this specification can realize the business logic through a computer program and record it on a storage medium, and the storage medium can be read and executed by a computer to realize the effect of the solution described in embodiment 1 of this specification. Therefore, the present invention also provides a computer-readable storage medium for biometric multimodal fusion recognition, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the biometric multimodal fusion recognition method including embodiment 1 are realized.

[0182] The present invention firstly dynamically adjusts the comparison threshold according to the quality score of the biometric image, and also uses the quality score of the biometric image to assist in decision-making on the recognition result. At the same time, combined with the comparison score fusion, the probability of identity authentication is gradually increased, thereby improving the security and reliability of biometric identity recognition.

[0183] The storage medium may include a physical device for storing information, which is usually a medium that digitizes the information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.

[0184] The above-mentioned device may also include other implementations according to the description of method embodiment 1. The specific implementation methods may refer to the description of the relevant method embodiment 1, and will not be described one by one here.

[0185] Embodiment 4:

[0186] The present invention also provides a device for biometric multimodal fusion recognition, which may be a separate computer, or may include an actual operating device that uses one or more of the methods or one or more of the embodiments of this specification. The device for biometric multimodal fusion recognition may include at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, the steps of the biometric multimodal fusion recognition method described in any one or more of Embodiment 1 are implemented.

[0187] The present invention firstly dynamically adjusts the comparison threshold according to the quality score of the biometric image, and also uses the quality score of the biometric image to assist in decision-making on the recognition result. At the same time, combined with the comparison score fusion, the probability of identity authentication is gradually increased, thereby improving the security and reliability of biometric identity recognition.

[0188] The above-mentioned device may also include other implementation modes according to the description of the method or device embodiment. The specific implementation mode may refer to the description of the relevant method embodiment 1, and will not be described one by one here.

[0189] It should be noted that the device or system described above in this specification may also include other implementation methods according to the description of the relevant method embodiments. The specific implementation methods can refer to the description of the method embodiments, which will not be described one by one here. The various embodiments in this specification are described in a progressive manner, and the same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program type and storage medium + program embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0190] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0191] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0192] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0193] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0194] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0195] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0196] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0197] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0198] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method or device including the element.

[0199] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0200] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0201] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0202] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A biometric multimodal fusion recognition method, characterized in that: The method comprises: The biometric image of each modality is recognized in turn; When the comparison score of the current modality is not less than the minimum threshold of the current modality, adjusting the comparison threshold of the subsequent modality according to the quality score of the biometric image of the subsequent modality; When the comparison score of the latter modality is not less than the minimum threshold of the latter modality and the adjusted comparison threshold of the latter modality, and the quality score of the biometric image of the latter modality is not less than the decision threshold of the latter modality, it is judged that the recognition is passed or the recognition of the subsequent modality is carried out; When the comparison score of the latter modality is not less than the minimum threshold of the latter modality, and the comparison score of the latter modality is less than the adjusted comparison threshold of the latter modality or the quality score of the biometric image of the latter modality is less than the decision threshold of the latter modality, the comparison scores of all the identified modalities are fused to obtain a fusion score; When the fusion score is not less than the corresponding fusion score threshold, it is judged that the recognition is passed or the recognition of the subsequent modality is performed; For any modality, the comparison threshold of the modality is adjusted as follows: The quality score of the biometric image of the modality is divided into three intervals [0, Qmin), [Qmin, Qref), [Qref, Qmax]; wherein Qmin, Qref and Qmax are the minimum quality score, reference quality score and maximum quality score of the biometric image of the modality respectively; If the quality score of the biometric image of the modality is within [0, Qmin), the biometric image of the modality is discarded; If the quality score of the biometric image of the modality is within [Qmin, Qref), the comparison threshold of the modality remains unchanged; If the quality score of the biometric image of the modality is within [Qref, Qmax], the comparison threshold of the modality increases with the increase of the quality score of the biometric image of the modality; its calculation formula is: T=Tmin+(Tmax-Tmin)*f((Q-Qref) / (QMax-Qref)) Where Q∈[Qmin,Qmax] 2. The biometric multimodal fusion recognition method according to claim 1, characterized in that: For any modality, the lowest threshold is lower than the threshold when the modality is normally recognized.

3. The biometric multimodal fusion recognition method according to claim 1, characterized in that: For any modality, the higher the quality score of the biometric image of the modality, the higher the comparison threshold of the modality is adjusted.

4. The biometric multimodal fusion recognition method according to any one of claims 1 to 3, characterized in that: The step of sequentially identifying the biometric feature image of each modality includes: Identify the biometric image of the first modality with the biometric template of the first modality to obtain a comparison score of the first modality, and determine whether the comparison score of the first modality is not less than a minimum threshold of the first modality. If so, proceed to the next step, otherwise determine that the recognition fails; Calculating a quality score of the biometric image of the second modality, and adjusting a comparison threshold of the second modality according to the quality score of the biometric image of the second modality, wherein the higher the quality score of the biometric image of the second modality, the higher the comparison threshold of the second modality; Identify the biometric image of the second modality with the biometric template of the second modality to obtain a comparison score of the second modality, and determine whether the comparison score of the second modality is not less than a minimum threshold of the second modality. If so, proceed to the next step, otherwise determine that the recognition fails; Determine whether the comparison score of the second modality is not less than the adjusted comparison threshold of the second modality and the quality score of the biometric image of the second modality is not less than the decision threshold of the second modality. If both conditions are met, the recognition is judged to be successful, otherwise, proceed to the next step. The comparison score of the first modality and the comparison score of the second modality are combined to obtain a first fusion score, and it is determined whether the first fusion score is less than a first fusion score threshold, and if so, it is determined that the recognition fails; If the first fusion score is not less than the first fusion score threshold and there is no unrecognized modality, it is determined that the recognition is successful; otherwise, recognition of subsequent modalities is performed.

5. The biometric multimodal fusion recognition method according to any one of claims 1 to 3, characterized in that: The step of sequentially identifying the biometric feature image of each modality includes: Identify the biometric image of the first modality with the biometric template of the first modality to obtain a comparison score of the first modality, and determine whether the comparison score of the first modality is not less than a minimum threshold of the first modality. If so, proceed to the next step, otherwise determine that the recognition fails; Calculating a quality score of the biometric image of the second modality, and adjusting a comparison threshold of the second modality according to the quality score of the biometric image of the second modality, wherein the higher the quality score of the biometric image of the second modality, the higher the comparison threshold of the second modality; Identify the biometric image of the second modality with the biometric template of the second modality to obtain a comparison score of the second modality, and determine whether the comparison score of the second modality is not less than a minimum threshold of the second modality. If so, proceed to the next step, otherwise determine that the recognition fails; Determine whether the comparison score of the second modality is not less than the adjusted comparison threshold of the second modality and the quality score of the biometric image of the second modality is not less than the decision threshold of the second modality. If both conditions are met, the recognition is judged to be successful, otherwise, proceed to the next step. The comparison score of the first modality and the comparison score of the second modality are combined to obtain a first fusion score, and it is determined whether the first fusion score is not less than a first fusion score threshold. If so, the next step is executed, otherwise, it is determined that the recognition fails; Calculating a quality score of the biometric image of the third modality, and adjusting a comparison threshold of the third modality according to the quality score of the biometric image of the third modality, wherein the higher the quality score of the biometric image of the third modality, the higher the comparison threshold of the third modality; Identify the biometric image of the third modality with the biometric template of the third modality to obtain a comparison score of the third modality, and determine whether the comparison score of the third modality is not less than a minimum threshold of the third modality. If so, proceed to the next step, otherwise determine that the recognition fails. Determine whether the comparison score of the third modality is not less than the adjusted comparison threshold of the third modality and the quality score of the biometric image of the third modality is not less than the decision threshold of the third modality. If both conditions are met, the recognition is judged to be passed, otherwise, the next step is executed; The comparison score of the first modality, the comparison score of the second modality and the comparison score of the third modality are integrated to obtain a second fusion score, and it is determined whether the second fusion score is less than a second fusion score threshold. If so, it is determined that the recognition fails. If the second fusion score is not less than the second fusion score threshold and there is no unrecognized modality, it is determined that the recognition is successful; otherwise, recognition of subsequent modalities is performed.

6. A biometric multimodal fusion recognition device, characterized in that: The device comprises: A multimodal recognition module, used to recognize the biometric image of each modality in turn; A comparison threshold adjustment module, used to adjust the comparison threshold of the next modality according to the quality score of the biometric image of the next modality when the comparison score of the current modality is not less than the minimum threshold of the current modality; The first judgment module is used to judge that the recognition is passed or to proceed to the recognition of the subsequent modality when the comparison score of the subsequent modality is not less than the minimum threshold of the subsequent modality and the adjusted comparison threshold of the subsequent modality, and the quality score of the biometric image of the subsequent modality is not less than the decision threshold of the subsequent modality; The second judgment module is used to fuse the comparison scores of all the identified modalities to obtain a fusion score when the comparison score of the latter modality is not less than the minimum threshold of the latter modality, and the comparison score of the latter modality is less than the adjusted comparison threshold of the latter modality or the quality score of the biometric image of the latter modality is less than the decision threshold of the latter modality; A fusion module, used to judge whether the recognition is passed or to perform recognition of subsequent modalities when the fusion score is not less than the corresponding fusion score threshold; For any modality, the comparison threshold of the modality is adjusted as follows: The quality score of the biometric image of the modality is divided into three intervals [0, Qmin), [Qmin, Qref), [Qref, Qmax]; wherein Qmin, Qref and Qmax are the minimum quality score, reference quality score and maximum quality score of the biometric image of the modality respectively; If the quality score of the biometric image of the modality is within [0, Qmin), the biometric image of the modality is discarded; If the quality score of the biometric image of the modality is within [Qmin, Qref), the comparison threshold of the modality remains unchanged; If the quality score of the biometric image of the modality is within [Qref, Qmax], the comparison threshold of the modality increases with the increase of the quality score of the biometric image of the modality; its calculation formula is: T=Tmin+(Tmax-Tmin)*f((Q-Qref) / (QMax-Qref)) Where Q∈[Qmin,Qmax] 7. The biometric multimodal fusion recognition device according to claim 6, characterized in that: For any modality, the higher the quality score of the biometric image of the modality, the higher the comparison threshold of the modality is adjusted.

8. A computer-readable storage medium for biometric multimodal fusion recognition, characterized in that: It comprises a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the biometric multimodal fusion recognition method according to any one of claims 1 to 5 are implemented.

9. A device for multimodal fusion recognition of biometric features, characterized in that: The method comprises at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the biometric multimodal fusion recognition method described in any one of claims 1 to 5 are implemented.

Images