Identity verification method and system based on multi-modal data fusion

Abstract

The invention relates to the technical field of identity verification, in particular to an identity verification method and system based on multi-modal data fusion, and the method comprises the following steps: based on user login information, comparing the login information with pre-stored information, recognizing a user identity, and extracting pre-stored face data and iris data of a user to obtain a user identity information extraction result. According to the method, the accuracy of identity verification is improved by combining face and iris recognition technologies, optimizing image quality and comparing pre-stored data, the possibility of wrong recognition and identity fraud can be remarkably reduced, and flexible response can be performed according to different security requirements by dynamically adjusting a matching degree verification threshold value; according to the method, the adaptability and the protection degree of the verification process are enhanced, the false identification rate and the false rejection rate can be effectively reduced, meanwhile, a stricter security verification process is provided, higher-level personal identity protection is realized, identity embezzlement and other related fraudulent behaviors are effectively prevented, and the trust of consumers to digital services is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of identity verification technology, and in particular to an identity verification method and system based on multimodal data fusion. Background Art

[0002] The field of identity verification technology involves methods used to confirm the identity of an individual. This field includes biometrics, cryptographic applications, digital certificates, and smart card technology. Biometrics use physiological or behavioral characteristics to confirm identity, such as fingerprints, irises, facial recognition, or voice recognition. Cryptographic applications involve the use of encryption algorithms to protect the security of authentication data and ensure the privacy and integrity of information transmission. Digital certificates provide users with digital signatures and public key infrastructure to verify digital identities. Smart card technology combines embedded integrated circuits to perform identity verification, data storage, and application execution.

[0003] Among them, the identity verification method is an important part of identity verification technology, and its main purpose is to confirm whether the identity information of an individual is accurate. This method is usually used in security-sensitive environments such as banks, government agencies, online services, etc. to prevent identity theft and other related fraud. By using identity verification technology, the identity verification method can effectively reduce security risks and protect the interests of individuals and institutions. This method not only helps to comply with various compliance requirements, but also enhances consumers' trust in e-commerce and digital services.

[0004] Traditional methods rely on a single biometric technology or simple cryptographic applications, which limits security and applicability. For example, single biometric technology is prone to errors in poor lighting or aging equipment, and passwords and smart cards can be stolen or copied. In addition, traditional methods of multi-factor authentication statically combine passwords and biometrics and lack the ability to dynamically adapt to different security levels. Limitations increase risks in scenarios with high security requirements, such as identity verification by banks and government agencies. The lack of the ability to flexibly adjust the strength of authentication can lead to over-authentication or under-authentication, affecting user experience or security assurance, and affecting user experience. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings existing in the prior art and to propose an identity verification method and system based on multimodal data fusion.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: an identity verification method based on multimodal data fusion, comprising the following steps: S1: Based on the user login information, the login information is compared with the pre-stored information to identify the user identity, and the user's pre-stored facial data and iris data are extracted to obtain the user identity information extraction result; S2: Based on the user identity information extraction result, capture the user's facial image, optimize the brightness and contrast of the facial image, optimize the image quality, extract the user's facial features, compare the user's facial features with pre-stored user facial data, evaluate the facial matching degree, and obtain a facial matching evaluation result; S3: Based on the user identity information extraction result, capture the user's current eye image, identify the iris edge in the iris image through image analysis, compare the iris feature with the pre-stored user iris data, evaluate the iris matching degree, and obtain the iris matching evaluation result; S4: Based on the iris matching evaluation result and the facial matching evaluation result, the matching degree of the user identity is evaluated, and in combination with the security level required by the device, the matching degree verification threshold is adjusted to determine whether the identity verification is passed, and the user identity verification result is obtained.

[0007] The present invention is improved in that the method for optimizing image quality is: S211: Based on the image capture device, capture the user's facial image, and extract the current brightness and contrast data of the facial image to obtain basic image data; S212: Based on the basic image data, by formula: and Calculate the adjusted screen brightness And the adjusted image contrast , optimize the brightness and contrast of the facial image to obtain an optimized facial image, where is the average brightness value of the current image, is the weight coefficient for brightness adjustment, is the target brightness value, is the adjusted brightness value, is the contrast value of the current image, is the weight coefficient for contrast adjustment, is the target contrast value, is the adjusted contrast value.

[0008] The present invention is improved in that the steps of obtaining the facial matching evaluation result are: S221: extracting the user's current facial features based on the optimized facial image, including feature point positions, facial expression parameters and skin color information; S222: Based on the current facial features of the user and the user identity information extraction result, the user facial features are compared with the pre-stored user facial data, and the formula is: Calculate facial matching , get the face matching evaluation result, where is the facial matching degree, is the scaling factor, is the number of facial features, For the The weight coefficient of each facial feature, Is the user's current face eigenvalues, It is the first feature values.

[0009] The present invention is improved in that the method for identifying the iris edge in the iris image is: S311: Based on the image capturing device, capture the user's eye image, and use the formula: Calculate the edge strength of the pixel ,in and are the gradients of the pixel in the x-axis and y-axis directions, is the edge strength, and is the gradient weight, is the noise suppression parameter; S312: Based on the edge strength of the pixel point , compared with the preset gradient threshold, determine whether the pixel is at the edge of the iris, identify the iris area in the iris image, and obtain the iris area recognition result.

[0010] The present invention is improved in that the steps of obtaining the iris matching evaluation result are: S321: extracting key features in the iris image based on the iris area recognition result, including annular area, texture and color features; S322: Based on the key features and user identity information extraction results in the iris image, the user's current iris features are compared with the pre-stored user iris data, and the formula is: Calculate iris matching , get the iris matching evaluation result, where is the iris matching degree, is the scaling factor, is the number of iris features, For the The weight of the iris feature, For the user's Iris feature values, It is the pre-stored iris feature values.

[0011] The present invention is improved in that the method for evaluating the matching degree of the user identity is: S411: extracting the user's iris matching degree and facial matching degree data based on the iris matching evaluation result and the facial matching degree evaluation result to obtain identity matching association data; S412: Match the associated data based on the identity, by formula: Calculating identity match scores , get the identity matching information, where Score the identity match, and is the weight coefficient, To adjust the parameters, is the iris matching degree, is the facial matching degree.

[0012] The present invention is improved in that the steps of obtaining the user identity verification result are: S421: extracting the security level and the benchmark verification threshold required by the device based on the identity authentication environment, and obtaining verification threshold associated data; S422: Based on the verification threshold associated data, by formula: Calculating the adjusted verification threshold ,in, is the adjusted verification threshold, is the baseline verification threshold, is the security level adjustment factor, is the environmental impact factor, and To adjust the parameters; S423: Based on the adjusted verification threshold and identity matching score , score the identity matching With the adjusted verification threshold For comparison, if the identity matching score Exceeding the adjusted verification threshold , determine whether the identity verification is passed and obtain the user identity verification result.

[0013] An identity verification system based on multimodal data fusion, the identity verification system based on multimodal data fusion is used to execute the above-mentioned identity verification method based on multimodal data fusion, the system comprises: The user information extraction module compares the login information with the pre-stored information based on the user login information, identifies the user identity, and extracts the user's pre-stored facial data and iris data to obtain the user identity information extraction result; The image quality optimization module captures the user's facial image based on the user identity information extraction result, optimizes the brightness and contrast of the facial image, optimizes the image quality, and obtains an optimized facial image; The facial matching module extracts the user's facial features, compares the user's facial features with the pre-stored user's facial data, evaluates the facial matching degree, and obtains the facial matching evaluation result; The iris matching module captures the user's current eye image based on the user identity information extraction result, identifies the iris edge in the iris image through image analysis, compares the iris features with the pre-stored user iris data, evaluates the iris matching degree, and obtains the iris matching evaluation result; The identity verification module evaluates the matching degree of the user identity based on the iris matching evaluation result and the facial matching evaluation result, and adjusts the matching degree verification threshold in combination with the security level required by the device to determine whether the identity verification is passed, and obtains the user identity verification result.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by combining facial and iris recognition technologies, optimizing image quality and comparing pre-stored data to improve the accuracy of identity verification, the possibility of misidentification and identity fraud can be significantly reduced. By dynamically adjusting the matching verification threshold, it can flexibly respond to different security needs, enhance the adaptability and protection of the verification process, effectively reduce the false recognition rate and rejection rate, and provide a more stringent security verification process, achieving a higher level of personal identity protection, which is particularly suitable for occasions requiring high security, effectively preventing identity theft and other related fraudulent behaviors, helping institutions meet more stringent compliance requirements, and enhancing consumers' trust in digital services. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the method of the present invention; Figure 2 A flow chart for optimizing image quality for the present invention; Figure 3 A flowchart for obtaining a facial matching evaluation result for the present invention; Figure 4 A flow chart of identifying an iris edge in an iris image according to the present invention; Figure 5 A flow chart of obtaining iris matching evaluation results for the present invention; Figure 6 A flow chart of evaluating the matching degree of a user identity for the present invention; Figure 7 The present invention is a flow chart of obtaining user identity verification results. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0017] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating positions or positional relationships, are based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0018] See also Figure 1 The present invention provides a technical solution: an identity verification method based on multimodal data fusion, comprising the following steps: S1: Based on the user login information, the login information is compared with the pre-stored information to identify the user identity, and the user's pre-stored facial data and iris data are extracted to obtain the user identity information extraction result; S2: based on the user identity information extraction result, capture the user's facial image, optimize the brightness and contrast of the facial image, optimize the image quality, extract the user's facial features, compare the user's facial features with the pre-stored user facial data, evaluate the facial matching degree, and obtain the facial matching evaluation result; S3: Based on the user identity information extraction result, capture the user's current eye image, identify the iris edge in the iris image through image analysis, compare the iris features with the pre-stored user iris data, evaluate the iris matching degree, and obtain the iris matching evaluation result; S4: Based on the iris matching evaluation results and the facial matching evaluation results, the matching degree of the user identity is evaluated, and in combination with the security level required by the device, the matching degree verification threshold is adjusted to determine whether the identity verification is passed, and the user identity verification result is obtained.

[0019] The user identity information extraction results include the user facial dataset, iris dataset and user identity identification; the facial matching evaluation results include feature similarity scores and facial recognition marks; the iris matching evaluation results include iris feature point information, iris matching level and iris recognition marks; the user identity verification results include identity matching score, security level information and identity authentication status.

[0020] See also Figure 2 , the method to optimize image quality is: S211: Based on the image capture device, capture the user's facial image, and extract the current brightness and contrast data of the facial image to obtain basic image data; S212: Based on the basic image data, by formula: and Calculate the adjusted screen brightness And the adjusted image contrast , optimize the brightness and contrast of the facial image to obtain an optimized facial image, where is the average brightness value of the current image, is the weight coefficient for brightness adjustment, is the target brightness value, is the adjusted brightness value, is the contrast value of the current image, is the weight coefficient for contrast adjustment, is the target contrast value, is the adjusted contrast value.

[0021] formula: and Parameter details and how to obtain them: : The average brightness value of the current image can be obtained by calculating the arithmetic mean of the grayscale histogram of the image.

[0022] : The weight coefficient of brightness adjustment, usually preset according to actual application requirements to adjust the amplitude of brightness change.

[0023] : Target brightness value, set as the ideal brightness level for image processing, usually determined based on the image content and the characteristics of the display device.

[0024] : The contrast value of the current image, calculated by analyzing the brightness distribution of the image.

[0025] : Contrast adjustment weight coefficient, used to adjust the amplitude of contrast change.

[0026] : Target contrast value, preset to the ideal image contrast level.

[0027] : The adjusted contrast value, the result directly calculated by the formula, represents the adjusted image contrast.

[0028] Calculation example: Set the average brightness of the current image is 120, the target brightness Set to 130, weight coefficient is 0.1.

[0029] The brightness adjustment is calculated as follows: Indicates that the adjusted image brightness is 121.

[0030] Set current contrast The target contrast is 1.5. Set to 2.0, weight coefficient is 0.05. The contrast adjustment formula is calculated as follows: This indicates that the contrast of the adjusted image is about 1.56.

[0031] Through calculation and adjustment, the natural feeling of the image is maintained while its brightness and contrast are improved to achieve better visual effects.

[0032] See also Figure 3 , the steps to obtain the facial matching evaluation results are: S221: extracting the user's current facial features based on the optimized facial image, including feature point locations, facial expression parameters, and skin color information; S222: Based on the current facial features of the user and the extraction result of the user identity information, the facial features of the user are compared with the pre-stored facial data of the user, and the formula is: Calculate facial matching , get the face matching evaluation result, where is the facial matching degree, is the scaling factor, is the number of facial features, For the The weight coefficient of each facial feature, Is the user's current face eigenvalues, It is the first feature values.

[0033] formula: Parameter details and how to obtain them: : Scaling factor, used to adjust the range of the scoring results, usually set based on system design requirements. For example, if the system requires the output of the matching degree in percentage form, Can be set to 100.

[0034] : The number of facial features determines the number of facial feature points to be considered when calculating the match. This value is usually determined by the design of the facial recognition system to ensure sufficient accuracy and computational efficiency.

[0035] : Weight coefficient, used to adjust the importance of each facial feature in the match calculation. Usually assigned empirically based on the recognition importance and stability of the feature, for example, the weight of the eye position may be higher than the weight of the hairline position.

[0036] Is the user's current face eigenvalues, It is the first The feature values ​​are extracted and obtained through facial recognition technology.

[0037] Calculation example: Set the following parameters: , indicating that three features are considered (e.g., eye position, nose position, mouth position). , , the weights reflect the importance of different features. , represents the eigenvalue. , so that The maximum value is 100.

[0038] Calculation process: Calculation results , indicating that the user's facial features have a high match with the features stored in the database, and the matching score is 92.83, reflecting a very high similarity. The calculation process ensures that the evaluation of the accuracy of facial feature matching is based on a comprehensive consideration of weights and feature differences.

[0039] See also Figure 4 , the method to identify the iris edge in the iris image is: S311: Based on the image capturing device, capture the user's eye image, and use the formula: Calculate the edge strength of the pixel ,in and are the gradients of the pixel in the x-axis and y-axis directions, is the edge strength, and is the gradient weight, is the noise suppression parameter; S312: Pixel-based edge strength , compared with the preset gradient threshold, determine whether the pixel is at the edge of the iris, identify the iris area in the iris image, and obtain the iris area recognition result.

[0040] formula: Parameter details and how to obtain them: and : It is the gradient value of the image in the horizontal and vertical directions, calculated by applying the Sobel operator. The Sobel operator is a commonly used edge detection method in image processing, which can estimate the gradient of each pixel by responding to the horizontal and vertical changes of the image through a high-pass filter.

[0041] and : Gradient weight coefficient, used to adjust the influence of the gradient in the calculation process. The weight can be optimized according to the specific directional characteristics of the iris image. For example, if the iris structure of the image is more obvious in the horizontal direction, the weight can be increased. The value of .

[0042] : Noise suppression parameter is an adjustment factor used to reduce the impact of image noise. In practical applications, this parameter can be set based on the noise characteristics of the image, for example, by evaluating the noise level of the image to determine the appropriate value.

[0043] Calculation example: Set the following parameters: , : The value indicates the grayscale change of the image in the horizontal and vertical directions at a certain pixel. , : The weights reflect the relative importance of horizontal and vertical gradients in the total gradient calculation. : Indicates the adjustment degree of noise suppression.

[0044] The calculation process is as follows: Calculation results , indicating that at this pixel, the edge strength of the iris is 43.89, which is a quantitative indicator that reflects the clarity and recognition of the iris edge at this point. Higher values ​​mean sharper edges, which helps in more accurate iris recognition.

[0045] See also Figure 5 , the steps to obtain the iris matching evaluation results are: S321: extracting key features in the iris image based on the iris region recognition result, including annular region, texture and color features; S322: Based on the key features and user identity information extraction results in the iris image, the user's current iris features are compared with the pre-stored user iris data, using the formula: Calculate iris matching , get the iris matching evaluation result, where is the iris matching degree, is the scaling factor, is the number of iris features, For the The weight of the iris feature, For the user's Iris feature values, It is the pre-stored iris feature values.

[0046] formula: Parameter details and how to obtain them: : Scaling factor, used to adjust the range of the matching degree. This factor is usually set according to the output requirements of the system. For example, if you need to output a matching degree from 0 to 100, It can be set to 100 to indicate the degree of match.

[0047] : The number of iris features represents the total number of iris feature points used in the matching calculation. The parameter is determined based on the feature extraction technology in the design of the iris recognition system and is usually set based on the optimal number of features tested in advance.

[0048] : Feature weights, which are used to adjust the importance of different iris features in the matching calculation. These weights are preset based on the importance of the features in iris recognition, and are usually determined through statistical analysis and experimental optimization.

[0049] For the user's Iris feature values, It is the pre-stored Iris feature values ​​are extracted through image processing.

[0050] Calculation example: Set the following parameters: , (considering three iris features), (Weight Distribution), .

[0051] Calculation process: Calculation results This means that the currently extracted iris features have a very high match with the pre-stored iris features, which is close to a perfect match. This match indicates that the iris recognition system has verified the user's identity efficiently and accurately.

[0052] See also Figure 6 , the method for evaluating the matching degree of user identity is: S411: extracting the iris matching degree and facial matching degree data of the user based on the iris matching evaluation result and the facial matching degree evaluation result to obtain identity matching association data; S412: Match the associated data based on the identity, by formula: Calculating identity match scores , get the identity matching information, where Score the identity match, and is the weight coefficient, To adjust the parameters, is the iris matching degree, is the facial matching degree.

[0053] formula: Parameter details and how to obtain them: , and : Weight parameters are adjusted based on demand. For example, a high-security facility may require iris data to have a higher weight because iris data is generally more difficult to forge than facial data. These parameters are usually set by the security strategy team based on past data and risk analysis. Controls the influence of iris matching on the overall matching. Controls the influence of the face match on the overall match. Adjust parameters to balance and refine the calculation of matching degree to meet the needs of different situations.

[0054] and : They are iris matching and face matching, respectively, calculated through the previous steps.

[0055] Calculation example: set up: , , where iris data is more important than facial data. , indicating that the iris matching degree is 85%. , indicating a facial match of 75%.

[0056] Calculation process: Calculation results , which represents the identity matching score after the comprehensive iris and facial matching. The score will be used to compare with the verification threshold to determine whether the identity verification is passed.

[0057] See also Figure 7, the steps to obtain the user identity verification results are: S421: extracting the security level and the benchmark verification threshold required by the device based on the identity authentication environment, and obtaining verification threshold associated data; S422: Based on the verification threshold correlation data, the formula is: Calculating the adjusted verification threshold ,in, is the adjusted verification threshold, is the baseline verification threshold, is the security level adjustment factor, is the environmental impact factor, and To adjust the parameters; S423: Based on the adjusted verification threshold and identity matching score , score the identity matching With the adjusted verification threshold For comparison, if the identity matching score Exceeding the adjusted verification threshold , determine whether the identity verification is passed and obtain the user identity verification result.

[0058] formula: Parameter details and how to obtain them :Baseline verification threshold, usually based on the system default settings or industry standards, is set as a starting point value that meets most security requirements. This value is determined by the system administrator or security policy to ensure the security of the baseline.

[0059] : Security level adjustment factor, set according to the security level required for the device. For example, higher risk or sensitive data processing may increase the value of this factor to increase the verification threshold and thus strengthen security.

[0060] :Environmental impact factor, which is set according to the operating environment. For example, for an online operating environment, its value will be set higher to prevent online cheating. For an offline operating environment, the value can be set smaller because there are staff on-site supervision.

[0061] and : Tuning parameters used to fine-tune the baseline thresholds based on the specific requirements of the security level. These parameters can be adjusted based on empirical data to best address the current security challenges and environmental conditions.

[0062] Calculation example: set up: : Baseline threshold, : Indicates that the security level of the device environment is low. :Environmental factors, : Safety adjustment factor, : Environmental adjustment coefficient.

[0063] Calculate the adjusted match check threshold Calculation results , showing that under the current security and environment settings, the adjusted match verification threshold is 7.2611. This threshold will be used to evaluate the user's match to determine the result of their identity verification.

[0064] An identity verification system based on multimodal data fusion, the identity verification system based on multimodal data fusion is used to execute the above-mentioned identity verification method based on multimodal data fusion, the system comprises: The user information extraction module compares the login information with the pre-stored information based on the user login information, identifies the user identity, and extracts the user's pre-stored facial data and iris data to obtain the user identity information extraction result; The image quality optimization module captures the user's facial image based on the user identity information extraction result, optimizes the brightness and contrast of the facial image, optimizes the image quality, and obtains an optimized facial image; The facial matching module extracts the user's facial features, compares the user's facial features with the pre-stored user's facial data, evaluates the facial matching degree, and obtains the facial matching evaluation result; The iris matching module captures the user's current eye image based on the user identity information extraction result, identifies the iris edge in the iris image through image analysis, compares the iris features with the pre-stored user iris data, evaluates the iris matching degree, and obtains the iris matching evaluation result; The identity verification module evaluates the matching degree of the user's identity based on the iris matching evaluation results and the facial matching evaluation results, and adjusts the matching degree verification threshold in combination with the security level required by the device to determine whether the identity verification is passed and obtain the user identity verification result.

[0065] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. An identity verification method based on multimodal data fusion, characterized in that: The following steps are involved: Based on the user login information, the login information is compared with the pre-stored information to identify the user's identity, and the user's pre-stored facial data and iris data are extracted to obtain the user identity information extraction result; Based on the user identity information extraction result, capture the user's facial image, optimize the brightness and contrast of the facial image, optimize the image quality, extract the user's facial features, compare the user's facial features with pre-stored user facial data, evaluate the facial matching degree, and obtain a facial matching evaluation result; Based on the user identity information extraction result, the user's current eye image is captured, the iris edge in the iris image is identified through image analysis, the iris feature is compared with the pre-stored user iris data, the iris matching degree is evaluated, and an iris matching evaluation result is obtained; Based on the iris matching evaluation result and the facial matching evaluation result, the matching degree of the user identity is evaluated, and in combination with the security level required by the device, the matching degree verification threshold is adjusted to determine whether the identity verification is passed, and the user identity verification result is obtained; The method for evaluating the matching degree of the user identity is: Based on the iris matching evaluation result and the facial matching evaluation result, extract the user's iris matching degree and facial matching degree data to obtain identity matching association data; Based on the identity matching associated data, by the formula: Calculating identity match scores , get the identity matching information, where Score the identity match, and is the weight coefficient, To adjust the parameters, is the iris matching degree, is the facial matching degree.

2. The identity verification method based on multimodal data fusion according to claim 1, characterized in that: The method for optimizing image quality is: Based on the image capture device, the facial image of the user is captured, and the current brightness and contrast data of the facial image are extracted to obtain basic image data; Based on the basic image data, by the formula: and Calculate the adjusted screen brightness And the adjusted image contrast , optimize the brightness and contrast of the facial image to obtain an optimized facial image, where is the average brightness value of the current image, is the weight coefficient for brightness adjustment, is the target brightness value, is the adjusted brightness value, is the contrast value of the current image, is the weight coefficient for contrast adjustment, is the target contrast value, is the adjusted contrast value.

3. The identity verification method based on multimodal data fusion according to claim 2, characterized in that: The steps for obtaining the face matching evaluation result are: Based on the optimized facial image, extract the user's current facial features, including feature point locations, facial expression parameters, and skin color information; Based on the current facial features of the user and the user identity information extraction results, the user facial features are compared with the pre-stored user facial data, and the formula is: Calculate facial matching , get the face matching evaluation result, where is the facial matching degree, is the scaling factor, is the number of facial features, For the The weight coefficient of each facial feature, Is the user's current face eigenvalues, It is the first feature values.

4. The identity verification method based on multimodal data fusion according to claim 1, characterized in that: The method for identifying the iris edge in the iris image is: Based on the image capture device, the user's eye image is captured and the formula is used: Calculate the edge strength of the pixel ,in and are the gradients of the pixels in the x-axis and y-axis directions, is the edge strength, and is the gradient weight, is the noise suppression parameter; Based on the edge strength of the pixel , compared with the preset gradient threshold, determine whether the pixel is at the edge of the iris, identify the iris area in the iris image, and obtain the iris area recognition result.

5. The identity verification method based on multimodal data fusion according to claim 4 is characterized in that: The steps for obtaining the iris matching evaluation result are: Based on the iris region recognition result, extract key features in the iris image, including annular region, texture and color features; Based on the key features and user identity information extraction results in the iris image, the user's current iris features are compared with the pre-stored user iris data, and the formula is: Calculate iris matching , get the iris matching evaluation result, where is the iris matching degree, is the scaling factor, is the number of iris features, For the The weight of the iris feature, For the user's Iris feature values, It is the pre-stored iris feature values.

6. The identity verification method based on multimodal data fusion according to claim 1, characterized in that: The steps for obtaining the user identity verification result are: Based on the identity authentication environment, the security level and the baseline verification threshold required by the device are extracted to obtain verification threshold associated data; Based on the verification threshold associated data, by formula: Calculating the adjusted verification threshold ,in, is the adjusted verification threshold, is the baseline verification threshold, is the security level adjustment factor, is the environmental impact factor, and To adjust the parameters; Based on the adjusted verification threshold and identity matching score , score the identity matching With the adjusted verification threshold For comparison, if the identity matching score Exceeding the adjusted verification threshold , determine whether the identity verification is passed and obtain the user identity verification result.

7. The identity verification system based on multimodal data fusion is characterized by: According to any one of claims 1 to 6, the identity verification method based on multimodal data fusion comprises: The user information extraction module compares the login information with the pre-stored information based on the user login information, identifies the user identity, and extracts the user's pre-stored facial data and iris data to obtain the user identity information extraction result; The image quality optimization module captures the user's facial image based on the user identity information extraction result, optimizes the brightness and contrast of the facial image, optimizes the image quality, and obtains an optimized facial image; The facial matching module extracts the user's facial features, compares the user's facial features with the pre-stored user's facial data, evaluates the facial matching degree, and obtains the facial matching evaluation result; The iris matching module captures the user's current eye image based on the user identity information extraction result, identifies the iris edge in the iris image through image analysis, compares the iris features with the pre-stored user iris data, evaluates the iris matching degree, and obtains the iris matching evaluation result; The identity verification module evaluates the matching degree of the user identity based on the iris matching evaluation result and the facial matching evaluation result, and adjusts the matching degree verification threshold in combination with the security level required by the device to determine whether the identity verification is passed, and obtains the user identity verification result.

Images