A method and system for intelligent verification of identity based on multimodal AI fusion

Abstract

This invention relates to the field of image processing technology, and discloses a method and system for intelligent identity verification based on multimodal AI fusion. The method includes: real-time acquisition of facial image data and document information of a target passenger; analysis of the target passenger's facial temperature gradient and temperature fluctuation characteristics; calculation of the temperature consistency coefficient of the facial infrared image; analysis of the target passenger's microvascular pulsation signal; calculation of the pulsation signal-to-noise ratio of the microvascular pulsation signal; fitting a three-dimensional AI model of the target passenger's face; calculation of the facial curvature fluctuation value and respiratory micro-motion frequency of the three-dimensional AI model to analyze the non-rigid deformation characteristics of the target passenger; analysis of the data confidence of the facial image data; when the data confidence is greater than a preset confidence threshold, analysis of the facial similarity between the facial depth image and the document face image; and execution of intelligent identity verification for the target passenger. This invention can improve the accuracy and reliability of identity verification.

Description

Technical Field

[0001] This invention relates to a human identity verification method and system based on multimodal AI fusion, belonging to the field of image processing technology. Background Technology

[0002] Intelligent verification of identity is an identity verification method based on multimodal biometric recognition and artificial intelligence technology. It compares the biometric features of the certificate holder with the registration information stored in the chip of the certificate in real time to ensure that the person, certificate and status are completely matched, thereby eliminating security risks such as identity theft and document forgery.

[0003] Traditional verification methods mainly rely on comparison of single biometric features (such as facial recognition) and manual identification documents. This approach depends solely on the comparison of faces and documents and lacks comprehensive analysis of multimodal data (such as infrared images and depth images). As a result, the verification dimension is singular, and it is easily affected by factors such as ambient lighting, angle, and occlusion. It cannot identify high-level imitations (such as masks and photo deception) and cannot dynamically verify liveness features. Consequently, it is impossible to accurately identify the specific identity of passengers, resulting in a high risk of impersonation and a low verification accuracy rate. Summary of the Invention

[0004] This invention provides a method and system for intelligent verification of identity based on multimodal AI fusion, the main purpose of which is to improve the accuracy and reliability of identity verification.

[0005] To achieve the above objectives, this invention provides a human-identity verification method based on multimodal AI fusion, comprising:

[0006] Real-time collection of facial image data and identification information of target passengers, including facial infrared images and facial depth images;

[0007] Based on the facial infrared image, analyze the facial temperature gradient and temperature fluctuation characteristics of the target passenger; based on the facial temperature gradient, calculate the temperature consistency coefficient of the facial infrared image; based on the temperature fluctuation characteristics, analyze the microvascular pulsation signal of the target passenger, and calculate the pulsation signal-to-noise ratio of the microvascular pulsation signal.

[0008] Based on the facial depth image, a facial AI 3D model of the target passenger is fitted, and the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI 3D model are calculated to analyze the non-rigid deformation characteristics of the target passenger. Combining the non-rigid deformation characteristics, the signal-to-noise ratio of the pulsation signal, and the temperature consistency coefficient, the data confidence of the facial image data is analyzed.

[0009] When the confidence level of the data is greater than the preset confidence level threshold, the contour matching degree and texture feature similarity of the facial depth image and the ID card face image corresponding to the ID card information are calculated respectively to analyze the face similarity between the facial depth image and the ID card face image.

[0010] Based on the facial similarity, perform intelligent verification of the target passenger's identity and identification.

[0011] Optionally, analyzing the facial temperature gradient and temperature fluctuation characteristics of the target passenger based on the facial infrared image includes:

[0012] The facial infrared image is denoised to obtain a denoised facial infrared image;

[0013] Radiometric correction is performed on the denoised facial infrared image to obtain a corrected facial infrared image;

[0014] Detect the facial region in the corrected facial infrared image and extract the key points of the facial region;

[0015] Based on the key points, the face region is divided into multiple regions of interest;

[0016] The pixel values ​​in the multi-interest regions are converted into temperature values ​​to calculate the average temperature of each region in the multi-interest regions;

[0017] Based on the average temperature, the temperature difference between the multiple interest regions is calculated to determine the facial temperature gradient of the target passenger.

[0018] The temperature values ​​are processed over time to obtain a temperature time series;

[0019] The temperature time series is transformed in the frequency domain to obtain a frequency domain signal;

[0020] Identify the dominant frequency of temperature fluctuation in the frequency domain signal and analyze the temperature fluctuation characteristics of the dominant frequency.

[0021] Optionally, calculating the temperature uniformity coefficient of the facial infrared image based on the facial temperature gradient includes:

[0022] Based on the facial temperature gradient, a temperature gradient vector is constructed for each pixel in the facial infrared image;

[0023] Based on the temperature gradient vector, calculate the temperature gradient intensity and direction of each pixel in the facial infrared image;

[0024] Calculate the temperature standard deviation of all pixels in the facial infrared image;

[0025] The temperature uniformity coefficient of the facial infrared image is calculated using the following formula based on the gradient intensity, the temperature gradient direction, and the temperature standard deviation:

[0026]

[0027] in, Indicates the temperature uniformity coefficient. Represents the weights of regional gradient similarity. This represents the total number of regions of interest corresponding to a facial infrared image. Indicates the first in the facial infrared image Areas of interest Indicates the first Total number of pixels in each region of interest Indicates the first The first interest area Temperature gradient vector of 1 pixel, Indicates the first The first interest area Temperature gradient vector of 1 pixel, Indicates the first The first interest area Temperature gradient intensity per pixel Indicates the first The first interest area Temperature gradient intensity per pixel Indicates the weights for gradient differences across regions. Indicates the first in the facial infrared image Areas of interest Represents the median function. Indicates the first The set of all temperature gradient directions in a region of interest. Indicates the first The set of all temperature gradient directions in a region of interest. Represents pi (π). Indicates the weight of temperature standard deviation. This represents the standard deviation of temperature.

[0028] Optionally, analyzing the microvascular pulsation signal of the target passenger based on the temperature fluctuation characteristics includes:

[0029] Construct the temperature feature vector of the temperature fluctuation characteristics;

[0030] Based on the feature vector, a temperature-pulse mapping algorithm is fitted to the target passenger.

[0031] Based on the temperature feature vector, the microvascular pulsation signal of the target passenger is calculated using the temperature-pulsation mapping algorithm.

[0032] Optionally, fitting a facial AI 3D model of the target passenger based on the facial depth image includes:

[0033] Determine the camera optical center and camera focal length of the camera corresponding to the facial depth image;

[0034] The coordinate transformation matrix of the facial depth image is determined based on the camera optical center and the camera focal point.

[0035] Based on the coordinate transformation matrix, the pixel coordinates corresponding to the facial depth image are converted into three-dimensional spatial coordinates;

[0036] Based on the three-dimensional spatial coordinates, point cloud data of the target passenger is generated;

[0037] Based on the point cloud data, a facial AI 3D model of the target passenger is fitted.

[0038] Optionally, calculating the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI 3D model includes:

[0039] Based on the point cloud data corresponding to the facial AI 3D model, fit a local quadratic surface of the facial AI 3D model;

[0040] Calculate the fitting coefficients of the local quadratic surface;

[0041] Calculate the Gaussian curvature of the local quadratic surface based on the fitting coefficients;

[0042] Construct the curvature time series of the Gaussian curvature;

[0043] The mean curvature and standard deviation of the curvature time series are calculated to analyze the facial curvature fluctuation values ​​of the facial AI 3D model;

[0044] Identify the nasal wing node of the facial AI 3D model;

[0045] Calculate the displacement of the nose wing node to construct a displacement time series of the nose wing node;

[0046] The displacement time series is transformed into the frequency domain to obtain the displacement frequency domain signal;

[0047] The respiratory micro-motion frequency of the facial AI 3D model is determined based on the displacement frequency domain signal.

[0048] Optionally, the step of analyzing the data confidence of the facial image data by combining the non-rigid deformation characteristics, the signal-to-noise ratio of the pulsating signal, and the temperature consistency coefficient includes:

[0049] Construct the non-rigid feature vector of the non-rigid deformation feature;

[0050] The non-rigid feature vector, the signal-to-noise ratio of the pulsating signal, and the temperature uniformity coefficient are fused to obtain a fused feature matrix;

[0051] Calculate the weight matrix of the fused feature matrix;

[0052] The data confidence level of the facial image data is calculated based on the fused feature matrix and the weight matrix.

[0053] Optionally, when the data confidence level is greater than a preset confidence threshold, the contour matching degree and texture feature similarity of the facial depth image and the ID card face image corresponding to the ID card information are calculated respectively, including:

[0054] Extract the actual face contour from the facial depth image and the document face contour from the document face image, respectively;

[0055] The actual face contour and the document face contour are normalized to obtain normalized actual face contour and normalized document face contour.

[0056] Calculate the set of contour point distances between the normalized actual face contour and the normalized ID card face contour;

[0057] Based on the set of contour point distances, determine the contour matching degree between the normalized actual face contour and the normalized ID card face contour;

[0058] Extract the actual texture features of the facial depth image and the document texture features of the document face image, respectively;

[0059] Based on the actual texture features and the document texture features, calculate the texture feature similarity between the facial depth image and the document face image.

[0060] Optionally, the real-time acquisition of facial image data and identification information of the target passenger includes:

[0061] Configure the target passenger with multiple sensors, a document reader, and a microprocessor;

[0062] Define the communication protocol for the multi-sensor, the document reader, and the microprocessor;

[0063] Based on the communication protocol, a network topology is constructed for the multi-sensor, the document reader, and the microprocessor.

[0064] Based on the network topology, a multi-sensor network is formed that integrates the multi-sensor, the document reader, and the microprocessor.

[0065] The target passenger's facial image data and identification information are collected using the multi-sensor network.

[0066] To address the aforementioned problems, this invention also provides a human-identity verification system based on multimodal AI fusion, the system comprising:

[0067] The data acquisition module is used to collect facial image data and document information of target passengers in real time. The facial image data includes facial infrared images and facial depth images.

[0068] The facial temperature analysis module allows the user to analyze the facial temperature gradient and temperature fluctuation characteristics of the target passenger based on the facial infrared image, calculate the temperature consistency coefficient of the facial infrared image based on the facial temperature gradient, and analyze the microvascular pulsation signal of the target passenger based on the temperature fluctuation characteristics, and calculate the pulsation signal-to-noise ratio of the microvascular pulsation signal.

[0069] The data confidence analysis module is used to fit the facial AI three-dimensional model of the target passenger based on the facial depth image, calculate the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI three-dimensional model, so as to analyze the non-rigid deformation characteristics of the target passenger, and combine the non-rigid deformation characteristics, the signal-to-noise ratio of the pulsation signal and the temperature consistency coefficient to analyze the data confidence of the facial image data.

[0070] The face similarity calculation module is used to calculate the contour matching degree and texture feature similarity of the facial depth image and the ID card face image corresponding to the ID card information respectively when the confidence degree of the data is greater than the preset confidence degree threshold, so as to analyze the face similarity between the facial depth image and the ID card face image.

[0071] The identity verification module is used to perform intelligent identity verification of the target passenger based on the facial similarity. Compared with the problems described in the background art, the present invention proposes an intelligent identity verification method and system based on multimodal AI fusion. By acquiring facial infrared and depth images of the target passenger in real time and combining them with document information for multi-dimensional feature extraction and intelligent analysis, it achieves high-precision and robust identity verification, with significant technical benefits. First, by analyzing the temperature gradient and temperature fluctuation characteristics of the facial infrared images, the system can effectively calculate the temperature consistency coefficient and extract microvascular pulsation signals, and then calculate the signal-to-noise ratio of the pulsation signals. This process not only realizes the quantitative assessment of body temperature distribution, but also captures the state of human microcirculation, providing a reliable physiological basis for liveness detection, greatly improving anti-counterfeiting capabilities, and resisting deception methods such as photos, videos, and masks. Second, based on the facial depth images, an AI three-dimensional model is constructed. The system can accurately calculate the facial curvature fluctuation value and respiratory micro-movement frequency, thereby identifying non-rigid deformation features. These features reflect subtle physiological changes in the target passenger under natural conditions. Combined with the pulse signal-to-noise ratio and temperature consistency coefficient, they jointly construct a multi-dimensional data confidence assessment system. The system only triggers subsequent facial similarity calculations when the data confidence exceeds a preset threshold, effectively reducing the false positive rate and improving the accuracy and reliability of verification. In the facial similarity analysis stage, the system compares the facial depth image with the ID card face image from two dimensions: contour matching degree and texture feature similarity. This dual-modal comparison strategy not only considers the consistency of geometric structure but also takes into account the detailed matching of surface textures, further enhancing the comprehensiveness and accuracy of identity verification. Finally, based on the dual judgment of high confidence and high similarity, this system achieves intelligent identity verification, significantly improving the security and efficiency of identity authentication. It is suitable for various high-security scenarios such as airports, train stations, finance, and government agencies. Simultaneously, the system adopts a non-contact data collection method, optimizing the user experience and possessing good scalability and application prospects. Therefore, the intelligent verification method for identity verification based on multimodal AI fusion provided in this embodiment of the invention can improve the accuracy and reliability of identity verification. Attached Figure Description

[0072] Figure 1 This is a flowchart illustrating an intelligent verification method for identity verification based on multimodal AI fusion, provided in an embodiment of the present invention.

[0073] Figure 2 This is a schematic diagram of the modules for implementing the intelligent verification system for identity verification based on multimodal AI fusion, provided in an embodiment of the present invention.

[0074] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0075] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0076] This application provides a method for intelligent identity verification based on multimodal AI fusion. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0077] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent verification method for identity verification based on multimodal AI fusion according to an embodiment of the present invention. In this embodiment, the intelligent verification method for identity verification based on multimodal AI fusion includes:

[0078] S1. Real-time acquisition of facial image data and identification information of target passengers, including facial infrared images and facial depth images.

[0079] This invention, through real-time acquisition of facial image data and identification information of target passengers, can verify identity from multiple dimensions, providing a data foundation for subsequent anti-counterfeiting and anti-fraud analysis. The facial image data refers to a composite dataset that includes not only visible light facial images but also multiple imaging modalities. The identification information refers to data included in official documents used for identity verification. The facial infrared image refers to a facial thermal radiation image acquired using infrared imaging technology. The facial depth image refers to a three-dimensional spatial information image of the face acquired through depth sensing technology.

[0080] As an embodiment of the present invention, the real-time acquisition of facial image data and identification information of the target passenger includes:

[0081] Configure the target passenger with multiple sensors, a document reader, and a microprocessor;

[0082] Define the communication protocol for the multi-sensor, the document reader, and the microprocessor;

[0083] Based on the communication protocol, a network topology is constructed for the multi-sensor, the document reader, and the microprocessor.

[0084] Based on the network topology, a multi-sensor network is formed that integrates the multi-sensor, the document reader, and the microprocessor.

[0085] The target passenger's facial image data and identification information are collected using the multi-sensor network.

[0086] The "multi-sensor" refers to a combination of various sensing devices used to collect facial image data of target passengers, such as infrared cameras, depth cameras, and visible light cameras. The "document reader" refers to a device used to automatically read the document information of target passengers. The "microprocessor" refers to the core computing unit responsible for system control, data processing, and communication coordination. The "communication protocol" refers to the set of rules for data transmission between the multi-sensor, the document reader, and the microprocessor, such as TCP / IP, UDP, and RS232 / 485 serial communication. The "network topology" refers to the physical and logical connection structure of the multi-sensor network, the document reader, and the microprocessor.

[0087] Optionally, the communication protocol of the multi-sensor, the document reader, and the microprocessor can be defined using blockchain technology. For example, a blockchain network layer composed of nodes such as the multi-sensor, the document reader, and the microprocessor can be responsible for data storage and consensus, and a smart contract layer can be deployed to define communication rules, data verification, and access control.

[0088] Optionally, the network topology of the multiple sensors, the document reader, and the microprocessor can be constructed using software-defined networking, such as OpenDaylight.

[0089] S2. Based on the facial infrared image, analyze the facial temperature gradient and temperature fluctuation characteristics of the target passenger; based on the facial temperature gradient, calculate the temperature consistency coefficient of the facial infrared image; based on the temperature fluctuation characteristics, analyze the microvascular pulsation signal of the target passenger, and calculate the pulsation signal-to-noise ratio of the microvascular pulsation signal.

[0090] This invention, through analysis of the facial infrared image and the facial temperature gradient and temperature fluctuation characteristics of the target passenger, can detect dynamic temperature changes and determine whether the person is real, effectively resisting deception attacks. The facial temperature gradient refers to the rate of change of facial surface temperature in space. The fluctuation characteristics refer to statistical features extracted based on the dominant frequency of temperature fluctuations, such as dominant frequency amplitude, band energy, phase information, fluctuation period, and stability indicators.

[0091] As an embodiment of the present invention, the step of analyzing the facial temperature gradient and temperature fluctuation characteristics of the target passenger based on the facial infrared image includes:

[0092] The facial infrared image is denoised to obtain a denoised facial infrared image;

[0093] Radiometric correction is performed on the denoised facial infrared image to obtain a corrected facial infrared image;

[0094] Detect the facial region in the corrected facial infrared image and extract the key points of the facial region;

[0095] Based on the key points, the face region is divided into multiple regions of interest;

[0096] The pixel values ​​in the multi-interest regions are converted into temperature values ​​to calculate the average temperature of each region in the multi-interest regions;

[0097] Based on the average temperature, the temperature difference between the multiple interest regions is calculated to determine the facial temperature gradient of the target passenger.

[0098] The temperature values ​​are processed over time to obtain a temperature time series;

[0099] The temperature time series is transformed in the frequency domain to obtain a frequency domain signal;

[0100] Identify the dominant frequency of temperature fluctuation in the frequency domain signal and analyze the temperature fluctuation characteristics of the dominant frequency.

[0101] The denoised facial infrared image refers to a facial infrared image after denoising processing. The corrected facial infrared image refers to a facial infrared image that has undergone radiometric correction processing based on denoising. The face region refers to the extracted rectangular region containing the complete facial structure. The key points refer to points in the face region with significant semantic or geometric features, such as the corners of the eyes, the tip of the nose, the corners of the mouth, and the center of the eyebrows. The multiple regions of interest refer to dividing the face region into multiple sub-regions with specific structural significance based on key points, such as the forehead, left cheek, right cheek, nose, and chin. The temperature value refers to converting the grayscale value of each pixel in the infrared image into the actual physical temperature using a calibration formula. The average temperature refers to averaging the temperature values ​​of all pixels in each region of interest to obtain the representative temperature value of that region. The temperature time series refers to sampling the temperature values ​​of the same region of interest on consecutive time frames to form a temperature data sequence that changes over time. The frequency domain signal refers to the signal obtained by converting the temperature time series from the time domain to the frequency domain. The dominant frequency of temperature fluctuation refers to the frequency component with the highest power spectral density in the frequency domain signal, which reflects the main periodic rhythm of temperature change.

[0102] Optionally, the denoised facial infrared image can be obtained using image denoising algorithms, such as Gaussian filtering, median filtering, wavelet denoising, etc.

[0103] Optionally, the face region can be extracted using face detection algorithms, such as Haar cascade, YOLO, MTCNN, etc.

[0104] Optionally, the key points of the face region can be obtained through key point detection algorithms, such as dlib, MediaPipe, HRNet, etc.

[0105] Optionally, the frequency domain signal can be obtained by frequency domain transformation algorithms, such as Fast Fourier Transform (FFT), wavelet transform, etc.

[0106] This invention, through calculating the temperature consistency coefficient of the facial infrared image based on the facial temperature gradient, can effectively distinguish between real and fake faces by identifying complex and dynamic temperature change patterns that forgery materials cannot simulate. The temperature consistency coefficient is a mathematical index used to quantify the consistency of facial temperature gradient distribution.

[0107] As an embodiment of the present invention, the step of calculating the temperature uniformity coefficient of the facial infrared image based on the facial temperature gradient includes:

[0108] Based on the facial temperature gradient, a temperature gradient vector is constructed for each pixel in the facial infrared image;

[0109] Based on the temperature gradient vector, calculate the temperature gradient intensity and direction of each pixel in the facial infrared image;

[0110] Calculate the temperature standard deviation of all pixels in the facial infrared image;

[0111] Based on the gradient intensity, the temperature gradient direction, and the temperature standard deviation, the temperature consistency coefficient of the facial infrared image is calculated using the following formula. The calculation method is not unique and other forms may also be used. This formula does not affect the implementation of the above technical solution but is merely another further implementation method:

[0112]

[0113] in, Indicates the temperature uniformity coefficient. Represents the weights of regional gradient similarity. This represents the total number of regions of interest corresponding to a facial infrared image. Indicates the first in the facial infrared image Areas of interest Indicates the first Total number of pixels in each region of interest Indicates the first The first interest area Temperature gradient vector of 1 pixel, Indicates the first The first interest area Temperature gradient vector of 1 pixel, Indicates the first The first interest area Temperature gradient intensity per pixel Indicates the first The first interest area Temperature gradient intensity per pixel Indicates the weights for gradient differences across regions. Indicates the first in the facial infrared image Areas of interest Represents the median function. Indicates the first The set of all temperature gradient directions in a region of interest. Indicates the first The set of all temperature gradient directions in a region of interest. Represents pi (π). Indicates the weight of temperature standard deviation. This represents the standard deviation of temperature.

[0114] Wherein, the temperature gradient vector refers to the vector representing the direction of temperature change of a pixel on the image plane. The temperature gradient intensity refers to the magnitude of the temperature gradient vector. The temperature gradient direction refers to the angle of the temperature gradient vector. The temperature standard deviation refers to the standard deviation of the temperature values ​​of all pixels in the entire infrared image. The regional gradient similarity weight refers to the contribution used to adjust the consistency of gradient direction within the same region of interest. The cross-regional gradient difference weight refers to the contribution used to adjust the difference in gradient direction between different regions of interest.

[0115] Optionally, the temperature gradient vector of each pixel in the facial infrared image can be constructed using image gradient operators, such as the Sobel operator, Prewitt operator, Roberts operator, etc.

[0116] Optionally, the temperature standard deviation of all pixels in the facial infrared image can be calculated using deep learning techniques, such as lightweight CNN networks, graph convolutional networks, etc.

[0117] It needs to be explained that in this application, the formula... This represents the weight of the region gradient similarity, with a value of 0.5. This represents the weight for gradient differences across regions, with a value of 0.3. This represents the weight of the temperature standard deviation, with a value of 0.2.

[0118] For example, there are two regions of interest in a facial infrared image, each containing two pixels, and the temperature gradient vectors are represented as follows: , , , The temperature gradient intensity is , , , The sets of temperature gradient directions are respectively represented as , The standard deviation of temperature is The weights for regional gradient similarity (0.5), cross-regional gradient difference (0.3), and temperature standard deviation (0.2) are used in the formula to calculate... .

[0119] This invention, through analysis of the microvascular pulsation signals of the target passenger based on the temperature fluctuation characteristics, can identify forgeries such as photos, videos, and masks, effectively resisting various deception attacks. The microvascular pulsation signals refer to the minute surface temperature fluctuations caused by the periodic blood flow changes in the subcutaneous microvascular network of the human face due to heartbeats.

[0120] As an embodiment of the present invention, the step of analyzing the microvascular pulsation signal of the target passenger based on the temperature fluctuation characteristics includes:

[0121] Construct the temperature feature vector of the temperature fluctuation characteristics;

[0122] Based on the feature vector, a temperature-pulse mapping algorithm is fitted to the target passenger.

[0123] Based on the temperature feature vector, the microvascular pulsation signal of the target passenger is calculated using the temperature-pulsation mapping algorithm.

[0124] The temperature feature vector refers to a set of structured numerical features used to characterize temperature fluctuation patterns. The temperature-pulsation mapping algorithm is a model for establishing a mathematical relationship between the temperature feature vector and the microvascular pulsation signal. The pulsation signal refers to a time-series signal that can characterize the microvascular pulsation state of the target passenger's face, calculated and reconstructed using the temperature-pulsation mapping algorithm.

[0125] Optionally, the temperature feature vector of the temperature fluctuation feature can be constructed using deep learning methods, such as CNN, LSTM, etc.

[0126] As another implementation, the temperature-pulse mapping algorithm can be expressed by the following formula:

[0127]

[0128] in, express Microvascular pulsation signals at any given time. This represents the fluctuation amplitude in the temperature eigenvector. Represents the sine function. Represents pi (π). This represents the fluctuation frequency in the temperature eigenvector. This represents the fluctuation phase in the temperature eigenvector. Indicates the noise intensity factor. This represents the noise function.

[0129] It needs to be explained that in this application, the formula... This represents the noise intensity factor, with a value ranging from [0.01, 0.1]. Its value determines the degree of noise's impact on the signal. This represents the noise function, which follows a normal distribution.

[0130] This invention, by calculating the signal-to-noise ratio (SNR) of the microvascular pulsation signal, can effectively distinguish real faces from forgeries such as photos, videos, and masks, thereby improving the anti-counterfeiting capabilities of identity verification systems in high-security scenarios. The SNR refers to the ratio of the power of the microvascular pulsation signal to the power of the noise.

[0131] Optionally, the signal-to-noise ratio of the pulsation signal of the microvascular pulsation signal can be calculated by the power spectral density method, such as the Welch method or the multi-window method.

[0132] S3. Based on the facial depth image, fit the facial AI 3D model of the target passenger, calculate the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI 3D model to analyze the non-rigid deformation characteristics of the target passenger, and combine the non-rigid deformation characteristics, the signal-to-noise ratio of the pulsation signal and the temperature consistency coefficient to analyze the data confidence of the facial image data.

[0133] This invention provides a realistic facial structure information by fitting a 3D AI model of the target tourist's face based on the facial depth image, facilitating subsequent deformation analysis. The 3D AI model refers to a digital facial model with spatial geometric structure information constructed based on the target tourist's facial depth image.

[0134] As an embodiment of the present invention, fitting a facial AI 3D model of the target passenger based on the facial depth image includes:

[0135] Determine the camera optical center and camera focal length of the camera corresponding to the facial depth image;

[0136] The coordinate transformation matrix of the facial depth image is determined based on the camera optical center and the camera focal point.

[0137] Based on the coordinate transformation matrix, the pixel coordinates corresponding to the facial depth image are converted into three-dimensional spatial coordinates;

[0138] Based on the three-dimensional spatial coordinates, point cloud data of the target passenger is generated;

[0139] Based on the point cloud data, a facial AI 3D model of the target passenger is fitted.

[0140] Wherein, the camera optical center refers to the convergence point of all imaging rays in the camera imaging model. The camera focal length refers to the distance from the camera lens to the imaging plane. The coordinate transformation matrix is ​​a mathematical tool that maps image pixel coordinates to three-dimensional spatial coordinates, and is composed of the camera intrinsic parameter matrix and the camera extrinsic parameter matrix. The three-dimensional spatial coordinates refer to the position of a point in three-dimensional Euclidean space. The point cloud data refers to a set composed of a large number of three-dimensional coordinate points.

[0141] Optionally, the camera optical center and camera focal length of the camera corresponding to the facial depth image can be determined by a corner detection algorithm.

[0142] Optionally, the coordinate transformation matrix of the facial depth image can be determined using deep learning methods, such as PoseNet, DepthNet, etc.

[0143] This invention, through calculating the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI 3D model, can serve as a supplement to biometrics, used to prevent attacks from photos, videos, and masks, effectively improving the accuracy of verification. The facial curvature fluctuation value refers to the degree of change in the geometric curvature of the facial surface of the 3D model over time. The respiratory micro-motion frequency refers to the frequency features related to respiratory rhythm extracted by analyzing the minute deformations of the 3D facial model over time.

[0144] As an embodiment of the present invention, the calculation of the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI three-dimensional model includes:

[0145] Based on the point cloud data corresponding to the facial AI 3D model, fit a local quadratic surface of the facial AI 3D model;

[0146] Calculate the fitting coefficients of the local quadratic surface;

[0147] Calculate the Gaussian curvature of the local quadratic surface based on the fitting coefficients;

[0148] Construct the curvature time series of the Gaussian curvature;

[0149] The mean curvature and standard deviation of the curvature time series are calculated to analyze the facial curvature fluctuation values ​​of the facial AI 3D model;

[0150] Identify the nasal wing node of the facial AI 3D model;

[0151] Calculate the displacement of the nose wing node to construct a displacement time series of the nose wing node;

[0152] The displacement time series is transformed into the frequency domain to obtain the displacement frequency domain signal;

[0153] The respiratory micro-motion frequency of the facial AI 3D model is determined based on the displacement frequency domain signal.

[0154] The local quadratic surface refers to a quadratic polynomial surface fitted to a point and its neighborhood points in a 3D facial point cloud. The fitting coefficients are parameters used to describe the geometry of this local surface. The Gaussian curvature is a geometric quantity describing the degree of curvature of the surface at a given point. The curvature time series is a one-dimensional sequence formed by calculating the Gaussian curvature of a specific region from 3D facial models acquired at different time points for the same target, and arranging them in chronological order. The average curvature is the arithmetic mean of all curvature values ​​in the curvature time series. The standard deviation of curvature refers to the degree of dispersion of the curvature values ​​relative to the average. The nasal alar nodes are key points located on both sides of the nasal alar in the 3D facial model. The displacement time series refers to the sequence formed by arranging the spatial position changes of the nasal alar nodes in multiple consecutive frames of the 3D model in chronological order. The displacement frequency domain signal is the signal obtained by converting the displacement time series to the frequency domain using a Fourier transform.

[0155] Optionally, the local quadratic surface of the facial AI 3D model can be fitted using the least squares method.

[0156] Optionally, the facial curvature fluctuation value of the facial AI 3D model can be calculated using deep learning, such as PointNet, PointNet, graph convolutional networks, etc.

[0157] Optionally, the displacement frequency domain signal can be obtained by Fourier transform, such as continuous-time Fourier transform, discrete-time Fourier transform, etc.

[0158] This invention, through analyzing the non-rigid deformation features of the target passenger, can determine whether the target is a real person by analyzing facial non-rigid deformations, effectively defending against deception methods such as photos, videos, and masks, and improving the accuracy of facial recognition. The non-rigid deformation features refer to the minute deformations of facial soft tissues during physiological activities such as breathing, facial expressions, and heartbeat, such as smiling, frowning, and tension.

[0159] Optionally, the non-rigid deformation characteristics of the target passenger can be analyzed by finite element analysis, such as modal analysis or nonlinear analysis.

[0160] This invention, by combining the non-rigid deformation features, the pulsation signal-to-noise ratio, and the temperature consistency coefficient, analyzes the data confidence level of facial image data to effectively identify photo and video attacks, reduce false rejection rates, and improve security check efficiency. The data confidence level refers to the degree of credibility of the data in reflecting true physiological state, identity authenticity, and information reliability, based on multi-dimensional features of facial image data (non-rigid deformation, pulsation signal, temperature distribution, etc.) through quantitative analysis and fusion evaluation.

[0161] As an embodiment of the present invention, the step of analyzing the data confidence of the facial image data by combining the non-rigid deformation characteristics, the pulsating signal-to-noise ratio, and the temperature consistency coefficient includes:

[0162] Construct the non-rigid feature vector of the non-rigid deformation feature;

[0163] The non-rigid feature vector, the signal-to-noise ratio of the pulsating signal, and the temperature uniformity coefficient are fused to obtain a fused feature matrix;

[0164] Calculate the weight matrix of the fused feature matrix;

[0165] The data confidence level of the facial image data is calculated based on the fused feature matrix and the weight matrix.

[0166] The non-rigid feature vector refers to a numerical representation used to describe the minute deformation features of facial soft tissue during physiological activities such as breathing, facial expressions, and heartbeat. The fused feature matrix is ​​a matrix formed by concatenating and combining the non-rigid feature vector, the pulse signal-to-noise ratio (SNR), and the temperature consistency coefficient. The weight matrix is ​​a vector whose dimensions match those of the fused feature matrix.

[0167] Optionally, the fused feature matrix can be obtained by feature concatenation.

[0168] Optionally, the weight matrix of the fused feature matrix can be calculated using regression coefficient methods, such as linear regression, ridge regression, LASSO, etc.

[0169] S4. When the confidence level of the data is greater than the preset confidence level threshold, calculate the contour matching degree and texture feature similarity of the facial depth image and the ID card face image corresponding to the ID card information, respectively, so as to analyze the face similarity between the facial depth image and the ID card face image.

[0170] This invention achieves near-zero false positive secure identity authentication based on high-confidence data by calculating the contour matching degree and texture feature similarity between the facial depth image and the corresponding ID document face image when the data confidence level is greater than a preset confidence threshold. This also meets the requirements for real-time performance and adaptability to complex scenarios. Specifically, the contour matching degree refers to a technical indicator that quantifies whether the facial depth image and the ID document face image belong to the same person by comparing their geometric consistency in facial contour structure. The texture feature similarity refers to a technical indicator that determines whether the real-time facial depth image and the ID document face image belong to the same person by comparing their similarity in surface texture details.

[0171] As an embodiment of the present invention, when the data confidence level is greater than a preset confidence level threshold, the contour matching degree and texture feature similarity of the facial depth image and the facial image of the document corresponding to the document information are calculated respectively, including:

[0172] Extract the actual face contour from the facial depth image and the document face contour from the document face image, respectively;

[0173] The actual face contour and the document face contour are normalized to obtain normalized actual face contour and normalized document face contour.

[0174] Calculate the set of contour point distances between the normalized actual face contour and the normalized ID card face contour;

[0175] Based on the set of contour point distances, determine the contour matching degree between the normalized actual face contour and the normalized ID card face contour;

[0176] Extract the actual texture features of the facial depth image and the document texture features of the document face image, respectively;

[0177] Based on the actual texture features and the document texture features, calculate the texture feature similarity between the facial depth image and the document face image.

[0178] The actual face contour refers to the outer edge structure of the face extracted from the facial depth image. The document face contour refers to the outer edge structure of the face extracted from a document face image (such as an ID card or passport photo). The normalized actual face contour refers to the contour that, after geometric transformation, is aligned with a standard template in terms of scale, rotation, and translation. The normalized document face contour refers to the contour that, after geometric transformation, is aligned with a standard template in terms of scale, rotation, and translation. The contour point distance set refers to the set of Euclidean distances (or other distance metrics) between corresponding points of the normalized actual face contour and the document face contour. The actual texture features refer to the surface texture information extracted from the facial depth image. The document texture features refer to the texture information extracted from the document face image.

[0179] Optionally, the actual face contour of the facial depth image and the face contour of the ID card image can be extracted using edge detection algorithms, such as Canny, Sobel, etc.

[0180] Optionally, the actual texture features of the facial depth image and the document texture features of the document face image can be obtained by texture extraction algorithms, such as local binary pattern, histogram of oriented gradients, etc.

[0181] Optionally, the similarity of texture features between the facial depth image and the ID card face image can be calculated using a similarity algorithm, such as cosine similarity, Euclidean distance, correlation coefficient, etc.

[0182] This invention, through analyzing the facial similarity between the facial depth image and the ID document face image, can effectively resist two-dimensional deception methods such as photos, videos, and masks, enhancing cross-modal matching robustness and thus improving the reliability of identity authentication and verification. Here, facial similarity refers to calculating the degree of similarity between two face images (e.g., a facial depth image and an ID document face image) in the feature space.

[0183] Optionally, the facial similarity between the facial depth image and the ID card face image can be analyzed using a dynamic weighting method.

[0184] S5. Based on the facial similarity, perform intelligent verification of the target passenger's identity and document.

[0185] The embodiments of the present invention can improve the security and accuracy of identity verification, and increase verification efficiency and automation level by performing intelligent identity verification of the target passenger based on the facial similarity.

[0186] Compared to the problems described in the background technology, the present invention proposes a human-identity verification method and system based on multimodal AI fusion. By acquiring real-time facial infrared and depth images of the target passenger and combining them with document information for multi-dimensional feature extraction and intelligent analysis, it achieves high-precision and robust identity verification, demonstrating significant technical benefits. First, by analyzing the temperature gradient and temperature fluctuation characteristics of the facial infrared images, the system can effectively calculate the temperature consistency coefficient and extract microvascular pulsation signals, thereby calculating the pulsation signal-to-noise ratio. This process not only achieves a quantitative assessment of body temperature distribution but also captures the state of human microcirculation, providing reliable physiological evidence for liveness detection and significantly improving anti-counterfeiting capabilities, resisting deception methods such as photos, videos, and masks. Second, by constructing an AI three-dimensional model based on facial depth images, the system can accurately calculate facial curvature fluctuation values ​​and respiratory micro-movement frequencies, thereby identifying non-rigid deformation features. These features reflect the subtle physiological changes of the target passenger in a natural state, and combined with the pulsation signal-to-noise ratio and temperature consistency coefficient, they jointly construct a multi-dimensional data confidence assessment system. The system only triggers subsequent face similarity calculations when the data confidence level exceeds a preset threshold, effectively reducing the false positive rate and improving the accuracy and reliability of verification. During the face similarity analysis phase, the system compares the facial depth image with the ID card face image from two dimensions: contour matching degree and texture feature similarity. This dual-modal comparison strategy not only considers the consistency of geometric structure but also takes into account the detailed matching of surface textures, further enhancing the comprehensiveness and accuracy of identity verification. Finally, based on the dual judgment of high confidence and high similarity, this system achieves intelligent identity verification, significantly improving the security and efficiency of identity authentication, and is suitable for various high-security scenarios such as airports, train stations, finance, and government affairs. Simultaneously, the system adopts a non-contact data collection method, optimizing the user experience and possessing good scalability and application prospects. Therefore, the intelligent identity verification method based on multimodal AI fusion provided in this embodiment of the invention can improve the accuracy and reliability of identity verification.

[0187] like Figure 2 The diagram shown is a functional module diagram of a human-identity verification intelligent verification system based on multimodal AI fusion according to the present invention.

[0188] The intelligent identity verification system 200 based on multimodal AI fusion described in this invention can be installed in an electronic device. Depending on the functions implemented, the intelligent identity verification system based on multimodal AI fusion may include a data acquisition module 201, a facial temperature analysis module 202, a data confidence analysis module 203, a facial similarity calculation module 204, and an identity verification module 205. The modules described in this invention can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0189] In this embodiment of the invention, the functions of each module / unit are as follows:

[0190] The data acquisition module 201 is used to collect facial image data and document information of the target passenger in real time, wherein the facial image data includes facial infrared images and facial depth images.

[0191] The facial temperature analysis module 202 allows the user to analyze the facial temperature gradient and temperature fluctuation characteristics of the target passenger based on the facial infrared image, calculate the temperature consistency coefficient of the facial infrared image based on the facial temperature gradient, analyze the microvascular pulsation signal of the target passenger based on the temperature fluctuation characteristics, and calculate the pulsation signal signal-to-noise ratio of the microvascular pulsation signal.

[0192] The data confidence analysis module 203 is used to fit the facial AI three-dimensional model of the target passenger based on the facial depth image, calculate the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI three-dimensional model, so as to analyze the non-rigid deformation characteristics of the target passenger, and combine the non-rigid deformation characteristics, the signal-to-noise ratio of the pulsation signal and the temperature consistency coefficient to analyze the data confidence of the facial image data.

[0193] The face similarity calculation module 204 is used to calculate the contour matching degree and texture feature similarity of the facial depth image and the ID card face image corresponding to the ID card information respectively when the data confidence degree is greater than the preset confidence degree threshold, so as to analyze the face similarity between the facial depth image and the ID card face image.

[0194] The identity verification module 205 is used to perform intelligent identity verification of the target passenger based on the facial similarity.

[0195] In detail, the modules in the multimodal AI fusion-based intelligent verification system 200 described in this embodiment of the invention employ the same methods as described above. Figure 1The method described herein is the same as the intelligent verification method for identity verification based on multimodal AI fusion, and can produce the same technical effect, so it will not be elaborated here.

[0196] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0197] Finally, it should be noted that in the above embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A human-identity verification method based on multimodal AI fusion, characterized in that, The method includes: Real-time collection of facial image data and identification information of target passengers, including facial infrared images and facial depth images; Based on the facial infrared image, analyze the facial temperature gradient and temperature fluctuation characteristics of the target passenger; based on the facial temperature gradient, calculate the temperature consistency coefficient of the facial infrared image; based on the temperature fluctuation characteristics, analyze the microvascular pulsation signal of the target passenger, and calculate the pulsation signal-to-noise ratio of the microvascular pulsation signal. Based on the facial depth image, a facial AI 3D model of the target passenger is fitted, and the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI 3D model are calculated to analyze the non-rigid deformation characteristics of the target passenger. Combining the non-rigid deformation characteristics, the signal-to-noise ratio of the pulsation signal, and the temperature consistency coefficient, the data confidence of the facial image data is analyzed. When the confidence level of the data is greater than the preset confidence level threshold, the contour matching degree and texture feature similarity of the facial depth image and the ID card face image corresponding to the ID card information are calculated respectively to analyze the face similarity between the facial depth image and the ID card face image. Based on the facial similarity, perform intelligent verification of the target passenger's identity and identification.

2. The intelligent verification method for identity verification based on multimodal AI fusion as described in claim 1, characterized in that, The step of analyzing the facial temperature gradient and temperature fluctuation characteristics of the target passenger based on the facial infrared image includes: The facial infrared image is denoised to obtain a denoised facial infrared image; Radiometric correction is performed on the denoised facial infrared image to obtain a corrected facial infrared image; Detect the facial region in the corrected facial infrared image and extract the key points of the facial region; Based on the key points, the face region is divided into multiple regions of interest; The pixel values ​​in the multi-interest regions are converted into temperature values ​​to calculate the average temperature of each region in the multi-interest regions; Based on the average temperature, the temperature difference between the multiple interest regions is calculated to determine the facial temperature gradient of the target passenger. The temperature values ​​are processed over time to obtain a temperature time series; The temperature time series is transformed in the frequency domain to obtain a frequency domain signal; Identify the dominant frequency of temperature fluctuation in the frequency domain signal and analyze the temperature fluctuation characteristics of the dominant frequency.

3. The intelligent verification method for identity verification based on multimodal AI fusion as described in claim 1, characterized in that, The step of calculating the temperature uniformity coefficient of the facial infrared image based on the facial temperature gradient includes: Based on the facial temperature gradient, a temperature gradient vector is constructed for each pixel in the facial infrared image; Based on the temperature gradient vector, calculate the temperature gradient intensity and direction of each pixel in the facial infrared image; Calculate the temperature standard deviation of all pixels in the facial infrared image; The temperature uniformity coefficient of the facial infrared image is calculated using the following formula based on the temperature gradient intensity, the temperature gradient direction, and the temperature standard deviation: in, Indicates the temperature uniformity coefficient. Represents the weights of regional gradient similarity. This represents the total number of regions of interest corresponding to a facial infrared image. Indicates the first in the facial infrared image Areas of interest Indicates the first Total number of pixels in each region of interest Indicates the first The first interest area Temperature gradient vector of 1 pixel, Indicates the first The first interest area Temperature gradient vector of 1 pixel, Indicates the first The first interest area Temperature gradient intensity per pixel Indicates the first The first interest area Temperature gradient intensity per pixel Indicates the weights for gradient differences across regions. Indicates the first in the facial infrared image Areas of interest Represents the median function. Indicates the first The set of all temperature gradient directions in a region of interest. Indicates the first The set of all temperature gradient directions in a region of interest. Represents pi (π). Indicates the weight of temperature standard deviation. This represents the standard deviation of temperature.

4. The intelligent verification method for identity verification based on multimodal AI fusion as described in claim 1, characterized in that, The step of analyzing the microvascular pulsation signal of the target passenger based on the temperature fluctuation characteristics includes: Construct the temperature feature vector of the temperature fluctuation characteristics; Based on the temperature feature vector, a temperature-pulse mapping algorithm is fitted to the target passenger. Based on the temperature feature vector, the microvascular pulsation signal of the target passenger is calculated using the temperature-pulsation mapping algorithm. Wherein, the temperature feature vector refers to a set of structured numerical features used to characterize temperature fluctuation patterns; the temperature-pulse mapping algorithm refers to a model for establishing a mathematical relationship between the temperature feature vector and the microvascular pulsation signal; the pulsation signal refers to a time-series signal that can characterize the microvascular pulsation state of the target passenger's face, calculated and reconstructed by the temperature-pulse mapping algorithm.

5. The intelligent verification method for identity verification based on multimodal AI fusion as described in claim 1, characterized in that, The step of fitting a facial AI 3D model of the target passenger based on the facial depth image includes: Determine the camera optical center and camera focal length of the camera corresponding to the facial depth image; The coordinate transformation matrix of the facial depth image is determined based on the camera optical center and the camera focal point. Based on the coordinate transformation matrix, the pixel coordinates corresponding to the facial depth image are converted into three-dimensional spatial coordinates; Based on the three-dimensional spatial coordinates, point cloud data of the target passenger is generated; Based on the point cloud data, a facial AI 3D model of the target passenger is fitted.

6. The intelligent verification method for identity verification based on multimodal AI fusion as described in claim 5, characterized in that, The calculation of the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI 3D model includes: Based on the point cloud data corresponding to the facial AI 3D model, fit a local quadratic surface of the facial AI 3D model; Calculate the fitting coefficients of the local quadratic surface; Calculate the Gaussian curvature of the local quadratic surface based on the fitting coefficients; Construct the curvature time series of the Gaussian curvature; The mean curvature and standard deviation of the curvature time series are calculated to analyze the facial curvature fluctuation values ​​of the facial AI 3D model; Identify the nasal wing node of the facial AI 3D model; Calculate the displacement of the nose wing node to construct a displacement time series of the nose wing node; The displacement time series is transformed into the frequency domain to obtain the displacement frequency domain signal; The respiratory micro-motion frequency of the facial AI 3D model is determined based on the displacement frequency domain signal.

7. The intelligent verification method for identity verification based on multimodal AI fusion as described in claim 1, characterized in that, The analysis of the data confidence of the facial image data, combining the non-rigid deformation characteristics, the signal-to-noise ratio of the pulsating signal, and the temperature consistency coefficient, includes: Construct the non-rigid feature vector of the non-rigid deformation feature; The non-rigid feature vector, the signal-to-noise ratio of the pulsating signal, and the temperature uniformity coefficient are fused to obtain a fused feature matrix; Calculate the weight matrix of the fused feature matrix; The data confidence level of the facial image data is calculated based on the fused feature matrix and the weight matrix.

8. The intelligent verification method for identity verification based on multimodal AI fusion as described in claim 1, characterized in that, When the data confidence level is greater than a preset confidence threshold, the contour matching degree and texture feature similarity of the facial depth image and the facial image corresponding to the document information are calculated respectively, including: Extract the actual face contour from the facial depth image and the document face contour from the document face image, respectively; The actual face contour and the document face contour are normalized to obtain normalized actual face contour and normalized document face contour. Calculate the set of contour point distances between the normalized actual face contour and the normalized ID card face contour; Based on the set of contour point distances, determine the contour matching degree between the normalized actual face contour and the normalized ID card face contour; Extract the actual texture features of the facial depth image and the document texture features of the document face image, respectively; Based on the actual texture features and the document texture features, calculate the texture feature similarity between the facial depth image and the document face image.

9. The intelligent verification method for identity verification based on multimodal AI fusion as described in claim 1, characterized in that, The real-time collection of facial image data and identification information of target passengers includes: Configure the target passenger with multiple sensors, a document reader, and a microprocessor; Define the communication protocol for the multi-sensor, the document reader, and the microprocessor; Based on the communication protocol, a network topology is constructed for the multi-sensor, the document reader, and the microprocessor. Based on the network topology, a multi-sensor network is formed that integrates the multi-sensor, the document reader, and the microprocessor. The target passenger's facial image data and identification information are collected using the multi-sensor network.

10. A human-identity verification intelligent system based on multimodal AI fusion, characterized in that, The system is used to execute the intelligent verification method for identity verification based on multimodal AI fusion as described in any one of claims 1-9, the system comprising: The data acquisition module is used to collect facial image data and document information of target passengers in real time. The facial image data includes facial infrared images and facial depth images. The facial temperature analysis module allows the user to analyze the facial temperature gradient and temperature fluctuation characteristics of the target passenger based on the facial infrared image, calculate the temperature consistency coefficient of the facial infrared image based on the facial temperature gradient, and analyze the microvascular pulsation signal of the target passenger based on the temperature fluctuation characteristics, and calculate the pulsation signal-to-noise ratio of the microvascular pulsation signal. The data confidence analysis module is used to fit the facial AI three-dimensional model of the target passenger based on the facial depth image, calculate the facial curvature fluctuation value and respiratory micro-motion frequency of the facial AI three-dimensional model, so as to analyze the non-rigid deformation characteristics of the target passenger, and combine the non-rigid deformation characteristics, the signal-to-noise ratio of the pulsation signal and the temperature consistency coefficient to analyze the data confidence of the facial image data. The face similarity calculation module is used to calculate the contour matching degree and texture feature similarity of the facial depth image and the ID card face image corresponding to the ID card information respectively when the confidence degree of the data is greater than the preset confidence degree threshold, so as to analyze the face similarity between the facial depth image and the ID card face image. The identity verification module is used to perform intelligent identity verification of the target passenger based on the facial similarity.

Images