Identity verification system based on visible light and infrared perception

Abstract

The invention relates to the technical field of biological feature recognition, in particular to an identity verification system based on visible light and infrared perception, which is characterized in that an image acquisition module acquires a visible light image sequence and an infrared image sequence of a target object, an environmental parameter acquisition module acquires environmental parameters, a data preprocessing module processes the received image sequence, and an identity verification module verifies the identity of the target object. The multi-modal feature extraction module extracts visible light texture features and infrared temperature features from a processed image sequence, the multi-modal feature fusion module combines environment parameters to perform fusion processing on the received features, and the living body identity verification module performs identity matching and living body verification on the received fusion features. And when both the identity matching result and the living body verification result meet conditions, outputting a verification passing signal. According to the method, high-precision and high-safety identity verification is realized through synchronous acquisition, space-time alignment, multi-modal adaptive feature fusion and living body temperature and texture dual verification.

Description

Technical Field

[0001] This invention relates to the field of biometric identification technology, and in particular to an identity verification system based on visible light and infrared sensing. Background Technology

[0002] The development of identity verification technology is closely linked to the security needs of the digital society. In the early days of the real-name registration system, users had to directly submit plaintext information such as their ID card number and name, leading to significant risks of personal information leakage and resulting in hundreds of billions of yuan in financial losses annually due to information exposure. To address this challenge, the government gradually built a network identity authentication system centered on network IDs and certificates, replacing traditional plaintext information with encrypted virtual credentials to achieve a verification model of front-end anonymity and back-end real-name authentication. Simultaneously, biometric technology has evolved rapidly. Facial recognition has become the mainstream solution due to its contactless and convenient nature, and liveness detection technology has evolved from early action command verification to multi-feature judgment combining micro-expressions and texture details. Some systems have begun to explore the integration of multimodal data such as visible light and infrared to improve accuracy. With the gradual improvement of relevant policies, identity verification has gradually moved towards standardization and security, forming a pattern of coordinated development between policy guidance and technological innovation.

[0003] Despite technological advancements, existing identity verification schemes still have significant limitations. In complex environments, single-modal feature extraction is susceptible to interference. For example, visible light images are affected by light intensity, leading to blurred texture features, while infrared temperature features suffer from temperature gradient distortion due to environmental radiation interference. Both can reduce the stability of feature extraction. The shortcomings of relying on single features in liveness detection become apparent. Verification based solely on facial movements or texture details is insufficient to defend against forgery attacks using high-precision photos or 3D-printed models. The misuse of AI face-swapping technology further exacerbates the risk of identity theft. Furthermore, feature fusion mechanisms often employ fixed weight allocation, failing to fully consider the dynamic impact of environmental parameters on the effectiveness of different modal features. This results in insufficient robustness of fused features, and in practical applications, the problem of fluctuating verification accuracy and security vulnerabilities persists. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned problems and provide an identity verification system based on visible light and infrared sensing. To achieve the above objective, this invention adopts the following technical solution:

[0005] An identity verification system based on visible light and infrared sensing includes an image acquisition module, an environmental parameter acquisition module, a data preprocessing module, a multimodal feature extraction module, a multimodal feature fusion module, and a liveness verification module. The image acquisition module acquires visible light and infrared image sequences of the target object and transmits them to the data preprocessing module. The environmental parameter acquisition module acquires environmental parameters and transmits them to the multimodal feature fusion module. The data preprocessing module processes the received image sequences and transmits them to the multimodal feature extraction module. The multimodal feature extraction module extracts visible light texture features and infrared temperature features from the processed image sequences and transmits them to the multimodal feature fusion module. The multimodal feature fusion module combines the received features with the environmental parameters to perform fusion processing and transmits the fusion to the liveness verification module. The liveness verification module performs identity matching and liveness verification on the received fused features, outputting a verification pass signal only if both the identity matching result and the liveness verification result meet set conditions.

[0006] Furthermore, the image acquisition module includes a visible light acquisition module, an infrared acquisition module, and a synchronization triggering module. The visible light acquisition module acquires the visible light image sequence of the target object, the infrared acquisition module acquires the infrared image sequence of the target object, and the synchronization triggering module triggers the acquisition start time of the visible light acquisition module and the infrared acquisition module through an electrical signal, so that the acquisition start deviation between the two does not exceed a set time interval. The set time interval does not exceed 10ms. When the acquisition start deviation exceeds the set time interval, the out-of-synchronization frame is repaired by linear interpolation.

[0007] Furthermore, the environmental parameter acquisition module includes a light intensity acquisition module and an infrared radiation intensity acquisition module; the light intensity acquisition module acquires the ambient light intensity, and the infrared radiation intensity acquisition module acquires the ambient infrared radiation intensity. The light intensity and infrared radiation intensity together constitute the environmental parameters.

[0008] Furthermore, the data preprocessing module includes a noise suppression module, a spatial registration module, and a timing synchronization module. The noise suppression module uses a median filtering algorithm to suppress salt-and-pepper noise in the visible light image sequence and a Gaussian filtering algorithm to suppress thermal noise in the infrared image sequence. The spatial registration module extracts key feature points in the facial image through a feature point detection algorithm, removes mismatched points using a random sampling consistency algorithm, and aligns the spatial coordinates of the visible light and infrared images based on the affine transformation matrix. The timing synchronization module performs frame alignment on the spatially registered image sequence by comparing the timestamps of the image acquisition time.

[0009] Furthermore, the multimodal feature extraction module includes a visible light texture feature extraction module and an infrared temperature feature extraction module. The visible light texture feature extraction module uses a deep convolutional network to extract visible light texture features from the visible light image sequence output by the data preprocessing module. The visible light texture features include the gradient direction histogram features of the facial contour, the local binary pattern features of eye details, and the Gabor filter response features of the nose texture. The infrared temperature feature extraction module uses a deep convolutional network to extract infrared temperature features from the infrared image sequence output by the data preprocessing module. The infrared temperature features include the first derivative features of the facial temperature gradient and the edge contour features of high-temperature areas.

[0010] Furthermore, the multimodal feature fusion module includes a spatial attention generation module, a channel attention generation module, and a dynamic weight allocation module. The spatial attention generation module generates visible light spatial attention weights based on the gradient magnitude of visible light texture features and infrared spatial attention weights based on the gradient magnitude of infrared temperature features. The channel attention generation module generates visible light channel attention weights based on the pixel value variance of each channel of the visible light texture features and infrared channel attention weights based on the temperature gradient variance of each channel of the infrared temperature features. The dynamic weight allocation module generates visible light dynamic weights and infrared dynamic weights by combining environmental parameters. After spatial registration and time-series synchronization processing by the data preprocessing module, the visible light texture feature map and the infrared temperature feature map have the same spatial resolution and number of channels. The multimodal feature fusion module calculates the fused features using the following formula:

[0011] F = (A v ⊙M v )·C v ·ω v +(A i ⊙M i )·C i ·ω i

[0012] Where F represents the fusion feature, and A v For visible light spatial attention weights, M v For visible light texture feature maps, C v For the attention weights of the visible light channel, ω v For visible light dynamic weights, A i For infrared spatial attention weights, M i For infrared temperature characteristic maps, C i For the attention weights of the infrared channel, ω i For infrared dynamic weights, ⊙ indicates element-wise multiplication, and · indicates channel-wise broadcast multiplication.

[0013] Furthermore, the dynamic weight allocation module calculates the visible light dynamic weight ω using the following formula. vand infrared dynamic weight ω i :

[0014]

[0015] Among them, E v E represents the information entropy of visible light texture features. i E represents the information entropy of infrared temperature characteristics. v M is a visible light texture feature map v Shannon entropy of pixel value distribution, M v The pixel values ​​are normalized to the [0, 1] interval, specifically E. v =-∑p v (x)logp v (x), where p v (x) is M v The probability distribution of pixel value x; E i Infrared temperature feature map M i The Shannon entropy of the temperature gradient distribution is specifically E. i =-∑p i (g)logp i (g), where p i (g) is M i The probability distribution of the temperature gradient value g; L is the light intensity in the environmental parameters, and N is the infrared radiation intensity in the environmental parameters. L and N are respectively processed by maximum-minimum normalization to the interval [0, 1].

[0016] Furthermore, the liveness verification module includes a feature association module, an identity matching module, and a liveness verification module. The feature association module associates the fused features output by the multimodal feature fusion module with the preset identity template features. The identity matching module performs identity matching by calculating the similarity between the associated features and the preset identity template features. The liveness verification module extracts skin temperature distribution features from infrared temperature features, which meet the normal human body temperature range of 35-37℃. It also extracts facial texture depth features from visible light texture features, which are inferred from the gradient changes and texture details of monocular images. The liveness verification is performed by combining the skin temperature distribution features and the facial texture depth features.

[0017] The advantages of this invention are:

[0018] 1. This invention achieves high-precision alignment of visible light and infrared image sequences in time and space through the synchronous triggering mechanism of the image acquisition module, linear interpolation time sequence repair technology, median filtering and Gaussian filtering noise suppression methods in the data preprocessing stage, and feature point detection spatial registration algorithm. This effectively improves the stability and accuracy of multimodal feature extraction and lays a high-quality data foundation for identity verification.

[0019] 2. This invention achieves adaptive fusion of visible light texture features and infrared temperature features by using spatial attention weights based on gradient magnitude, channel attention weights based on reference pixel values ​​or temperature gradient variance in the multimodal feature fusion module, and dynamic weight allocation combining normalized parameters of illumination and infrared radiation intensity. This enhances the robustness of features to complex environmental changes and significantly improves the accuracy and reliability of identity matching.

[0020] 3. This invention extracts skin temperature distribution features that meet the normal human body temperature range of 35-37℃ through a liveness verification module, and combines them with facial texture depth features inferred from monocular image gradient changes and texture details to effectively resist forgery attacks, while ensuring the accuracy of identity verification and the security of liveness detection. Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application.

[0022] In the attached diagram:

[0023] Figure 1 This is a system framework diagram of an identity verification system based on visible light and infrared sensing in Example 1.

[0024] Figure 2 This is a flowchart of an identity verification system based on visible light and infrared sensing in Example 1. Detailed Implementation

[0025] The present invention will now be described in detail and specifically through specific embodiments to enable a better understanding of the invention. However, the following embodiments do not limit the scope of protection of the present invention.

[0026] Example 1

[0027] like Figure 1-2As shown, an identity verification system based on visible light and infrared sensing includes an image acquisition module, an environmental parameter acquisition module, a data preprocessing module, a multimodal feature extraction module, a multimodal feature fusion module, and a liveness verification module. The image acquisition module acquires visible light and infrared image sequences of the target object and transmits them to the data preprocessing module. The environmental parameter acquisition module acquires environmental parameters and transmits them to the multimodal feature fusion module. The data preprocessing module processes the received image sequences and transmits them to the multimodal feature extraction module. The multimodal feature extraction module extracts visible light texture features and infrared temperature features from the processed image sequences and transmits them to the multimodal feature fusion module. The multimodal feature fusion module combines the received features with the environmental parameters to perform fusion processing and transmits them to the liveness verification module. The liveness verification module performs identity matching and liveness verification on the received fused features. It outputs a verification pass signal only when both the identity matching result and the liveness verification result meet the set conditions.

[0028] In a specific embodiment, the modules in the identity verification system are sequentially connected according to a process. The image acquisition module simultaneously acquires visible light image sequences and infrared image sequences for the target object and transmits these two types of sequences to the data preprocessing module. Simultaneously, the environmental parameter acquisition module collects environmental parameters and sends them directly to the multimodal feature fusion module. After processing the received image sequences, the data preprocessing module transmits the results to the multimodal feature extraction module, which extracts visible light texture features and infrared temperature features respectively, and then sends both features to the multimodal feature fusion module. The multimodal feature fusion module combines the previously acquired environmental parameters to perform fusion processing on the two features, and the fusion result is then transmitted to the liveness verification module. The liveness verification module performs identity matching and liveness verification on the fused features. Only when both results meet the system's set conditions will a verification pass signal be output. Simultaneously integrating visible light and infrared dual-modal information, along with environmental parameters and dual verification standards, can reduce misjudgments that may occur with a single modality or a single standard, improving the reliability of identity verification.

[0029] Furthermore, the image acquisition module includes a visible light acquisition module, an infrared acquisition module, and a synchronization triggering module. The visible light acquisition module acquires the visible light image sequence of the target object, the infrared acquisition module acquires the infrared image sequence of the target object, and the synchronization triggering module triggers the acquisition start time of the visible light acquisition module and the infrared acquisition module through an electrical signal, so that the acquisition start deviation between the two does not exceed a set time interval. The set time interval does not exceed 10ms. When the acquisition start deviation exceeds the set time interval, the out-of-synchronization frame is repaired by linear interpolation.

[0030] In a specific embodiment, the image acquisition module comprises three functional sub-modules. The visible light acquisition module specifically acquires visible light image sequences of the target object, the infrared acquisition module focuses on acquiring infrared image sequences of the target object, and the synchronization triggering module controls the acquisition start time of the first two modules via electrical signals. The system is configured to control the acquisition start deviation between the two modules within 10ms. If the deviation exceeds this range during actual acquisition, a linear interpolation method is used to perform temporal repair on the out-of-synchronization image frames. Through the coordination of synchronization triggering and out-of-synchronization repair, the acquisition time difference between the two image sequences can be minimized, ensuring the temporal consistency of the dual-modal data and preventing temporal deviations from interfering with feature extraction and fusion effects.

[0031] Furthermore, the environmental parameter acquisition module includes a light intensity acquisition module and an infrared radiation intensity acquisition module; the light intensity acquisition module acquires the ambient light intensity, and the infrared radiation intensity acquisition module acquires the ambient infrared radiation intensity. The light intensity and infrared radiation intensity together constitute the environmental parameters.

[0032] In a specific embodiment, the environmental parameter acquisition module consists of a light intensity acquisition module and an infrared radiation intensity acquisition module. The light intensity acquisition module is responsible for acquiring the light intensity of the current environment, while the infrared radiation intensity acquisition module acquires the infrared radiation intensity in the environment. These two parameters together constitute the environmental parameters required for system processing. Since ambient light affects the quality of visible light images, and ambient infrared radiation affects the quality of infrared images, acquiring these two key parameters provides environmental information for multimodal feature fusion, helping the fusion module adjust its strategy according to the actual environment and reducing the impact of environmental factors on fusion accuracy.

[0033] Furthermore, the data preprocessing module includes a noise suppression module, a spatial registration module, and a timing synchronization module. The noise suppression module uses a median filtering algorithm to suppress salt-and-pepper noise in the visible light image sequence and a Gaussian filtering algorithm to suppress thermal noise in the infrared image sequence. The spatial registration module extracts key feature points in the facial image through a feature point detection algorithm, removes mismatched points using a random sampling consistency algorithm, and aligns the spatial coordinates of the visible light and infrared images based on the affine transformation matrix. The timing synchronization module performs frame alignment on the spatially registered image sequence by comparing the timestamps of the image acquisition time.

[0034] In a specific embodiment, the data preprocessing module optimizes the image sequence step by step through three sub-modules. The noise suppression module employs differentiated processing for different noise types in different images: median filtering is used to suppress salt-and-pepper noise in visible light image sequences, and Gaussian filtering is used to suppress thermal noise in infrared image sequences. The spatial registration module first extracts key feature points from facial images using a feature point detection algorithm, then uses a random sampling consistency algorithm to remove mismatched points, and finally aligns the spatial coordinates of the visible light and infrared images based on an affine transformation matrix. The temporal synchronization module performs frame alignment on the spatially registered image sequence by comparing the timestamps of the image acquisition timestamps. This preprocessing significantly reduces noise interference and achieves dual spatial and temporal alignment of the images, providing a high-quality data foundation for multimodal feature extraction.

[0035] Furthermore, the multimodal feature extraction module includes a visible light texture feature extraction module and an infrared temperature feature extraction module. The visible light texture feature extraction module uses a deep convolutional network to extract visible light texture features from the visible light image sequence output by the data preprocessing module. The visible light texture features include the gradient direction histogram features of the facial contour, the local binary pattern features of eye details, and the Gabor filter response features of the nose texture. The infrared temperature feature extraction module uses a deep convolutional network to extract infrared temperature features from the infrared image sequence output by the data preprocessing module. The infrared temperature features include the first derivative features of the facial temperature gradient and the edge contour features of high-temperature areas.

[0036] In a specific embodiment, the multimodal feature extraction module extracts features for both image sequences separately. The visible light texture feature extraction module uses a deep convolutional network to extract features from the preprocessed visible light image sequence, specifically including the gradient orientation histogram features of facial contours, local binary pattern features of eye details, and Gabor filter response features of nose texture. The infrared temperature feature extraction module also uses a deep convolutional network to extract features from the preprocessed infrared image sequence, covering the first derivative features of facial temperature gradients and the edge contour features of high-temperature areas. By accurately capturing key features in both modes through deep convolutional networks, it can cover facial texture details and obtain temperature information, providing rich and effective feature support for feature fusion.

[0037] Furthermore, the multimodal feature fusion module includes a spatial attention generation module, a channel attention generation module, and a dynamic weight allocation module. The spatial attention generation module generates visible light spatial attention weights based on the gradient magnitude of visible light texture features and infrared spatial attention weights based on the gradient magnitude of infrared temperature features. The channel attention generation module generates visible light channel attention weights based on the pixel value variance of each channel of the visible light texture features and infrared channel attention weights based on the temperature gradient variance of each channel of the infrared temperature features. The dynamic weight allocation module generates visible light dynamic weights and infrared dynamic weights by combining environmental parameters. After spatial registration and time-series synchronization processing by the data preprocessing module, the visible light texture feature map and the infrared temperature feature map have the same spatial resolution and number of channels. The multimodal feature fusion module calculates the fused features using the following formula:

[0038] F = (A v ⊙M v )·C v ·ω v +(A i ⊙M i )·C i ·ω i

[0039] Where F represents the fusion feature, and A v For visible light spatial attention weights, M v For visible light texture feature maps, C v For the attention weights of the visible light channel, ω v For visible light dynamic weights, A i For infrared spatial attention weights, M i For infrared temperature characteristic maps, C i For the attention weights of the infrared channel, ω i For infrared dynamic weights, ⊙ indicates element-wise multiplication, and · indicates channel-wise broadcast multiplication.

[0040] In a specific embodiment, the multimodal feature fusion module achieves feature fusion through three sub-modules. The spatial attention generation module generates visible light spatial attention weights based on the gradient magnitude of visible light texture features, and simultaneously generates infrared spatial attention weights based on the gradient magnitude of infrared temperature features. The channel attention generation module generates visible light channel attention weights based on the pixel value variance of each channel of the visible light texture features, and generates infrared channel attention weights based on the temperature gradient variance of each channel of the infrared temperature features. The dynamic weight allocation module generates visible light dynamic weights and infrared dynamic weights in conjunction with environmental parameters. It is important to note that after spatial registration and temporal synchronization by the preprocessing module, the visible light texture feature map and the infrared temperature feature map have the same spatial resolution and number of channels, which provides a prerequisite for fusion calculation. This fusion method, combining attention mechanisms and dynamic weights, can adjust weights according to feature importance and environmental conditions, highlighting information valuable for verification and improving the fusion effect.

[0041] Furthermore, the dynamic weight allocation module calculates the visible light dynamic weight ω using the following formula. v and infrared dynamic weight ω i :

[0042]

[0043] Among them, E v E represents the information entropy of visible light texture features. i E represents the information entropy of infrared temperature characteristics. v M is a visible light texture feature map v Shannon entropy of pixel value distribution, M v The pixel values ​​are normalized to the [0, 1] interval, specifically E. v =-∑p v (x)logp v (x), where p v (x) is M v The probability distribution of pixel value x; E i Infrared temperature feature map M i The Shannon entropy of the temperature gradient distribution is specifically E. i =-∑p i (g)logp i (g), where p i (g) is M i The probability distribution of the temperature gradient value g; L is the light intensity in the environmental parameters, and N is the infrared radiation intensity in the environmental parameters. L and N are respectively processed by maximum-minimum normalization to the interval [0, 1].

[0044] In a specific embodiment, when calculating the two dynamic weights corresponding to visible light and infrared features, the dynamic weight allocation module first performs preprocessing operations on the parameters involved in the calculation. For example, when obtaining the pixel distribution probability of the visible light texture feature map, it counts the frequency of different pixel values ​​in the feature map and divides the frequency of each pixel value by the total number of pixels to obtain the corresponding probability. The temperature gradient distribution probability of the infrared temperature feature map is obtained by counting the proportion of the frequency of different temperature gradient values. In the environmental parameters, the light intensity and infrared radiation intensity are preprocessed by first determining the maximum and minimum values ​​of each during the acquisition period, and then adjusting them to the range of 0 to 1 by subtracting the minimum value from the actual acquired value and dividing by the difference between the maximum and minimum values.

[0045] In actual calculations, when ambient lighting conditions are good, resulting in richer effective information in visible light texture features, their corresponding dynamic weights will increase with the increase of the product of information entropy and light intensity. Conversely, if the ambient infrared radiation is more stable and the infrared temperature features can more clearly reflect the target characteristics, the dynamic weights corresponding to the infrared features will increase accordingly. This weight adjustment logic based on actual parameter changes allows for more precise emphasis on features with higher information quality during multimodal feature fusion, thereby providing a more reliable fusion basis for identity verification.

[0046] Furthermore, the liveness verification module includes a feature association module, an identity matching module, and a liveness verification module. The feature association module associates the fused features output by the multimodal feature fusion module with the preset identity template features. The identity matching module performs identity matching by calculating the similarity between the associated features and the preset identity template features. The liveness verification module extracts skin temperature distribution features from infrared temperature features, which meet the normal human body temperature range of 35-37℃. It also extracts facial texture depth features from visible light texture features, which are inferred from the gradient changes and texture details of monocular images. The liveness verification is performed by combining the skin temperature distribution features and the facial texture depth features.

[0047] In a specific embodiment, the liveness verification module completes the final verification through three sub-modules. The feature association module associates the fused features output by the multimodal feature fusion module with the system's preset identity template features. The identity matching module calculates the similarity between the associated features and the preset identity template features to complete identity matching. The liveness verification module extracts skin temperature distribution features from infrared temperature features (which must conform to the normal human body temperature range of 35-37℃) and facial texture depth features from visible light texture features (inferred from gradient changes and texture details in monocular images). Finally, these two features are combined to complete liveness verification. This verification method combines identity matching and liveness verification, and the liveness verification utilizes both human body temperature characteristics and facial texture depth information, effectively distinguishing between real people and forged identity carriers, thus improving the security and accuracy of the verification.

[0048] The specific embodiments of the present invention have been described in detail above, but they are merely examples, and the present invention is not equivalent to the specific embodiments described above. For those skilled in the art, any equivalent modifications and substitutions to the present invention are also within the scope of the present invention. Therefore, all equivalent transformations and modifications made without departing from the spirit and scope of the present invention should be covered within the scope of the present invention.

Claims

1. An identity verification system based on visible light and infrared sensing, characterized in that, The system includes an image acquisition module, an environmental parameter acquisition module, a data preprocessing module, a multimodal feature extraction module, a multimodal feature fusion module, and a liveness verification module. The image acquisition module acquires visible light and infrared image sequences of the target object and transmits them to the data preprocessing module. The environmental parameter acquisition module acquires environmental parameters and transmits them to the multimodal feature fusion module. The data preprocessing module processes the received image sequences and transmits them to the multimodal feature extraction module. The multimodal feature extraction module extracts visible light texture features and infrared temperature features from the processed image sequences and transmits them to the multimodal feature fusion module. The multimodal feature fusion module combines the received features with the environmental parameters and performs fusion processing, then transmits the fusion to the liveness verification module. The liveness verification module performs identity matching and liveness verification on the received fused features, outputting a verification pass signal only if both the identity matching result and the liveness verification result meet set conditions.

2. The identity verification system based on visible light and infrared sensing according to claim 1, characterized in that, The image acquisition module includes a visible light acquisition module, an infrared acquisition module, and a synchronization triggering module. The visible light acquisition module acquires a visible light image sequence of the target object, and the infrared acquisition module acquires an infrared image sequence of the target object. The synchronization triggering module triggers the acquisition start time of the visible light acquisition module and the infrared acquisition module through an electrical signal, ensuring that the acquisition start deviation between the two does not exceed a set time interval. The set time interval does not exceed 10ms. When the acquisition start deviation exceeds the set time interval, the out-of-synchronization frames are repaired by linear interpolation.

3. The identity verification system based on visible light and infrared sensing according to claim 2, characterized in that, The environmental parameter acquisition module includes a light intensity acquisition module and an infrared radiation intensity acquisition module; the light intensity acquisition module acquires the ambient light intensity, and the infrared radiation intensity acquisition module acquires the ambient infrared radiation intensity. The light intensity and infrared radiation intensity together constitute the environmental parameters.

4. The identity verification system based on visible light and infrared sensing according to claim 3, characterized in that, The data preprocessing module includes a noise suppression module, a spatial registration module, and a timing synchronization module. The noise suppression module uses a median filtering algorithm to suppress salt-and-pepper noise in the visible light image sequence and a Gaussian filtering algorithm to suppress thermal noise in the infrared image sequence. The spatial registration module extracts key feature points in the facial image through a feature point detection algorithm, removes mismatched points using a random sampling consistency algorithm, and aligns the spatial coordinates of the visible light and infrared images based on the affine transformation matrix. The timing synchronization module performs frame alignment on the spatially registered image sequence by comparing the timestamps of the image acquisition time.

5. The identity verification system based on visible light and infrared sensing according to claim 4, characterized in that, The multimodal feature extraction module includes a visible light texture feature extraction module and an infrared temperature feature extraction module. The visible light texture feature extraction module uses a deep convolutional network to extract visible light texture features from the visible light image sequence output by the data preprocessing module. The visible light texture features include the gradient direction histogram features of the facial contour, the local binary pattern features of eye details, and the Gabor filter response features of the nose texture. The infrared temperature feature extraction module uses a deep convolutional network to extract infrared temperature features from the infrared image sequence output by the data preprocessing module. The infrared temperature features include the first derivative features of the facial temperature gradient and the edge contour features of high-temperature areas.

6. The identity verification system based on visible light and infrared sensing according to claim 5, characterized in that, The multimodal feature fusion module includes a spatial attention generation module, a channel attention generation module, and a dynamic weight allocation module. The spatial attention generation module generates visible light spatial attention weights based on the gradient magnitude of visible light texture features and infrared spatial attention weights based on the gradient magnitude of infrared temperature features. The channel attention generation module generates visible light channel attention weights based on the pixel value variance of each channel of the visible light texture features and infrared channel attention weights based on the temperature gradient variance of each channel of the infrared temperature features. The dynamic weight allocation module generates visible light dynamic weights and infrared dynamic weights by combining environmental parameters. After spatial registration and time-series synchronization processing by the data preprocessing module, the visible light texture feature map and the infrared temperature feature map have the same spatial resolution and number of channels. The multimodal feature fusion module calculates the fused features using the following formula: F=(A v ⊙M v )·C v ·oh v +(A i ⊙M i )·C i ·oh i Where F represents the fusion feature, and A v For visible light spatial attention weights, M v For visible light texture feature maps, C v For the attention weights of the visible light channel, ω v For visible light dynamic weights, A i For infrared spatial attention weights, M i For infrared temperature characteristic maps, C i For the attention weights of the infrared channel, ω i For infrared dynamic weights, ⊙ indicates element-wise multiplication, and · indicates channel-wise broadcast multiplication.

7. The identity verification system based on visible light and infrared sensing according to claim 6, characterized in that, The dynamic weight allocation module calculates the visible light dynamic weight ω using the following formula. v and infrared dynamic weight ω i : Among them, E v E represents the information entropy of visible light texture features. i E represents the information entropy of infrared temperature characteristics. v M is a visible light texture feature map v The Shannon entropy of the pixel value distribution, M v The pixel values ​​are normalized to the [0, 1] interval, specifically E. v =-∑p v (x)logp v (x), where p v (x) is M v The probability distribution of pixel value x; E i Infrared temperature feature map M i The Shannon entropy of the temperature gradient distribution is specifically E. i =-∑p i (g)logp i (g), where p i (g) is M i The probability distribution of the temperature gradient value g; L is the light intensity in the environmental parameters, and N is the infrared radiation intensity in the environmental parameters. L and N are respectively processed by maximum-minimum normalization to the interval [0, 1].

8. The identity verification system based on visible light and infrared sensing according to claim 7, characterized in that, The liveness verification module includes a feature association module, an identity matching module, and a liveness verification module. The feature association module associates the fused features output by the multimodal feature fusion module with the preset identity template features. The identity matching module performs identity matching by calculating the similarity between the associated features and the preset identity template features. The liveness verification module extracts skin temperature distribution features from infrared temperature features, which meet the normal human body temperature range of 35-37℃. It also extracts facial texture depth features from visible light texture features, which are inferred based on gradient changes and texture details in monocular images. Liveness verification is performed by combining the skin temperature distribution features and the facial texture depth features.

Images