Cross-modal biological feature generation type enhancement method and system

By using a cross-modal biometric generation method, combined with pre-identification and fine identification by long-distance and short-distance data acquisition modules, and utilizing multiple modal features such as gait, voiceprint, face, and iris, the problem of performance fluctuation in single-modal recognition under environmental changes is solved, achieving high-precision and high-efficiency identity recognition.

CN121148028APending Publication Date: 2025-12-16HANGZHOU MINGGUANG MICROELECTRONICS TECH CO LTD

Patent Information

Application Number
CN202511686129.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-11-18
Publication Date
2025-12-16

AI Technical Summary

Technical Problem

Existing single-modal biometric identification methods exhibit significant performance fluctuations when faced with environmental changes, and multimodal identification systems fail to fully integrate different modal features, resulting in insufficient identification accuracy and stability.

Method used

A cross-modal biometric generation method is adopted, and pre-identification and fine identification are performed through long-distance and short-distance data acquisition modules respectively. By combining multiple modal features such as gait, voiceprint, face and iris, different fusion strategies and secondary identification mechanisms are used to improve the recognition accuracy and efficiency.

Benefits of technology

It achieves high-precision and high-efficiency identity recognition in complex environments, reduces the false judgment rate, and improves the robustness and recognition performance of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121148028A_ABST
    Figure CN121148028A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a cross-modal biological feature generation type enhancement method and system. The method comprises the following steps of: constructing a cross-modal biological feature recognition model, respectively acquiring long-distance modal data and short-distance modal data of a user, extracting feature vectors, pre-recognizing the user based on the long-distance feature vectors, finely recognizing the user based on the short-distance feature vectors, and carrying out mutual complementation and fusion on various modal features, so as to improve the recognition accuracy of the user. According to the method, the limitation that a single mode is low in recognition accuracy and prone to being interfered under different distance scenes is overcome, the identity recognition safety is improved, and through a secondary recognition mechanism combining pre-recognition and fine recognition, the efficiency and precision of identity recognition are improved while the system load is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a cross-modal biological feature generative enhancement method and system. BACKGROUND

[0002] With the increasing demand for security in society, identity recognition technology plays a crucial role in many fields such as security monitoring, access control systems, financial transaction authentication, etc. Traditional single biological feature recognition methods, such as relying solely on face recognition, fingerprint recognition, etc., gradually reveal many limitations in practical applications. Not only that, single modal usually has strong dependence on specific environmental conditions, and is also highly sensitive to external noise and interference factors. When the application environment or background conditions change (such as sudden changes in light, cluttered background or signal interference), the recognition performance of the model often fluctuates significantly. The core problem is that single modal systems cannot obtain supplementary information from other perception channels to correct biases or fill information gaps. It is these inherent defects that drive researchers to turn to multi-modal technology.

[0003] However, existing multi-modal recognition systems simply concatenate or simply weighted combine the recognition results of different modalities, without fully considering the respective feature advantages of different modalities and the deep fusion relationship between the features of different modalities, and lack effective strategies to deal with problems such as unbalanced training data, thereby affecting the overall user recognition accuracy and the stability and robustness of the system, making it difficult to meet the growing demand for high security and high precision identity recognition. SUMMARY

[0004] Therefore, the embodiments of the present application provide a cross-modal biological feature generative enhancement method and system, which fully combines the advantages of different biological feature modalities and overcomes the limitations of single modal in different distance scenarios, such as low recognition accuracy and susceptibility to interference.

[0005] In a first aspect, the embodiments of the present application provide a cross-modal biological feature generative enhancement method, comprising: Step 1: Construct a cross-modal biological feature recognition model, the cross-modal biological feature recognition model comprising a data acquisition layer, a feature extraction layer, an identity recognition layer and a decision output layer; wherein the data acquisition layer comprises a long-distance data acquisition module and a short-distance data acquisition module, the feature extraction layer comprises a long-distance feature extraction module and a short-distance feature extraction module, the identity recognition layer comprises a long-distance identity recognition module and a short-distance identity recognition module, and the decision output layer comprises an administrator module.

[0006] Step 2: When a user approaches the monitoring area, the long-distance data acquisition module collects the user's long-distance modal data, the long-distance feature extraction module extracts the long-distance modal features, and then the long-distance identity recognition module pre-identifies the user's identity.

[0007] Step 3: When a user enters the monitored area, the near-field data acquisition module collects the user's near-field modal data, the near-field feature extraction module extracts the near-field modal features, and then the near-field identity recognition module performs precise identification of the user's identity. Step 4: Determine the current user's identity based on the results of pre-identification and fine identification. If the current user is an authorized user, confirm the user's identity and grant the corresponding permissions to the current user. If the current user is a risky user, generate an alarm message to indicate that the current user does not have the necessary permissions, and send the current user's modal data and modal characteristics to the administrator module.

[0008] Optionally, in the above-described method embodiments of the present invention, the method further includes, The long-range data acquisition module includes a gait acquisition unit and a voiceprint acquisition unit, which are used to acquire the user's long-range modal data. The short-range data acquisition module includes a face acquisition unit and an iris acquisition unit, which are used to acquire the user's short-range modal data. The long-range feature extraction module includes a gait feature extraction unit, a voiceprint feature extraction unit, and a long-range feature fusion unit; The close-range feature extraction module includes a face feature extraction unit, an iris feature extraction unit, and a close-range feature fusion unit.

[0009] Optionally, in the above-described method embodiments of the present invention, the method further includes, The specific steps for the long-distance identity recognition module to pre-identify the user's identity are as follows: The long-distance identity recognition module calculates the similarity between the current user's long-distance feature vector and the long-distance feature vectors of all registered users in the database. If the similarity between the current user's long-distance feature vector and the long-distance feature vector of a registered user in the database is... satisfy When the registered user is identified as a candidate user, the process is repeated for all registered users to obtain a candidate user set. The current user is then identified as a trusted user. The similarity between the current user's long-distance feature vector and the long-distance feature vectors of all registered users in the database is then considered. None of them meet the requirements At that time, the current user is marked as a risk user, thus completing the pre-identification of the current user's identity.

[0010] Optionally, in the above-described method embodiments of the present invention, the method further includes, The specific steps for the near-field identity recognition module to perform precise identification of the user's identity are as follows: If the current user is a trusted user, the near-field identity recognition module calculates the similarity between the current user's near-field feature vector and the near-field feature vectors of all candidate users in the candidate user set. If the similarity between the current user's near-field feature vector and the near-field feature vector of a candidate user in the candidate user set is... satisfy When the similarity between the candidate user with the highest similarity is used as the identity information of the current user, the current user is marked as an authorized user. If the similarity between the current user's near-field feature vector and the near-field feature vectors of all candidate users in the candidate user set is... None of them meet the requirements At that time, the current user is marked as a high-risk user, thus completing a precise identification of the current user's identity.

[0011] Secondly, embodiments of the present invention provide a cross-modal biometric generative enhancement system, comprising: The model building module is used to build a cross-modal biometric recognition model, which includes a data acquisition layer, a feature extraction layer, an identity recognition layer, and a decision output layer. The data acquisition layer includes a long-distance data acquisition module and a short-distance data acquisition module; the feature extraction layer includes a long-distance feature extraction module and a short-distance feature extraction module; the identity recognition layer includes a long-distance identity recognition module and a short-distance identity recognition module; and the decision output layer includes an administrator module.

[0012] The pre-identification module is used to collect the user's long-distance modal data by the long-distance data acquisition module when the user approaches the monitoring area, extract the long-distance modal features by the long-distance feature extraction module, and then pre-identify the user's identity by the long-distance identity recognition module. The precision identification module is used to collect the user's close-range modal data by the close-range data acquisition module when the user enters the monitoring area, extract the close-range modal features by the close-range feature extraction module, and then perform precision identification of the user's identity by the close-range identity recognition module. The identity determination module is used to determine the identity of the current user based on the results of pre-identification and fine identification. If the current user is an authorized user, the user's identity is confirmed and the corresponding permissions are granted to the current user. If the current user is a risky user, an alarm message is generated to prompt that the current user does not have the necessary permissions, and the current user's modal data and modal features are sent to the administrator module.

[0013] Optionally, in the above-described system embodiments of the present invention, the system further includes, The long-range data acquisition module includes a gait acquisition unit and a voiceprint acquisition unit, which are used to acquire the user's long-range modal data. The short-range data acquisition module includes a face acquisition unit and an iris acquisition unit, which are used to acquire the user's short-range modal data. The long-range feature extraction module includes a gait feature extraction unit, a voiceprint feature extraction unit, and a long-range feature fusion unit; The close-range feature extraction module includes a face feature extraction unit, an iris feature extraction unit, and a close-range feature fusion unit.

[0014] Optionally, in the above-described system embodiments of the present invention, the system further includes, The specific steps for the long-distance identity recognition module to pre-identify the user's identity are as follows: The long-distance identity recognition module calculates the similarity between the current user's long-distance feature vector and the long-distance feature vectors of all registered users in the database. If the similarity between the current user's long-distance feature vector and the long-distance feature vector of a registered user in the database is... satisfy When the registered user is identified as a candidate user, the process is repeated for all registered users to obtain a candidate user set. The current user is then identified as a trusted user. The similarity between the current user's long-distance feature vector and the long-distance feature vectors of all registered users in the database is then considered. None of them meet the requirements At that time, the current user is marked as a risk user, thus completing the pre-identification of the current user's identity.

[0015] Optionally, in the above-described system embodiments of the present invention, the system further includes, The specific steps for the near-field identity recognition module to perform precise identification of the user's identity are as follows: If the current user is a trusted user, the near-field identity recognition module calculates the similarity between the current user's near-field feature vector and the near-field feature vectors of all candidate users in the candidate user set. If the similarity between the current user's near-field feature vector and the near-field feature vector of a candidate user in the candidate user set is... satisfy When the similarity between the candidate user with the highest similarity is used as the identity information of the current user, the current user is marked as an authorized user. If the similarity between the current user's near-field feature vector and the near-field feature vectors of all candidate users in the candidate user set is... None of them meet the requirements At that time, the current user is marked as a high-risk user, thus completing a precise identification of the current user's identity.

[0016] Thirdly, embodiments of the present invention provide a cross-modal biometric generative enhancement device, the device comprising: a memory and a processor, the memory and the processor being coupled; the memory storing program instructions, which, when executed by the processor, cause the device to perform the aforementioned cross-modal biometric generative enhancement method.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium, characterized in that it includes a computer program, which, when run on an electronic device, causes the electronic device to perform the above-described cross-modal biometric generative enhancement method.

[0018] Existing technologies often rely on single-dimensional gait features, such as stride length and cadence, for identification. This invention, however, not only encompasses these conventional features but also emphasizes extracting more detailed and in-depth features, such as leg swing amplitude. It comprehensively depicts gait information from multiple dimensions, enabling more accurate capture of individual unique walking patterns. This significantly enriches the feature set available for identification, thereby improving the accuracy and discriminative power of the identification.

[0019] This invention employs different fusion strategies to splice together different modal features, comprehensively considering the correlation and complementarity between various features, so that the final feature vector can completely and efficiently carry the multimodal information of an individual, further improving the overall recognition performance.

[0020] This invention employs a two-stage recognition mechanism combining pre-identification and fine-grained identification. In the pre-identification stage, the system only needs to process a portion of modal data with a relatively small amount of data and low computational complexity. Through simple and rapid feature extraction and preliminary matching, a portion of the target objects with higher probability are quickly screened out. In this way, the system avoids being burdened with the heavy task of processing all the data at the beginning, effectively distributing the computational pressure. In the fine-grained identification stage, the candidate target objects can be strictly screened to accurately identify the real target objects. This allows the system to maintain a high recognition accuracy while efficiently processing a large number of recognition tasks, achieving a good balance between recognition efficiency and recognition accuracy. This is an advantage that traditional single-stage recognition mechanisms find difficult to achieve simultaneously. Attached Figure Description

[0021] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a flowchart of a cross-modal biometric generative enhancement method provided by an exemplary embodiment of the present invention.

[0022] Figure 2This is a structural diagram of a cross-modal biometric generative enhancement system provided by an exemplary embodiment of the present invention. Detailed Implementation

[0023] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0024] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0025] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0026] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0027] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0028] Example 1 like Figure 1 As shown, this embodiment provides a cross-modal biometric generative enhancement method, including the following steps: Step 1: Construct a cross-modal biometric recognition model, which includes a data acquisition layer, a feature extraction layer, an identity recognition layer, and a decision output layer. The data acquisition layer includes a long-distance data acquisition module and a short-distance data acquisition module; the feature extraction layer includes a long-distance feature extraction module and a short-distance feature extraction module; the identity recognition layer includes a long-distance identity recognition module and a short-distance identity recognition module; and the decision output layer includes an administrator module.

[0029] The data acquisition layer is used to collect users' biometric modal data, the feature extraction layer is used to extract users' modal features based on the biometric modal data, the identity recognition layer is used to match users' modal features with registered users stored in the database to determine whether the user to be verified is a registered user, and the decision output layer makes decision outputs based on the recognition results of the identity recognition layer.

[0030] Step 2: When a user approaches the monitoring area, the long-distance data acquisition module collects the user's long-distance modal data, the long-distance feature extraction module extracts the long-distance modal features, and then the long-distance identity recognition module pre-identifies the user's identity.

[0031] Traditional multimodal recognition requires the simultaneous acquisition and processing of all biometric data, resulting in a massive instantaneous computational load and response delay. The proposed solution employs a two-stage recognition mechanism. When a user approaches the monitored area, for example, when the user is 50 meters away, long-range modal data is first collected for pre-identification. Then, when the user enters the monitored area, short-range modal data is collected for precise identification. This separates the acquisition, processing, and feature extraction of multiple modal data, improving recognition efficiency. Furthermore, the two-stage user identification using pre-identification and precise identification enhances accuracy and reduces the likelihood of false positives. Even if the data for one modality is of poor quality due to environmental interference (such as lighting or obstruction), data from another modality can provide crucial supplementary verification, significantly reducing the false positive rate caused by deficiencies in a single information source and achieving a higher level of accurate identification. Here, the monitored area can be considered as an area accessible only to authorized users, or an area where only authorized users can operate devices.

[0032] There are many types of biometric modal data for users, such as physiological feature data such as fingerprint data, facial data, iris data, retinal data, and voiceprint data, as well as behavioral feature data such as gait data and gesture data. Based on the distance requirements for acquiring various modal data, modal data that can be acquired at a distance, such as gait data, gesture data, and voiceprint data, can be set as long-distance modal data, while modal data that needs to be acquired at close range, such as facial data, fingerprint data, and iris data, can be set as short-distance modal data.

[0033] For example, this application selects gait data and voiceprint data as long-range modal data, and selects face data and iris data as short-range modal data. Accordingly, the long-range data acquisition module includes a gait acquisition unit and a voiceprint acquisition unit for acquiring the user's long-range modal data, and the short-range data acquisition module includes a face acquisition unit and an iris acquisition unit for acquiring the user's short-range modal data.

[0034] The gait acquisition unit consists of multiple high-definition cameras distributed in different locations within the monitoring area. The coverage area can typically reach a distance of 10-20 meters in radius. The cameras acquire video image data of people walking at a frame rate of about 20 frames per second to ensure that the human movement posture can be completely captured for subsequent gait feature extraction.

[0035] The voiceprint acquisition unit includes multiple high-sensitivity microphone arrays with noise reduction capabilities, evenly distributed at key locations in the monitoring area. It can use technologies such as beamforming to enhance the pickup of target sounds and effectively suppress environmental noise, collecting sound signals emitted by personnel from all directions for subsequent voiceprint feature extraction.

[0036] The long-range feature extraction module includes a gait feature extraction unit, a voiceprint feature extraction unit, and a long-range feature fusion unit. The gait feature extraction unit receives video data from the gait data acquisition unit, preprocesses the video data (e.g., adjusting brightness and contrast), extracts keyframes at fixed intervals (e.g., one frame every two frames to reduce redundant data), locates key joints of the human body using a human pose estimation algorithm, tracks its movement trajectory, and extracts stride length, stride frequency, and leg swing amplitude. These features are then quantified and normalized to form a gait feature vector.

[0037] When locating key joints in the human body, the following leg joint coordinates can be obtained: Hip joint: ,

[0038] Knee joint: ,

[0039] Ankle joint: ,

[0040] When calculating stride S, it can be estimated by the change in the coordinates of the ankle joint of the same foot in two adjacent frames. For example, the coordinates of the right ankle in frame t and frame t+1 are respectively... and .

[0041] The formula for calculating stride S can be approximated as:

[0042] The step frequency F can be determined by counting the number of steps per unit time. Assuming that N steps are detected within T frames and the video frame rate is f, the step frequency calculation formula is: .

[0043] Next, calculate the leg swing amplitude: Lateral swing amplitude (left and right direction): Using the body's center of mass (or the midpoint of the hip joint) as a reference point, calculate the maximum displacement difference between the left and right ankle joints in the horizontal direction (x-axis).

[0044] Coordinates of the midpoint of the hip joint:

[0045] Lateral displacement of the left ankle joint relative to the center of mass in a given frame: , Traverse the video sequence and record the maximum lateral displacement of the left / right ankle joints: and , The lateral swing amplitude is: .

[0046] Longitudinal swing amplitude (forward and backward direction): Calculate the range of displacement of the knee joint relative to the hip joint in the vertical direction (y-axis).

[0047] Longitudinal displacement of the left knee joint relative to the left hip joint in a given frame: , Traverse the video sequence and record the maximum longitudinal displacement of the left / right knee joint: and , The longitudinal swing amplitude is: .

[0048] The above gait features are normalized to form a gait feature vector. .

[0049] After receiving audio data from the voiceprint data acquisition unit, the voiceprint feature extraction unit processes the acquired audio signal. (where n represents the audio sample point number), using a speech endpoint detection algorithm based on features such as energy and zero-crossing rate, the start and end positions of the speech segment are determined, and the valid speech signal is extracted. Then, it is filtered using a Butterworth bandpass filter to obtain the filtered audio signal. Finally, the Mel frequency cepstral coefficients (MFCCs) are extracted as the voiceprint feature vector. .

[0050] Finally, the long-range feature fusion unit performs feature fusion using a principal component analysis (PCA)-based method, assuming the gait feature vector... The dimension is Voiceprint feature vector The dimension is Combine them into a matrix , dimension (m is the number of samples, assumed here to be 1 sample, i.e., the sample corresponding to the currently collected data). First, calculate the covariance matrix. Then solve for the eigenvalues ​​of the covariance matrix. and the corresponding feature vector Select the top k eigenvectors by sorting them from largest to smallest eigenvalue. The projection matrix P is formed. The fused long-range feature vectors are then used. Calculated using the following formula: .

[0051] Both the long-range and short-range identity recognition modules are connected to the user database, which stores registered users and their corresponding long-range and short-range feature vectors. The long-range and short-range identity recognition modules determine whether the current user is a registered user by calculating the similarity between the feature vectors of the current user and each registered user in the user database, thereby realizing the identity recognition of the current user.

[0052] The specific steps for the long-distance identity recognition module to pre-identify the user's identity are as follows: The long-distance identity recognition module calculates the similarity between the current user's long-distance feature vector and the long-distance feature vectors of all registered users in the database. If the similarity between the current user's long-distance feature vector and the long-distance feature vector of a registered user in the database is... satisfy When the registered user is marked as a candidate user, after traversing all registered users, a candidate user set is obtained, and the current user is marked as a trusted user; if the similarity between the current user's long-distance feature vector and the long-distance feature vectors of all registered users in the database is... None of them meet the requirements At that time, the current user is marked as a risk user, thus completing the pre-identification of the current user's identity. The preset threshold can be flexibly adjusted according to the actual situation.

[0053] Step 3: When a user enters the monitored area, the near-field data acquisition module collects the user's near-field modal data, the near-field feature extraction module extracts the near-field modal features, and then the near-field identity recognition module performs precise identification of the user's identity.

[0054] The close-range data acquisition module includes a face acquisition unit and an iris acquisition unit. The face acquisition unit uses a high-resolution camera with autofocus and is equipped with appropriate lighting to ensure uniform and sufficient illumination. When a person walks to a distance of approximately 0.1-0.3 meters from the face acquisition unit, it can quickly and clearly acquire facial images from different angles and with varying expressions for subsequent facial feature extraction. The iris acquisition unit uses specialized iris image acquisition equipment, equipped with a near-infrared light source and a high-definition imaging sensor. It features automatic focus adjustment, pupil positioning, and anti-glare functions, acquiring high-quality iris images within a 0.1-0.3-meter range from the person's eyes, accurately presenting key details such as iris texture for subsequent iris feature extraction.

[0055] The close-range feature extraction module includes a face feature extraction unit, an iris feature extraction unit, and a close-range feature fusion unit.

[0056] The face feature extraction unit uses a face feature extraction algorithm based on convolutional neural networks (such as ResNet). Let the input face image be... After multiple convolutions, pooling, and activation functions, the face feature vector is output at the fully connected layer of the network. Taking ResNet-50 as an example, its residual block structure contains skip connections, and the calculation formula (simplified illustration) is as follows: Where x is the feature input to the residual block, y is the output feature, and F is the residual function. These are the corresponding weight parameters. By continuously stacking such residual blocks and subsequent fully connected layer operations, the final face feature vector is obtained.

[0057] The iris feature extraction unit employs a Gabor filter-based feature extraction method. Let the iris image be... The functional expression for the Gabor filter is:

[0058] in, These are image coordinates. It is the direction of the filter. It's the wavelength. It is the standard deviation. It is the aspect ratio of the space. , By convolving Gabor filters of different directions and scales with the iris image, multi-channel texture feature descriptions are obtained. These features are then combined and normalized to obtain the iris feature vector. .

[0059] The near-field feature fusion unit adopts a weighted fusion method, assuming that the iris feature vector is assigned a weighted fusion method. The weight is Assigning facial feature vectors The weight is For example, in high-security scenarios, , The fused feature vector Calculated using the following formula:

[0060] The specific steps for the near-field identity recognition module to perform precise identification of the user's identity are as follows: If the current user is a trusted user, the near-field identity recognition module calculates the similarity between the current user's near-field feature vector and the near-field feature vectors of all candidate users in the candidate user set. If the current user's near-field feature vector is similar to the far-field feature vector of a candidate user in the candidate user set, the similarity is calculated accordingly. satisfy When the similarity between the candidate user with the highest similarity is used as the identity information of the current user, the current user is marked as an authorized user. If the similarity between the current user's near-field feature vector and the near-field feature vectors of all candidate users in the candidate user set is... None of them meet the requirements At that time, the current user is marked as a high-risk user, thus completing a precise identification of the current user's identity. The preset threshold can be flexibly adjusted according to the actual situation.

[0061] Step 4: Determine the current user's identity based on the results of pre-identification and fine identification. If the current user is an authorized user, confirm the user's identity and grant the corresponding permissions to the current user. If the current user is a risky user, generate an alarm message to indicate that the current user does not have the necessary permissions, and send the current user's modal data and modal characteristics to the administrator module.

[0062] Based on a secondary identification mechanism that combines pre-identification and fine-grained identification, the identity of the current user can be accurately determined. The decision output layer then outputs corresponding decisions based on the user's identity information. If the current user is an authorized user, the corresponding permissions can be granted, such as opening access control or allowing the user to operate devices. If the current user is a risky user, the system can remind the user that they do not have the necessary operating or access permissions, and send relevant information about the user to the administrator module so that the administrator can take appropriate measures to ensure security.

[0063] While long-range pre-identification has limited accuracy, it can quickly filter out a small number of candidate users from a massive database, greatly narrowing the scope of subsequent identification. Once the user approaches, the system then activates the close-range acquisition module for precise identification. At this point, the system only needs to perform high-precision comparisons within a very small candidate set, ensuring the reliability of the final identification result. This fusion of multimodal biometrics aims to overcome the limitations of a single modality, improving overall identification accuracy through information complementarity.

[0064] Example 2 like Figure 2 As shown, this embodiment provides a cross-modal biometric generative enhancement system for implementing the above-mentioned cross-modal biometric generative enhancement method, including the following modules: The model building module is used to build a cross-modal biometric recognition model, which includes a data acquisition layer, a feature extraction layer, an identity recognition layer, and a decision output layer. The data acquisition layer includes a long-distance data acquisition module and a short-distance data acquisition module; the feature extraction layer includes a long-distance feature extraction module and a short-distance feature extraction module; the identity recognition layer includes a long-distance identity recognition module and a short-distance identity recognition module; and the decision output layer includes an administrator module.

[0065] The pre-identification module is used to collect the user's long-distance modal data by the long-distance data acquisition module when the user approaches the monitoring area, extract the long-distance modal features by the long-distance feature extraction module, and then pre-identify the user's identity by the long-distance identity recognition module. The precision identification module is used to collect the user's close-range modal data by the close-range data acquisition module when the user enters the monitoring area, extract the close-range modal features by the close-range feature extraction module, and then perform precision identification of the user's identity by the close-range identity recognition module. The identity determination module is used to determine the identity of the current user based on the results of pre-identification and fine identification. If the current user is an authorized user, the user's identity is confirmed and the corresponding permissions are granted to the current user. If the current user is a risky user, an alarm message is generated to prompt that the current user does not have the necessary permissions, and the current user's modal data and modal features are sent to the administrator module.

[0066] Example 3 This application also provides a cross-modal biometric generative enhancement device, including a memory and a processor, wherein the memory and the processor are coupled; the memory stores program instructions, which, when executed by the processor, cause the device to perform the above-described cross-modal biometric generative enhancement method.

[0067] Example 4 This application also provides a computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform the above-described cross-modal biometric generative enhancement method.

[0068] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A cross-modal biometric generative enhancement method, characterized in that, include: Step 1: Construct a cross-modal biometric recognition model, which includes a data acquisition layer, a feature extraction layer, an identity recognition layer, and a decision output layer; wherein, the data acquisition layer includes a long-range data acquisition module and a short-range data acquisition module, the feature extraction layer includes a long-range feature extraction module and a short-range feature extraction module, the identity recognition layer includes a long-range identity recognition module and a short-range identity recognition module, and the decision output layer includes an administrator module; Step 2: When a user approaches the monitoring area, the long-distance data acquisition module collects the user's long-distance modal data, the long-distance feature extraction module extracts the long-distance modal features, and then the long-distance identity recognition module pre-identifies the user's identity. Step 3: When a user enters the monitoring area, the near-field data acquisition module collects the user's near-field modal data, the near-field feature extraction module extracts the near-field modal features, and then the near-field identity recognition module performs precise identification of the user's identity; Step 4: Determine the current user's identity based on the results of the pre-identification and the fine identification. If the current user is an authorized user, confirm the user's identity and grant the corresponding permissions to the current user. If the current user is a risky user, generate an alarm message to indicate that the current user does not have the permissions, and send the current user's modal data and modal features to the administrator module.

2. The cross-modal biometric generative enhancement method as described in claim 1, characterized in that, The long-range data acquisition module includes a gait acquisition unit and a voiceprint acquisition unit, used to acquire the user's long-range modal data; the near-range data acquisition module includes a face acquisition unit and an iris acquisition unit, used to acquire the user's near-range modal data. The long-distance feature extraction module includes a gait feature extraction unit, a voiceprint feature extraction unit, and a long-distance feature fusion unit; The near-field feature extraction module includes a face feature extraction unit, an iris feature extraction unit, and a near-field feature fusion unit.

3. The cross-modal biometric generative enhancement method as described in claim 2, characterized in that, The specific steps for the long-distance identity recognition module to pre-identify the user's identity are as follows: The long-distance identity recognition module calculates the similarity between the current user's long-distance feature vector and the long-distance feature vectors of all registered users in the database. If the similarity between the current user's long-distance feature vector and the long-distance feature vector of a registered user in the database is... satisfy When the registered user is marked as a candidate user, after traversing all registered users, a candidate user set is obtained, and the current user is marked as a trusted user; if the similarity between the long-distance feature vector of the current user and the long-distance feature vectors of all registered users in the database is... None of them meet the requirements When the current user is identified as a risky user, the pre-identification of the current user's identity is completed. This is a preset threshold.

4. The cross-modal biometric generative enhancement method as described in claim 3, characterized in that, The specific steps for the near-field identity recognition module to perform precise identity verification on the user are as follows: If the current user is a trusted user, the near-field identity recognition module calculates the similarity between the current user's near-field feature vector and the near-field feature vectors of all candidate users in the candidate user set. If the similarity between the current user's near-field feature vector and the near-field feature vector of a candidate user in the candidate user set is... satisfy When the similarity between the candidate user with the highest similarity is used as the identity information of the current user, and the current user is marked as an authorized user, if the similarity between the current user's proximity feature vector and the proximity feature vectors of all candidate users in the candidate user set is high, then the identity information of the candidate user with the highest similarity is used as the identity information of the current user. None of them meet the requirements At that time, the current user is marked as a risk user, completing a precise identification of the current user's identity, wherein the... This is a preset threshold.

5. A cross-modal biometric generative enhancement system, characterized in that, include: The model building module constructs a cross-modal biometric recognition model, which includes a data acquisition layer, a feature extraction layer, an identity recognition layer, and a decision output layer. The data acquisition layer includes a long-range data acquisition module and a short-range data acquisition module; the feature extraction layer includes a long-range feature extraction module and a short-range feature extraction module; the identity recognition layer includes a long-range identity recognition module and a short-range identity recognition module; and the decision output layer includes an administrator module. The pre-identification module is used to collect the user's long-distance modal data by the long-distance data acquisition module when the user approaches the monitoring area, extract long-distance modal features by the long-distance feature extraction module, and then pre-identify the user's identity by the long-distance identity recognition module. The precise identification module is used to collect the user's near-field modal data by the near-field data acquisition module when the user enters the monitoring area, extract the near-field modal features by the near-field feature extraction module, and then perform precise identification of the user's identity by the near-field identity recognition module. The identity determination module is used to determine the identity of the current user based on the results of the pre-identification and the fine identification. If the current user is an authorized user, the user's identity is confirmed and the corresponding permissions are granted to the current user. If the current user is a risky user, an alarm message is generated to indicate that the current user does not have the permissions, and the modal data and modal features of the current user are sent to the administrator module.

6. The cross-modal biometric generative enhancement system as described in claim 5, characterized in that, The long-range data acquisition module includes a gait acquisition unit and a voiceprint acquisition unit, used to acquire the user's long-range modal data; the near-range data acquisition module includes a face acquisition unit and an iris acquisition unit, used to acquire the user's near-range modal data. The long-distance feature extraction module includes a gait feature extraction unit, a voiceprint feature extraction unit, and a long-distance feature fusion unit; The near-field feature extraction module includes a face feature extraction unit, an iris feature extraction unit, and a near-field feature fusion unit.

7. The cross-modal biometric generative enhancement system as described in claim 6, characterized in that, The specific steps for the long-distance identity recognition module to pre-identify the user's identity are as follows: The long-distance identity recognition module calculates the similarity between the current user's long-distance feature vector and the long-distance feature vectors of all registered users in the database. If the similarity between the current user's long-distance feature vector and the long-distance feature vector of a registered user in the database is... satisfy When the registered user is marked as a candidate user, after traversing all registered users, a candidate user set is obtained, and the current user is marked as a trusted user; if the similarity between the long-distance feature vector of the current user and the long-distance feature vectors of all registered users in the database is... None of them meet the requirements When the current user is identified as a risky user, the pre-identification of the current user's identity is completed. This is a preset threshold.

8. The cross-modal biometric generative enhancement system as described in claim 7, characterized in that, The specific steps for the near-field identity recognition module to perform precise identity verification on the user are as follows: If the current user is a trusted user, the near-field identity recognition module calculates the similarity between the current user's near-field feature vector and the near-field feature vectors of all candidate users in the candidate user set. If the similarity between the current user's near-field feature vector and the near-field feature vector of a candidate user in the candidate user set is... satisfy When the similarity between the candidate user with the highest similarity is used as the identity information of the current user, and the current user is marked as an authorized user, if the similarity between the current user's proximity feature vector and the proximity feature vectors of all candidate users in the candidate user set is high, then the identity information of the candidate user with the highest similarity is used as the identity information of the current user. None of them meet the requirements At that time, the current user is marked as a risk user, completing a precise identification of the current user's identity, wherein the... This is a preset threshold.

9. A cross-modal biometric generative enhancement device, the device comprising: A memory and a processor, wherein the memory and the processor are coupled; The memory stores program instructions that, when executed by the processor, cause the device to perform the cross-modal biometric generative enhancement method as described in any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, The method includes a computer program that, when run on an electronic device, causes the electronic device to perform the cross-modal biometric generative enhancement method as described in any one of claims 1-5.

Citation Information

Patent Citations

  • Identification method based on multi-mode non-contact identity characteristics

    CN105825176A

  • Branch user identity identification method and device

    CN115578101A

  • Multi-modal biological feature fusion personnel identity recognition method and platform based on deep learning

    CN118247818A

  • Identity authentication system and method based on multi-mode biological feature recognition

    CN120850269A

Cited By

  • Object identification method and system based on multi-modal biological characteristic dynamic weight

    CN121330782A

  • Security check method and system based on voice and smell fusion data recognition

    CN121598267A