An auditory perception authentication method and device based on selective attention mechanism
By utilizing auditory perception-motor response through selective attention mechanisms on smart head-mounted devices, static statistical and dynamic temporal features are constructed. Combined with Siamese neural networks for identity authentication, the problems of interaction interruption and privacy risks in existing technologies are solved, achieving low-cost and high-security identity authentication.
Patent Information
- Application Number
- CN202611196457.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-07
- Publication Date
- 2026-11-17
AI Technical Summary
Existing identity authentication methods on smart head-mounted wearable devices suffer from issues such as interaction interruption, shoulder screen attacks, irrevocable privacy risks related to biometrics, and the ease with which shallow behavioral characteristics can be imitated, making it difficult to achieve efficient and secure identity authentication on low-cost consumer devices.
By utilizing selective attention mechanisms, auditory stimulus scenarios with spatial orientation information are designed to induce natural head responses in users. Head posture and kinematic data are recorded, and static statistical representations and dynamic temporal representations are constructed. The Siamese neural network model is then used for identity authentication to achieve reliable identification of user identities.
It achieves low-cost, natural interactive identity authentication on smart head-mounted devices, has strong anti-counterfeiting capabilities, is hardware-friendly and minimally disruptive to users, and boasts excellent authentication performance with a low error rate and high accuracy.
Smart Images

Figure CN123116237A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of human-computer interaction and information security, and relates to user identity authentication and wearable device interaction technology. Specifically, it relates to a method and apparatus for identity authentication using auditory perception-motor response features triggered by selective attention mechanisms. Background Technology
[0002] Selective attention is a human auditory selective ability where attention is focused on one person's conversation while ignoring other background conversations or noise. In this process, the auditory system relies on perceptual cues such as loudness, pitch, and spatial location for sound stream separation and selective attention. Simultaneously, to optimize binaural cues, users unconsciously perform head movements and tracking actions, forming a closed-loop process of "hearing-perception-action response." Because individuals differ in cochlear filtering, head correlation transfer function (HRTF), perceptual sensitivity, attentional control, and reaction time, this closed-loop response pattern possesses distinguishable individual characteristics and holds promise as a liveness detection signal that is difficult to record, play back, or simply imitate.
[0003] With the development of head-mounted wearable devices such as smart glasses, smart headphones, and VR / XR headsets, their capabilities are constantly expanding, increasingly supporting security-sensitive operations such as payment authorization, privacy access, account login, and privacy information broadcasting. However, these devices often face temporary device sharing and short-term borrowing, leading to a growing demand from users for seamless, low-intrusion, and impersonation-resistant identity authentication in daily environments. Traditional explicit authentication methods (such as passwords and patterns) typically require users to actively input information, easily interrupting the user experience and potentially posing a risk of theft by bystanders in public settings. Physiological biometrics (such as eyes, irises, pupils, and heart rate) often rely on dedicated sensors and are sensitive to the environment; their static and irreversible nature may also pose privacy and impersonation risks. Existing behavioral biometrics (such as gait and simple head / hand movements) primarily reflect superficial movement patterns, failing to capture deeper perceptual and cognitive processes, thus limiting their anti-imitation capabilities. While cognitive authentication methods based on eye tracking or electroencephalography (EEG) can utilize deeper signals, their high hardware costs and invasiveness make them difficult to popularize in consumer wearable devices. Therefore, there is an urgent need for an identity authentication method and device that can be implemented in commercial wearable devices at low cost, utilizes human depth perception, and has strong resistance to imitation attacks. Summary of the Invention
[0004] The purpose of this invention is to address the common problems in existing identity authentication methods for smart wearable head-mounted devices when performing security-sensitive operations such as payment authorization, privacy access, and account login. These problems include interactive interruption, shoulder screen attacks, irreversible privacy risks related to biometrics, and the ease with which shallow behavioral features can be imitated. The invention provides an auditory perception-based identity authentication method and device based on a selective attention mechanism. Specifically, it utilizes the user's selective attention to target sound sources in complex sound fields and the unique "auditory perception-head movement response" closed-loop characteristics exhibited during sound source localization / tracking to reliably identify the user's identity. In this process, the user only needs to make a natural perceptual response to the auditory stimulus scene; there is no need to explicitly input passwords or patterns, nor is it necessary to rely on additional dedicated acquisition equipment.
[0005] The present invention first designs an auditory stimulus scenario with spatial orientation information, focusing on acoustic factors that affect auditory perception, to induce a natural head-sensing response in the user, enabling the user to discover, locate, or track a target sound source without explicit input. Then, during the task, the user's head posture and kinematic data, as well as the spatial state information of the target sound source, are recorded simultaneously. Based on the correspondence between head orientation and target orientation, error sequences and trajectory sequences that characterize the localization behavior are constructed. Subsequently, the present invention further forms static statistical representations and dynamic temporal representations from these sequences, and inputs both types of representations into a Siamese model to map them into a unified embedding representation. Authentication is determined by comparing the similarity between the authentication sample and the registration template on this embedding representation.
[0006] The technical solution adopted in this invention is as follows: An auditory perception authentication method based on selective attention mechanism includes the following steps: Construct auditory stimulus scenarios that simulate selective attention mechanisms; Spatial audio stimuli are generated using the aforementioned auditory stimulation scenario to induce natural head responses in users and collect user data. The collected user data is preprocessed to obtain trial segments; For each of the user's trial segments, a feature representation for identity authentication is constructed, the feature representation including static statistical representation and dynamic temporal representation; The static statistical representation and dynamic temporal representation are used as samples to be certified and input into the neural network model to obtain the embedding representations corresponding to the samples to be certified and the registration template, respectively. The similarity between the embedding representations corresponding to the samples to be certified and the registration template is calculated to obtain the certification score. The authentication score is compared with a threshold to determine whether the identity authentication is successful.
[0007] Furthermore, the constructed auditory stimulus scenarios all include background noise from multiple speakers, and these auditory stimulus scenarios include: 1) Loudness perception task scenario: Apply sound intensity cues, the target sound source is a single pure tone, the intensity gradually increases from silence, and the user turns their head to locate the sound source after noticing it; 2) Frequency discrimination task scenario: Applying pitch cues, the target is a double pure tone in a symmetrical orientation, where the frequency of the target pure tone changes linearly. The user identifies the high-pitched target pure tone and turns their head to locate it. 3) Sound source localization task scenario: Applying spatial cues, the target is a single pure tone that moves continuously along a horizontal arc, and the user keeps turning their head to track it; 4) Speaker localization task scenario: Applying spatial cues, the target is natural speech moving along a horizontal arc, and the user continuously tracks the target speaker in a multi-speaker context.
[0008] Furthermore, the collected user data includes user head motion data, spatial state information of the target sound source, interactive input, and timestamps. User head motion data includes the user's position, Euler angles, velocity, angular velocity, acceleration, and angular acceleration. Spatial state information of the target sound source includes its spatial coordinates, relative azimuth, frequency, and volume intensity, and interactive input and timestamps are recorded. Interactive input refers to the operation signals generated by the user through the handheld controller during the auditory stimulus task, used to characterize the user's perception and confirmation of the target sound source, selection judgment, or task completion indication.
[0009] Furthermore, the preprocessing operations for user data include: calculating the positioning error between the head orientation and the target direction, calibrating the position trajectory relative to the initial position at the beginning, and dividing the continuously collected user data into multiple trial segments according to the stimulus start / end time or user confirmation input event, wherein each trial segment corresponds to an authentication sample.
[0010] Furthermore, the static statistical representation includes 165 dimensions such as mean, variance, peak value, energy, zero crossover rate, convergence slope, pause ratio, and mean absolute error, while the dynamic time series representation is a time series formed by three-dimensional position and three-dimensional rotation.
[0011] Furthermore, the neural network model is a Siamese neural network model. The static branch of the Siamese neural network model adopts an MLP structure and projects the static statistical descriptor in the order of Linear-BN-ReLU-Dropout. The dynamic branch includes three Conv1D convolutional blocks and attention pooling, which are used to aggregate temporal information.
[0012] Furthermore, the step of comparing the authentication score with a threshold to determine whether identity authentication is successful includes: determining that authentication is successful when the authentication score is greater than the threshold, and rejecting it otherwise.
[0013] An auditory perception authentication device based on a selective attention mechanism includes: Auditory Scene Construction Module: Used to construct auditory stimulus scenes that simulate selective attention mechanisms; Stimulus generation module: Used to generate spatial audio stimuli using auditory stimulation scenarios to induce natural head responses in users; Data acquisition module: used to synchronously acquire user data during the execution of the auditory stimulus scenario; Preprocessing module: Used to preprocess the collected user data to obtain trial segments; Feature extraction module: used to generate static statistical representations and dynamic temporal representations for each trial segment; The authentication module is used to input static statistical representations and dynamic temporal representations into a neural network model to obtain the embedding representations of the sample to be authenticated and the registered template, calculate the similarity between the embedding representations of the sample to be authenticated and the registered template to obtain the authentication score, and compare the authentication score with a threshold to output whether the authentication is passed or rejected.
[0014] The method of this invention can authenticate identity on wearable devices based on the unique auditory perception-motor response of humans, and has the following advantages compared with the prior art: 1) This invention utilizes the auditory perception-motor response closed loop triggered by selective attention mechanism as an authentication signal, which has natural living attributes and can improve the ability to resist imitation attacks and security. 2) This invention is designed for wearable head devices and can be achieved solely through spatial audio output and head motion capture. It is hardware-friendly, offers natural interaction, and minimizes user disruption. 3) This invention uses a fusion model of static statistical features and dynamic temporal features, and trains and learns through a Siamese network, which can achieve high authentication performance and achieve a low error rate and a high balanced accuracy in public experiments. Attached Figure Description
[0015] Figure 1 This is a flowchart of the steps of the auditory perception authentication method based on selective attention mechanism of the present invention.
[0016] Figure 2 This is a schematic diagram of the process for identity authentication using the method of the present invention.
[0017] Figure 3 This is a schematic diagram of the module composition of the auditory perception identity authentication device based on selective attention mechanism of the present invention. Detailed Implementation
[0018] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0019] The auditory perception authentication method based on selective attention mechanism provided by this invention is applicable to smart wearable head-mounted devices such as smart glasses, smart headphones, and VR / XR headsets.
[0020] One embodiment of the present invention provides an auditory perception authentication method based on a selective attention mechanism, such as... Figure 1 As shown, it includes the following steps: Construct auditory stimulus scenarios that simulate selective attention mechanisms; Spatial audio stimuli are generated using auditory stimulation scenarios to induce natural head responses in users and collect user data. The collected user data is preprocessed to obtain trial segments; For each trial segment of a user, a feature representation for identity authentication is constructed, which includes static statistical representation and dynamic temporal representation; Static statistical representation and dynamic temporal representation are used as samples to be certified and input into the neural network model to obtain the embedding representations corresponding to the samples to be certified and the registered template, respectively. The similarity between the embedding representations corresponding to the samples to be certified and the registered template is calculated to obtain the certification score. The authentication score is compared with a threshold to determine whether the identity authentication is successful.
[0021] One embodiment of the present invention provides an auditory perception authentication method based on a selective attention mechanism, the process of which is as follows: Figure 2 As shown, the main steps include: First, constructing an auditory stimulus scene with spatial orientation information and outputting spatial audio to induce a natural head response in the user during the perception of the target stimulus. During task execution, information such as the user's head movements and the spatial state of the target sound source is recorded synchronously and preprocessed. Then, static statistical representations and dynamic temporal representations are calculated based on this, and the representations are input into a neural network model to obtain the user embedding representation. Finally, the authentication result is output by comparing the similarity between the sample to be authenticated and the registered template and combining it with a threshold strategy.
[0022] The method of this invention consists of two parts: registration and authentication. In the registration phase, auditory task response data of the same user is collected, and a registration template for that user is generated and stored through a Siamese network. In the authentication phase, samples to be authenticated are collected and their similarity is calculated with the registration template in the registration database to complete the pass / reject decision. Furthermore, the parameters of the Siamese network structure can be pre-trained in the offline training phase and deployed on the terminal side or the server side. In practical use, the terminal side can complete the auditory stimulus output and head movement data acquisition. Feature extraction and similarity calculation can be performed on the terminal side or uploaded from the terminal side to the server side for execution, to meet the deployment requirements of different computing power, power consumption, and privacy strategies.
[0023] One embodiment of the present invention provides an auditory perception authentication method based on a selective attention mechanism, implemented and verified using a self-collected and constructed VR head-mounted device dataset, wherein the dataset includes auditory task response data of 50 subjects. The method includes the following steps: 1) Design and construct auditory task scenarios on the Unity platform to generate auditory stimuli with spatial orientation information, thereby inducing users to selectively pay attention to target sound sources in complex sound fields and generate natural head responses during sound source localization / tracking. The auditory task scenarios include four task types: loudness perception task scenario, frequency discrimination task scenario, sound source localization task scenario, and speaker localization task scenario. Furthermore, multiple interfering sound sources are superimposed in the auditory task scenarios to form a multi-speaker background or background noise, thereby enhancing the selective attention process.
[0024] 2) Collect the response data of the VR headset when the user performs the auditory task scenario described in step 1), i.e., user data. The user data includes: user head movement data, spatial state information of the target sound source, interactive input, and timestamps.
[0025] 3) Preprocess the user data collected in step 2). The preprocessing operations include: calculating the positioning error between the head orientation and the target direction, calibrating the position trajectory relative to the initial position at the beginning, and dividing the continuously collected user data into multiple trial segments according to the stimulus start / end time or user confirmation input event, wherein each trial segment corresponds to an authentication sample.
[0026] The calibration of the position trajectory relative to the initial position at the start refers to calibrating the user's head position trajectory based on the user's head position when starting the task, in order to eliminate the impact of differences in the starting positions of different tasks on subsequent feature extraction.
[0027] The calculation of the positioning error between the head orientation and the target direction refers to calculating the positioning error between the user's head orientation and the target sound source direction based on the correspondence between the user's real-time head orientation and the spatial azimuth angle of the target sound source, and forming an error sequence that can characterize the positioning or tracking behavior.
[0028] 4) Construct a feature representation for identity authentication for each trial segment obtained in step 3), including static statistical representation and dynamic temporal representation. The data for each trial segment includes calibrated user head motion data and an error sequence obtained from the correspondence between head orientation and the direction of the target sound source.
[0029] 5) Input the static statistical representation and dynamic temporal representation obtained in step 4) into the Siamese network model to obtain the embedded representations corresponding to the authentication sample and the registration template respectively, and calculate the similarity between the embedded representations to obtain the authentication score.
[0030] 6) Compare the authentication score obtained in step 5) with the threshold. If the authentication score is not less than the threshold, output that authentication is successful; otherwise, output that authentication is rejected. The threshold can be configured according to the system security policy.
[0031] 7) In actual operation, this embodiment includes a registration stage and an authentication stage: In the registration stage, a preset number of trial fragments of the same user are collected, and multiple embedded representations are obtained through steps 3) to 5) to form a registration template and store it; In the authentication stage, samples to be authenticated are collected and embedded representations are obtained through steps 3) to 5), similarity is calculated with the registration template, and authentication decision is completed according to step 6).
[0032] 8) To evaluate the effectiveness of the method of the present invention, this embodiment conducts experimental verification based on the self-collected dataset. In one implementation, the Siamese network model is trained and evaluated using a preset training / validation / testing partitioning method, and the equal error rate (EER), area under the ROC curve (AUC), and balanced accuracy (BAC) of 10-fold cross-validation are used for measurement during the certification stage. The overall results are shown in Table 1, and the results for each auditory task scenario are shown in Table 2.
[0033] Table 1. Overall Results on Self-Collected Datasets Table 2. Results for various auditory task scenarios on the self-collected dataset (mean ± standard deviation) Based on the results in Tables 1 and 2, the overall authentication performance of the method of the present invention on the self-collected dataset and the authentication performance in various auditory task scenarios are shown, respectively, verifying that the method of the present invention has good authentication effectiveness and stability.
[0034] One embodiment of the present invention provides an auditory perception authentication device based on a selective attention mechanism, such as... Figure 3 As shown, it includes: Auditory Scene Construction Module: Used to construct auditory stimulus scenes that simulate selective attention mechanisms; Stimulus generation module: Used to generate spatial audio stimuli using auditory stimulation scenarios to induce natural head responses in users; Data acquisition module: used to synchronously acquire user data during the execution of the auditory stimulus scenario; Preprocessing module: Used to preprocess the collected user data to obtain trial segments; Feature extraction module: used to generate static statistical representations and dynamic temporal representations for each trial segment; The authentication module is used to input static statistical representations and dynamic temporal representations into a neural network model to obtain the embedding representations of the sample to be authenticated and the registered template, calculate the similarity between the embedding representations of the sample to be authenticated and the registered template to obtain the authentication score, and compare the authentication score with a threshold to output whether the authentication is passed or rejected.
[0035] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be found in the corresponding processes in the aforementioned method embodiments.
[0036] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.
[0037] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.
[0038] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the various steps of the method of the present invention.
[0039] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Those skilled in the art can modify or make equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention. The scope of protection of the present invention should be determined by the claims.
Claims
1. An auditory perception authentication method based on selective attention mechanism, characterized in that, Includes the following steps: Construct auditory stimulus scenarios that simulate selective attention mechanisms; Spatial audio stimuli are generated using auditory stimulation scenarios to induce natural head responses in users and collect user data. The collected user data is preprocessed to obtain trial segments; For each trial segment of a user, a feature representation for identity authentication is constructed, which includes static statistical representation and dynamic temporal representation; Static statistical representation and dynamic temporal representation are used as samples to be certified and input into the neural network model to obtain the embedding representations corresponding to the samples to be certified and the registered template, respectively. The similarity between the embedding representations corresponding to the samples to be certified and the registered template is calculated to obtain the certification score. The authentication score is compared with a threshold to determine whether the identity authentication is successful.
2. The method according to claim 1, characterized in that, The auditory stimulation scenarios include: Loudness perception task scenario: Apply sound intensity cues, the target sound source is a single pure tone, the intensity gradually increases from silence, and the user turns their head to locate the sound source after noticing it; Frequency discrimination task scenario: Applying pitch cues, the target is a double pure tone in a symmetrical orientation. The frequency of the target pure tone changes linearly. The user identifies the high-pitched target pure tone and turns their head to locate it. Sound source localization task scenario: Applying spatial clues, the target is a single pure tone that moves continuously along a horizontal arc, and the user keeps turning their head to track it; Speaker localization task scenario: Applying spatial cues, the target is natural speech moving along a horizontal arc, and the user continuously tracks the target speaker in a multi-speaker context.
3. The method according to claim 1, characterized in that, The user data includes user head motion data, spatial state information of the target sound source, interactive input, and timestamps; the user head motion data includes the user's position, Euler angles, velocity, angular velocity, acceleration, and angular acceleration; the spatial state information of the target sound source includes the target sound source's spatial coordinates, relative azimuth angle, frequency, and volume intensity.
4. The method according to claim 1, characterized in that, The preprocessing includes: calculating the positioning error between the head orientation and the target direction, calibrating the position trajectory relative to the initial position at the beginning, and dividing the continuously collected user data into multiple trial segments based on the stimulus start / end time or user confirmation input event, with each trial segment corresponding to an authentication sample.
5. The method according to claim 1, characterized in that, The static statistical representation includes mean, variance, peak value, energy, zero crossover rate, convergence slope, pause ratio, and mean absolute error. The dynamic time series representation is a time series formed by three-dimensional position and three-dimensional rotation.
6. The method according to claim 1, characterized in that, The neural network model is the Siamese neural network model. The static branch of the Siamese neural network model adopts an MLP structure and projects the static statistical descriptor in the order of Linear-BN-ReLU-Dropout. The dynamic branch includes three convolutional blocks and attention pooling, which are used to aggregate temporal information.
7. The method according to claim 1, characterized in that, The step of comparing the authentication score with a threshold to determine whether identity authentication is successful includes: determining that authentication is successful when the authentication score is greater than the threshold, and rejecting it otherwise.
8. An auditory perception authentication device based on a selective attention mechanism, characterized in that, include: The auditory scene construction module is used to construct auditory stimulus scenes that simulate selective attention mechanisms. The stimulus generation module is used to generate spatial audio stimuli using auditory stimulus scenarios to induce natural head responses in users. The data acquisition module is used to synchronously acquire user data during the execution of the auditory stimulus scenario; The preprocessing module is used to preprocess the collected user data to obtain trial segments; The feature extraction module is used to generate feature representations for identity authentication for each trial segment, including static statistical representations and dynamic temporal representations. The authentication module is used to input static statistical representations and dynamic temporal representations into a neural network model to obtain the embedding representations of the sample to be authenticated and the registered template, calculate the similarity between the embedding representations of the sample to be authenticated and the registered template to obtain the authentication score, and compare the authentication score with a threshold to output whether the authentication is passed or rejected.
9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.
