Rapid two-factor identity authentication method based on equipment position proximity relation

Abstract

The invention discloses a rapid two-factor identity authentication method based on an equipment position proximity relationship. The method comprises the following steps of: generating an acoustic sensing signal by adopting a broadband high-autocorrelation linear frequency modulation signal, playing the signal by a mobile terminal, synchronously acquiring a signal sample pair recorded by the mobile terminal and login equipment, and inputting the signal sample pair into an authentication model network after environmental noise elimination and segmentation. The authentication model network is based on a twin neural network architecture, microphone frequency response difference interference can be eliminated, and equipment position acoustic multipath feature extraction and position proximity relation identification are realized. The authentication model network extracts the frequency response specificity characteristics of the loudspeaker of the mobile terminal, and constructs a personalized authentication model of the registered terminal in combination with a weighted joint loss function, thereby realizing accurate authentication of the registered terminal in an authentication stage. According to the method, bidirectional acoustic communication between devices is not needed, the device synchronization complexity and hardware requirements are reduced, same-position attacks and replay attacks can be resisted, and rapidness, adaptability and safety are taken into account.

Description

Technical Field

[0001] This invention belongs to the field of device security and security authentication, and involves combining acoustic perception and deep learning models to realize location proximity authentication between mobile smart terminals and login devices, which is applicable to security access authentication scenarios of various terminal devices. Background Technology

[0002] With the widespread adoption of mobile smart terminals (smartphones, smartwatches, etc.), two-factor authentication has gradually become the mainstream identity verification method. Its core mechanism requires users to provide two types of "identity credentials," and verification is only possible after both are verified successfully. The username and password typically serve as the first layer of identity credentials, while the second layer is determined by whether the user possesses a pre-registered mobile terminal. In practice, after the user's submitted username and password are verified, a second layer of authentication is required on the pre-registered mobile terminal by entering or clicking a specific verification code issued by the authentication server. This authentication method, which requires input or clicking, relies on the user to actively perform additional interactive actions, severely impacting the user experience.

[0003] In two-factor authentication scenarios, when a legitimate user authenticates, their registered mobile terminal and the device they are about to log into (hereinafter referred to as the login device) are typically in close proximity. However, when an attacker attempts to log in, they are often located far from the legitimate user's registered mobile terminal. Based on these characteristics, existing technologies analyze the proximity between the registered mobile terminal and the login device to achieve two-factor authentication without the user's active participation. For example, the Sound-proof scheme analyzes whether there are shared audio events in the environment of the mobile terminal and the login device (such as an office computer) to determine the proximity between the two devices. However, attackers can actively construct shared audio events in the environment of their unauthorized login device and the legitimate user, making the security of this scheme difficult to guarantee effectively. To address these security vulnerabilities, the Proximity-proof scheme extracts the hardware acoustic fingerprint (i.e., the frequency response specific characteristics of the microphone and speaker) of the mobile terminal through direct acoustic communication between the mobile terminal and the login device, and combines this with audio ranging technology to reject login requests exceeding a distance threshold, thereby improving the security of two-factor authentication. In addition, the Proximity-Echo solution uses active acoustic sensing technology to identify the proximity relationship between devices by comparing the similarity of the acoustic location fingerprints (i.e., the spatial acoustic multipath features corresponding to the location) of the current location of the mobile terminal and the logged-in device.

[0004] However, both of the above schemes rely on two-way acoustic communication between the mobile terminal and the login device in the process of extracting the hardware acoustic fingerprint and spatial acoustic multipath features of the device: that is, the mobile terminal device and the login device need to play the specified acoustic signal in sequence and complete the audio recording through their own microphones. This two-way acoustic communication mechanism has the following three limitations. (1) High complexity of synchronization mechanism: It is necessary to control the speakers and microphones of the two devices to work together through a complex synchronization mechanism to complete the two-way acoustic communication, which greatly reduces the usability of the relevant scheme. (2) Limited hardware compatibility: There are specific requirements for the hardware configuration of the login device (it needs to be equipped with both speakers and microphones), and this requirement cannot be applied to ordinary desktop computers and other devices that are not equipped with microphones or speakers, which limits the application scenarios of the scheme. (3) Security risks: During the two-way acoustic communication process, attackers may obtain the frequency response information of the login device's speaker, which may lead to the risk of launching an acoustic fingerprint forgery attack and threatening the authentication security. Therefore, it is urgent to design a fast and secure two-factor authentication based on the proximity of device locations to improve the usability and security of mobile authentication scenarios. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing two-factor authentication methods based on mobile smart terminals. By using active audio perception technology, this invention obtains the acoustic multipath characteristics of the corresponding locations of the user's mobile terminal and the login device, as well as the specific characteristics of the frequency response of the mobile terminal's speaker, to achieve a fast and secure two-factor authentication method based on location proximity.

[0006] The innovations of this invention are as follows: On the one hand, the authentication method involved in this invention, through the design of a deep learning network architecture based on Siamese neural networks, achieves location acoustic multipath feature extraction and device proximity relationship identification independent of microphone frequency response differences between devices, without requiring bidirectional acoustic communication between devices. On the other hand, this invention mines the specific characteristics of the mobile terminal speaker hardware frequency response contained in the audio samples recorded by the microphone of the login device, and uses this information as the acoustic fingerprint of the mobile terminal speaker for terminal device authentication. Simultaneously, considering that the audio samples also contain interference information such as environmental multipath features and propagation distance, an additional mobile terminal identification module is added to the aforementioned Siamese neural network deep learning network architecture to achieve registration terminal identification independent of location multipath features.

[0007] The objective of this invention is achieved through the following technical solutions.

[0008] The core technology of this invention serves the second layer of verification in two-factor authentication based on mobile terminals. The specific scheme includes two stages: device registration and device authentication. Each stage involves four key steps: sample collection, preprocessing, model training, and device identification. Details are as follows:

[0009] Step 1: Acoustic sensing signal generation. A linear frequency modulated signal with a wide frequency range and high autocorrelation is used as the acoustic sensing signal carrier, and key parameters are optimized and set as follows: starting frequency 12kHz, ending frequency 20kHz, and duration 500ms.

[0010] Step 2: Environmental Multipath Sample Acquisition. Registration Phase: The user places the mobile terminal to be registered in locations near and not near the login device (distance ≥ 20cm). The mobile terminal plays the acoustic sensing signal generated in Step 1. Acoustic signals recorded by the microphones of both the mobile terminal and the login device are simultaneously acquired and paired. Location proximity tags and registration terminal tags are added to form a training sample set. Authentication Phase: The user places the registered mobile terminal in a location near the login device, and a set of the above acoustic signal sample pairs is acquired.

[0011] Step 3: Signal Preprocessing and Segmentation. Environmental Noise Reduction: Based on the preset frequency range of the acoustic sensing signal in Step 1, an FIR bandpass filter with a Hamming window is designed to perform bandpass filtering on the acquired acoustic signal to eliminate out-of-band noise. Signal Segmentation: The target signal interval is located using cross-correlation, and the starting position of the signal is determined by threshold filtering. Segments are then cut according to a preset length.

[0012] Step 4: Authentication Model Training. The authentication model training network comprises three main modules: a feature extractor, a location proximity recognizer, and a registered terminal recognizer. First, the feature extractor consists of two weight-sharing sub-networks. It takes the time-spectrum features of the input signal segment after short-time Fourier transform and outputs standardized compressed features. Second, the location proximity recognizer calculates the distance between samples and corresponding compressed features based on Euclidean distance. It uses a contrastive loss function to aggregate features of registered devices at nearby locations and separate features corresponding to devices at non-nearby locations or non-registered devices. Then, the registered terminal recognizer consists of two fully connected layers. It uses a cross-entropy loss function to enhance the extraction of device acoustic fingerprint features, achieving accurate identification of registered and non-registered devices. Finally, a weighted joint loss function is constructed during joint training, and after training, the user account is bound and stored in the authentication model database.

[0013] Step 5: Device Identity Authentication. After the first authentication (account and password) is successful, the second authentication is initiated, performing the preprocessing and segmentation procedures described above on the collected sample pairs. The user-bound authentication model is retrieved, and the processed features are input into the model. First, a mobile terminal identifier is used for identification. If the predicted probability of "registered terminal" is less than a preset threshold, authentication is rejected; otherwise, a location proximity identifier is used for further judgment. If the predicted probability of "location proximity" is greater than a preset threshold, authentication is successful; otherwise, it is rejected (the threshold can be calibrated according to security requirements).

[0014] Compared with the prior art, the present invention has the following advantages:

[0015] 1. This invention utilizes only the built-in speakers and microphones commonly found in mobile registration terminals and login devices to achieve fast and reliable two-factor authentication based on device proximity without the need for two-way acoustic communication.

[0016] 2. The present invention further constructs an authentication model training network based on a Siamese neural network to achieve device location multipath feature extraction and accurate identification of location proximity relationships that are not affected by differences in microphone frequency response between the login device and the registration terminal, thereby effectively resisting same-location attacks.

[0017] 3. Based on the symmetrical subnetwork structure of the twin neural network, this invention further constructs a device identification module to ensure that the specific characteristics of the speaker frequency response of the mobile registration terminal are effectively preserved during the model training process, and constructs a joint training method to effectively resist regression attacks and improve the security and reliability of identity authentication. Attached Figure Description

[0018] Figure 1 This is a diagram illustrating the authentication method architecture of an embodiment of the present invention.

[0019] Figure 2 This is a scene diagram of multipath acoustic sensing according to an embodiment of the present invention.

[0020] Figure 3 This is a user authentication network architecture diagram according to an embodiment of the present invention.

[0021] Figure 4 This is a diagram of the sub-network structure in the user authentication network architecture of this invention.

[0022] Figure 5 This is a structural diagram of the location proximity identifier in the user authentication network architecture of this invention.

[0023] Figure 6 This is a structural diagram of the device identifier in the user authentication network architecture of this invention.

[0024] Figure 7 The registration device pass rate in this embodiment of the invention

[0025] Figure 8 The pass rate of registration proximity determination in this embodiment of the invention

[0026] Figure 9 The error rate of unregistered devices in this embodiment of the invention

[0027] Figure 10 Error acceptance rate in non-nearest neighbor states Detailed Implementation

[0028] The proposed fast two-factor authentication method based on device proximity in this invention uses account and password verification as the first layer of authentication. This first layer of authentication must be completed before proceeding to the second layer of mobile terminal device authentication. Figure 1 As shown, the second level of authentication comprises two core stages: device registration and device authentication. Before each stage begins, acoustic sensing signals must be generated based on preset parameters. Device registration stage (process indicated by dashed lines): Based on the generated acoustic sensing signals, environmental multipath samples recorded by the login device and the registered mobile terminal device are simultaneously collected and paired. These sample pairs undergo environmental noise cancellation and signal segmentation preprocessing to obtain multipath sample fragments. These fragments are then input into the authentication model training module containing a feature extractor to complete training, ultimately resulting in two types of models: one is a device proximity relationship identifier unaffected by differences in microphone frequency response between devices, and the other is a registered device identifier unaffected by multipath feature interference. Device authentication stage (process indicated by solid lines): After simultaneously collecting environmental multipath samples from the login device and the mobile terminal device to be authenticated, the same signal preprocessing and segmentation process as in the registration stage is used to obtain corresponding multipath sample fragments. These fragments are then input into the two types of models trained in the registration stage. The proximity relationship identifier determines whether the terminal device is in a proximity state, while the mobile device identifier completes the identity verification of the device to be authenticated.

[0029] The detailed design of its core steps is as follows:

[0030] Step 1: Generation of Acoustic Sensing Signals

[0031] This step involves modulating the acoustic sensing signal and setting parameters such as frequency range and duration. This invention requires active acoustic sensing technology to simultaneously acquire the location multipath characteristics of the mobile terminal device and the login device, as well as the speaker frequency response specific characteristics of the mobile terminal device. Given that a wider frequency range facilitates accurate extraction of speaker frequency response characteristics, and higher autocorrelation is beneficial for efficient extraction of environmental acoustic multipath characteristics, this invention selects a linear frequency modulated signal with both wide frequency range and high autocorrelation characteristics for the acoustic sensing signal. The discrete real number expression for modulation is:

[0032] ,

[0033] in: Signal amplitude (set to 0.5); The index is the sampling point index, and satisfies ( The total number of sampling points, and , The duration of the signal. (Sampling rate). and These are the start and end frequencies of the signal, respectively. Considering the sampling rate limitations of commercial mobile terminal devices and the distribution range of environmental noise, this invention uses the start frequency... With termination frequency The frequencies were set to 12kHz and 20kHz respectively. A longer signal duration results in a higher signal-to-noise ratio, which is more conducive to extracting environmental multipath features, but it also increases the delay in the acoustic sensing process. Considering these trade-offs, this invention, after experimental verification, empirically sets the signal duration T to 500ms.

[0034] Step 2: Environmental multipath sample collection

[0035] Both the device registration and device authentication phases require the acoustic sensing signals generated in step 1. The user performs environmental multipath sample collection by using a mobile terminal device that is to be registered or has already been registered. As shown in Figure 2, the specific process of the sample collection stage is as follows: the mobile terminal device's speaker actively plays a pre-generated acoustic sensing signal. The signal propagates through both the direct propagation path and the environmental multipath reflection path, and is then collected by the built-in microphone of the mobile terminal device and the microphone of the login device, respectively. The corresponding acoustic samples are denoted as follows: (Data collection from mobile terminal devices) and (Data collected from logged-in devices) and together they form a set of sample pairs. .

[0036] To enable the subsequent authentication model to learn the location proximity features between devices, during the registration phase, the user needs to place the mobile terminal to be registered in both the "nearby location" and "non-nearby location" of the login device (e.g., the distance between devices should not be less than 20cm), and add location proximity status labels to the collected sample pairs. Correspondingly, the user-provided first A registered sample can be represented as . Specifically, when Then the sample corresponds to the sample pair collected when the mobile terminal to be registered and the login device are in a state of proximity; if This corresponds to the sample pairs collected when the two devices are not in close proximity. Furthermore, to eliminate the interference of propagation distance and multipath effects on the extraction of speaker frequency response features, when the devices are not in close proximity, the user needs to collect sample pairs when the mobile terminal to be registered and the login device are at different distances.

[0037] The environmental sample collection process during the device certification phase is relatively simple: users only need to place their registered mobile terminal near the login device to collect a set of sample pairs.

[0038] Step 3: Signal preprocessing and segmentation

[0039] Step 3.1: Environmental noise elimination

[0040] This step performs environmental noise cancellation processing on the acoustic sample pairs collected during the registration or authentication phase. Since step 1 already adequately adapted to the distribution characteristics of everyday noise when setting the frequency range of the acoustic sensing signal, this step uses bandpass filters to process the samples collected by the mobile registration terminal and login device respectively. ( Bandpass filtering is performed on the discrete-time index to eliminate environmental noise.

[0041] Specifically, based on the acoustic sensing signal frequency range preset in step 1 By designing an FIR bandpass filter using a window function (introducing a Hamming window to suppress spectral sidelobes), the discrete-time impulse response of the filter is obtained. (in Let f(x) be the discrete index of the impulse response, representing the i-th element in the impulse response. (each sampling point); then, through temporal convolution operations, the samples are processed respectively. Noise cancellation is performed to obtain the corresponding signal segment after out-of-band noise removal. and .Right now:

[0042]

[0043]

[0044] in The length of the excitation response of the FIR filter. For temporary summation variables, and The FIR bandpass filter used in this step has linear phase characteristics. While achieving noise cancellation, it can avoid phase distortion of multipath features in the linearized real audio samples, thus providing a high-purity signal foundation for subsequent signal processing.

[0045] Step 3.2: Signal Segmentation

[0046] This step focuses on the audio samples obtained after noise removal in step 3.1 (denoised samples from the registered terminal device). Denoising samples of login devices The core of precise segmentation is to utilize the characteristics of linear frequency modulated signals—"strong autocorrelation and concentrated cross-correlation peaks"—to locate the target signal interval, ensuring that the extracted acoustic segment contains the complete multipath features of the acoustic sensing signal. The specific process is as follows:

[0047] Denoising acoustic samples , (when for or These represent the samples collected by the corresponding mobile registration terminals. Or collected by the login device ) and pre-generated acoustic sensing signals Perform cross-correlation operations: .in, This is the delayed index in the cross-correlation output. For temporary summation variables, and . For the denoised acoustic samples Total sample size. Cross-correlation output results. Peak corresponding sample The starting position of the acoustic sensing signal. To eliminate false detections caused by residual noise interference, a peak detection strategy of "threshold constraint + main peak selection" is adopted below:

[0048] First, calculate the global maximum value of the cross-correlation results. Secondly, set a threshold. ,in This is an empirical threshold coefficient, set to a value of 0.8. Then, for... Perform a traversal and filter out the first value exceeding... The peak that satisfies the condition of "monotonically decreasing values ​​in its left and right neighborhoods" has a corresponding delay index that is the starting position of the target signal, denoted as . Finally, starting from the beginning position. Based on the discrete length of the acoustic sensing signal in step 1, By extracting segments, we can obtain the segmented segments corresponding to the registered terminal device and login device samples. ( This represents the number of samples of the acoustic sensing signal generated in step 1. and These represent the mobile registration terminal and login device corresponding to the sample, respectively. Indicates from Extracting from signal samples arrive (Signal segments). This step solves the problem of distinguishing target signals from noise in complex environments by using "cross-correlation peak localization + adaptive threshold peak detection". It ensures that the segmented segments completely retain the direct wave and multipath reflection wave characteristics of the acoustic sensing signal, providing standardized input with "uniform length and complete features" for feature extraction in subsequent model training or identity recognition.

[0049] Step 4: Training the Authentication Model

[0050] During the device registration phase, the acoustic samples submitted by the user are processed sequentially using the environmental noise cancellation and signal segmentation steps in step 3 to obtain the corresponding target acoustic signal segments. ),in This represents the acoustic signal segment collected by the logged-in device. This represents an acoustic signal segment collected by the registered terminal device. A short-time Fourier transform (SFT) is performed on the aforementioned acoustic signal segment. ), generating the corresponding time-spectrum features The spectral features at that time are then input into the authentication model constructed in this step for training. As shown in Figure 3, the authentication model training network architecture includes three core functional modules: a feature extractor, a location proximity recognizer, and a registered device recognizer. The specific composition and working mechanism of each module are detailed below:

[0051] Step 4.1: Feature Extractor

[0052] The feature extractor consists of two structurally consistent sub-networks with shared weights to achieve consistent extraction of acoustic features from both the login device and the mobile terminal. As shown in Figure 4, each sub-network comprises three cascaded convolutional blocks, each containing a 2D convolutional layer, a batch normalization layer, a ReLU activation function layer, and a max-pooling layer. First, the input sample... ( First, a 2D convolutional layer is passed in for the first convolutional block. This layer is configured with 32 elements of size 1. A convolutional kernel with a stride of 1 is used to perform spatial feature mapping and compression on the input features through convolution operations, resulting in an initial feature map. This initial feature map is then processed by a batch normalization layer to standardize the data distribution and mitigate the risk of gradient vanishing. A ReLU activation function layer is then used to achieve non-linear feature transformation (accelerating model training convergence). Finally, a kernel with a size of [missing information] is applied. The max-pooling layer (with a stride of 2) is used for downsampling to reduce the feature dimension while preserving key feature information. The output feature map of the first convolutional processing submodule is fed into the 2D convolutional layer of the second convolutional processing submodule, which is configured with 64 pixels of size 1. With a stride of 1, after repeatedly executing the above "convolution operation - batch normalization - ReLU activation" process, a convolutional kernel of the same size as the first convolutional block is used. A max-pooling layer with a stride of 2 is used for downsampling to further enhance the abstract representation of features. The output feature map of the second convolutional processing submodule is fed into the two-dimensional convolutional layer of the third convolutional processing submodule, which is configured with 128 pixels of size 128. After performing the "convolution operation - batch normalization - ReLU activation" process again with a convolution kernel of size 1, the kernel size is also used. A max-pooling layer with a stride of 2 (the same as the first convolutional block) is used for downsampling to output the final compressed feature representation. .

[0053] Step 4.2: Location Proximity Identifier:

[0054] The core function of a proximity detector is to realize the "acoustic sample features collected by the login device". "Acoustic sample features collected by mobile terminal" Accurate identification of the proximity relationship between "". For example Figure 5 As shown, the location proximity recognizer uses a fully connected layer (512 neurons) to transform the high-dimensional local features of login device samples and mobile phone-collected samples into a compact low-dimensional feature vector, facilitating the evaluation of the distance (similarity) between features. For computational convenience, this step uses Euclidean distance to calculate the distance between features of different samples, as specifically implemented below:

[0055] Define the training sample pair set as .in, For training sample pair indexes, This represents the total number of sample pairs to be trained. This is a label for location proximity. When the... When the corresponding "login device" and "mobile phone" of a sample pair are within a preset proximity range, ;otherwise . Register tags for mobile terminals. When the first When the sample pairs correspond to the "phone number" used for registration, ;otherwise . For the first Group sample pairs In weight The Euclidean distance is given. At this point, the loss function of the proximity recognizer is... for

[0056]

[0057] in, A preset interval threshold (e.g., an empirical value of 1.2) is used to define the lower limit of the feature distance for "non-locationally proximate" sample pairs. This applies when the sample pair is provided by a registered terminal device in a locationally proximate state. The loss function will minimize This makes the two sets of feature vectors as close as possible. This applies when sample pairs come from unregistered terminal devices or registered terminal devices that are not in a location proximity state. The loss function will minimize That is, the distance between two sets of feature vectors must be no less than [a certain value]. This achieves effective separation.

[0058] Step 4.3: Device Identifier:

[0059] The loss function setting in step 4.2 helps the proximity relationship recognizer to only apply to registered terminal devices, but it doesn't explicitly emphasize the need to retain the acoustic fingerprint information of the registered terminal device's speaker during the feature extraction stage. Therefore, this invention adds a device recognizer based on step 4.2 to further improve the effective extraction of acoustic fingerprint information of registered terminal devices by the feature extractor. The network structure of the registered terminal device recognizer contains two fully connected layers: the first layer has 512 neurons for high-dimensional mapping of the compressed features output by the feature extractor. The second layer has 2 neurons, corresponding to the two categories of labels: "registered terminal device" and "non-registered terminal device". Its training process is as follows: during the training phase, acoustic samples collected by non-registered terminal devices are introduced as negative samples, which together with the registered terminal device samples (positive samples) submitted by the user constitute the training set; the login device samples (the first, second, and third samples) output by the feature extractor are used for the training set. Features of each sample The input registered terminal device identifier is mapped through a fully connected layer to obtain a 2D feature vector, which is then converted into a class prediction probability vector through a softmax function. The first The true label for each sample is (Registered or unregistered terminal devices) are converted into one-hot encoded vectors. The cross-entropy loss function is used to calculate the deviation between the predicted and actual values. Defined as:

[0060]

[0061] in For the total number of tamed samples, For training sample index, and They represent the first One-hot encoded vectors corresponding to each sample With category prediction probability vector The The probability corresponding to the dimension.

[0062] Step 4.4: Joint Model Training

[0063] To achieve coordinated optimization of location proximity recognition and registered terminal device recognition, this step constructs a weighted joint loss function. The formal definition is as follows:

[0064]

[0065] and These are the weight coefficients, used to balance the training priorities of the two tasks, and are empirically set to 0.65 and 0.35 respectively.

[0066] The sample collection process for model training is as follows: First, users place their registered mobile devices in the "nearby login devices" area. "and non-proximity login devices" At the location of "", multiple sets of acoustic sample pairs are collected to form a positive sample set (corresponding to) Then, the service provider provides multiple unregistered mobile terminals, which are then logged in from nearby devices. "and non-proximity login devices" Multiple acoustic sample pairs were collected at the location of "", forming a negative sample set (corresponding to After training, the optimized feature extractor, location proximity recognizer, and registered terminal device recognizer are bound to the user account and stored in the authentication model database for subsequent device identity verification.

[0067] Step 5: Device Identification

[0068] In the two-factor authentication process, once the first authentication credentials (account and password) are verified, the second authentication (device authentication) is automatically initiated. First, the legitimate user places the mobile terminal used during registration at a preset proximity location to the login device. The mobile terminal's speaker plays a preset acoustic sensing signal, and the login device and the mobile terminal's microphones simultaneously record this signal, forming a multipath acoustic sample pair. The collected multipath acoustic sample pairs are processed sequentially using step 3 (noise cancellation and signal segmentation) to obtain the corresponding target acoustic signal segments. Based on the user account that has passed the first authentication, the authentication model corresponding to that user (including a feature extractor, a registration device identifier, and a location proximity identifier) ​​is retrieved from the authentication model database. The segmented acoustic signal segments are converted into time-spectral features and input into the retrieved feature extractor to obtain a compressed feature representation. Then, the obtained compressed feature representation is input into the registration device identifier to obtain the predicted probability of the "registered device" category. If the predicted probability is less than a certain threshold (preferably set to 0.9), device authentication is directly rejected. Otherwise, the obtained compressed feature representation is input into the location proximity identifier to obtain the predicted probability of the "location proximity" relationship. If the predicted probability is greater than a certain threshold (preferably set to 0.9), the device authentication passes; otherwise, the device authentication fails. The threshold value involved in this step can be adjusted through experimental data calibration based on the security requirements of the actual application scenario. In this invention, it is preferably set to 0.9 to balance authentication security and user experience.

[0069] Example

[0070] This embodiment uses a ThinkPad T14 Gen2 business laptop as the login device and five business smartphones—Redmi K40, Redmi K60, Redmi K80, ONEPUS Ace5, and MOTO G100—as mobile terminal devices (hereinafter referred to as devices 1 to 5). Two typical indoor environments—a laboratory and a conference room—were selected for the experiment. The specific settings were as follows: In each scenario, for each mobile terminal, 100 acoustic sample pairs were collected at different oriented angles from a nearby location (less than 20cm from the login device). Simultaneously, when the mobile terminal was used as a registration device, an additional 100 sample pairs were collected from a non-nearby location (greater than 20cm from the login device). The sample allocation scheme was as follows: 20 sample pairs were randomly selected from both the nearby and non-nearby locations for training the authentication model (i.e., the device registration phase); the remaining samples were used for subsequent experimental performance evaluation.

[0071] Figures 7 and 8 show the experimental results for the registration device identification pass rate and the registration device proximity determination pass rate, respectively. Specifically, when the device identification threshold is set to 0.9, the identification pass rates for mobile terminals 1 to 5 as registration devices are 0.93, 0.99, 1.0, 0.92, and 0.97, respectively. These results demonstrate that the registration device identifier proposed in this invention can accurately identify registration devices and possesses excellent device differentiation capabilities. Furthermore, under the same threshold (0.9), the proximity determination pass rates for the five mobile terminals as registration devices are 0.94, 0.98, 0.98, 0.97, and 0.98, respectively, indicating that the location proximity identifier of this invention can accurately determine whether the registration device and the login device are in a proximity state, and the location differentiation accuracy meets the requirements of practical applications.

[0072] To verify the anti-attack performance of the authentication method, this embodiment further tested its defense against replay attacks. The experiment was set up as follows: each mobile terminal was sequentially used as the target of the attack, and four other unregistered mobile terminals played pre-recorded acoustic sensing signals played by the target devices near the login device, simulating a replay attack scenario. Figure 9 shows the false recognition rate results for each attacked mobile terminal and the corresponding unregistered device. The average false recognition rates for mobile devices 1-5 under different unregistered device replay attacks were 0.03, 0.04, 0.04, 0.02, and 0.03, respectively. This indicates that the authentication method proposed in this invention has a significant defense effect against replay attacks initiated by unregistered devices and exhibits high security.

[0073] Finally, this embodiment tested the authentication method's resistance to co-location attacks. The attack scenario was set as follows: an attacker had stolen the victim's account and password and attempted to log in on the login device; at this time, the victim's registration device and login device were in the same room, but not within a proximity range (>20cm). Figure 10 shows the experimental results of the false acceptance rate for each registration device in this scenario. The average false acceptance rates for mobile devices 1-5 under different non-proximity location states were 0.02, 0.03, 0.04, 0.01, and 0.02, respectively. These results fully demonstrate that the authentication method of the present invention can effectively resist co-location attacks, further ensuring the security and reliability of identity authentication.

[0074] In this embodiment, the mobile terminal carried by the registered user can be a portable smart device with integrated speakers and microphones, such as a smartphone or smartwatch. The login device can be a laptop, desktop computer, in-vehicle system, or server. If the login device does not have an integrated microphone, additional adaptation is required.

Claims

1. A fast two-factor authentication method based on device proximity, characterized in that: Step 1: Generation of acoustic sensing signals; Step 2: Environmental multipath sample collection; Registration phase: The user places the mobile terminal to be registered in a location near and not near the login device, i.e., at a distance of ≥20cm. The mobile terminal plays the acoustic sensing signal generated in Step 1. The acoustic signals recorded by the microphones of the mobile terminal and the login device are collected simultaneously and form a sample pair. A location proximity label and a registration terminal label are added to form a training sample set; Authentication phase: The user places the registered mobile terminal in a location near the login device and collects a set of the above acoustic signal sample pairs; Step 3: Signal preprocessing and segmentation; Environmental noise elimination: Based on the preset frequency band range of the acoustic sensing signal in Step 1, an FIR bandpass filter with a Hamming window is designed to perform bandpass filtering on the acquired acoustic signal to eliminate out-of-band noise; Signal segmentation: The target signal interval is located by cross-correlation method, and the starting position of the signal is determined by threshold screening, and segments are cut according to the preset length. Step 4: Authentication Model Training; The authentication model training network consists of three main modules: a feature extractor, a location proximity recognizer, and a registered terminal recognizer. First, the feature extractor consists of two weight-sharing sub-networks. The input signal segment is processed by a short-time Fourier transform, and the output is a standardized compressed feature. Second, the location proximity recognizer calculates the distance between samples and corresponding compressed features based on Euclidean distance. It uses a contrastive loss function to aggregate features of registered devices in nearby locations and separate features corresponding to non-nearby locations or non-registered devices. Then, the registered terminal recognizer consists of two fully connected layers. It uses a cross-entropy loss function to enhance the extraction of device acoustic fingerprint features, achieving accurate identification of registered and non-registered devices. Finally, a weighted joint loss function is constructed during joint training, and after training, the user account is bound and stored in the authentication model database. Step 5: Device Identity Authentication; After the first authentication, i.e., the account and password are passed, the second authentication is initiated. The above preprocessing and segmentation process is performed on the collected sample pairs; the authentication model bound to the user is retrieved, and the processed features are input into the model; first, the mobile terminal recognizer is used for identification. If the predicted probability of "registered terminal" is less than the preset threshold, authentication is rejected; otherwise, the location proximity recognizer is used for further judgment. If the predicted probability of "location proximity" is greater than the preset threshold, authentication is passed; otherwise, it is rejected.

2. The method according to claim 1, characterized in that: Step 1: Generation of Acoustic Sensing Signals Selecting a linear frequency modulated signal with both wide frequency range and high autocorrelation characteristics for acoustic sensing signals The discrete real number expression for modulation is: ， in: The amplitude of the signal; The index is the sampling point index, and satisfies , The total number of sampling points, and , The duration of the signal. Sampling rate; and These are the start frequency and end frequency of the signal, respectively; the start frequency... With termination frequency Set the frequencies to 12kHz and 20kHz respectively; set the signal duration T to 500ms.

3. The method according to claim 2, characterized in that: Step 2: Environmental multipath sample collection Both the device registration and device authentication phases require the acoustic sensing signals generated in step 1. The user performs the environmental multipath sample collection operation using a mobile terminal device that is to be registered or has already been registered. The specific process of the sample collection stage is as follows: the mobile terminal device's speaker actively plays the pre-generated acoustic sensing signal. The signal propagates through both the direct propagation path and the environmental multipath reflection path, and is then collected by the built-in microphone of the mobile terminal device and the microphone of the login device, respectively. The corresponding acoustic samples are denoted as follows: That is, mobile terminal device data collection and... That is, the data collected from the logged-in device and together they form a set of sample pairs. ; During the registration phase, users need to place the mobile device to be registered in both the "nearby" and "non-nearby" locations of the login device, and add location proximity status tags to the collected samples. Correspondingly, the user-provided first A registered sample can be represented as ;when Then the sample corresponds to the sample pair collected when the mobile terminal to be registered and the login device are in a state of proximity; if This corresponds to the sample pair collected when the two are not in close proximity. In addition, in order to eliminate the interference of propagation distance and multipath effect on the extraction of speaker frequency response features, when the two are not in close proximity, the user needs to collect sample pairs when the mobile terminal to be registered and the login device are at different distances. Environmental sample collection process during device certification: Users only need to place their registered mobile terminal near the login device to collect a set of sample pairs.

4. The method according to claim 1, characterized in that: Step 3: Signal preprocessing and segmentation Step 3.1: Environmental noise elimination Samples collected from mobile registration terminals and login devices were processed using bandpass filters. , Bandpass filtering is applied to the discrete-time index to eliminate environmental noise; Based on the preset acoustic sensing signal frequency range in step 1 By combining window functions, an FIR bandpass filter is designed, and the discrete-time impulse response corresponding to the filter is obtained. ,in Let f(x) be the discrete index of the impulse response, representing the i-th element in the impulse response. Each sample point is then processed through temporal convolution. Noise cancellation is performed to obtain the corresponding signal segment after out-of-band noise removal. and ;Right now: in The length of the excitation response of the FIR filter; For temporary summation variables, and ; Step 3.2: Signal Segmentation The audio samples obtained after noise removal in step 3.1 are the denoised samples for the registered terminal device. Denoising samples for login devices The precise segmentation is performed as follows: Denoising acoustic samples , ,when for or These represent the samples collected by the corresponding mobile registration terminals. Or collected by the login device ) and pre-generated acoustic sensing signals Perform cross-correlation operations: ;in, For the delayed index in the cross-correlation output results; For temporary summation variables, and For the denoised acoustic samples Total sample size; cross-correlation output results Peak corresponding sample The starting position of the acoustic sensing signal is determined. To eliminate false detections caused by residual noise interference, a peak detection strategy of "threshold constraint + main peak selection" is adopted below: First, calculate the global maximum value of the cross-correlation results. Secondly, set a threshold. ,in An empirical threshold coefficient is set to 0.8; then, for Perform a traversal and filter out the first value exceeding... The peak that satisfies the condition of "monotonically decreasing values ​​in its left and right neighborhoods" has a corresponding delay index that is the starting position of the target signal, denoted as . Finally, starting from the beginning position. Based on the discrete length of the acoustic sensing signal in step 1, Extract segments to obtain the segmented segments corresponding to the registered terminal device and login device samples. , This represents the number of samples of the acoustic sensing signal generated in step 1. and These represent the mobile registration terminal and login device corresponding to the sample, respectively. Indicates from Extracting from signal samples arrive The signal segment.

5. The method according to claim 1, characterized in that: Step 4: Training the Authentication Model During the device registration phase, the acoustic samples submitted by the user are processed sequentially using the environmental noise cancellation and signal segmentation steps in step 3 to obtain the corresponding target acoustic signal segments. ),in This represents the acoustic signal segment collected by the logged-in device. This represents the acoustic signal segment collected by the registered terminal device; a short-time Fourier transform is performed on the aforementioned acoustic signal segment. ), generating the corresponding time-spectrum features The spectral features of that time are then input into the authentication model constructed in this step for training. The authentication model training network architecture includes three core functional modules: a feature extractor, a location proximity recognizer, and a registered device recognizer. The specific composition and working mechanism of each module are detailed below: Step 4.1: Feature Extractor The feature extractor consists of two structurally consistent sub-networks with shared weights to achieve consistent extraction of acoustic features from login devices and mobile terminals. Each sub-network comprises three cascaded convolutional blocks, each containing a 2D convolutional layer, a batch normalization layer, a ReLU activation function layer, and a max-pooling layer. First, the input sample... , First, a two-dimensional convolutional layer is passed in for the first convolutional block. This layer is configured with 32 elements of size 1. A convolutional kernel with a stride of 1 is used to perform spatial feature mapping and compression on the input features through convolution operations, resulting in an initial feature map. This initial feature map is then processed by a batch normalization layer for data distribution standardization, followed by a ReLU activation function layer for non-linear feature transformation, and finally passed through a layer with a size of [missing information - likely a specific value]. The max pooling layer, with a stride of 2, performs downsampling to reduce feature dimensionality while preserving key feature information; the output feature map of the first convolutional processing submodule is fed into the two-dimensional convolutional layer of the second convolutional processing submodule, which is configured with 64 pixels of size 1. With a stride of 1, after repeatedly executing the above "convolution operation - batch normalization - ReLU activation" process, a convolutional kernel of the same size as the first convolutional block is used. A max-pooling layer with a stride of 2 is used for downsampling to further enhance the abstract representation of features; the output feature map of the second convolutional processing submodule is fed into the two-dimensional convolutional layer of the third convolutional processing submodule, which is configured with 128 pixels of size 128. After performing the "convolution operation - batch normalization - ReLU activation" process again with a convolution kernel of size 1, the kernel size is also used. Downsampling is performed using a max-pooling layer with a stride of 2, and the final compressed feature representation is output. ; Step 4.2: Location Proximity Identifier: The location proximity detector uses a fully connected layer with 512 neurons to transform the high-dimensional local features of login device samples and mobile phone collected samples into a compact low-dimensional feature vector, which facilitates the evaluation of the distance between features. The specific implementation is as follows: Define the training sample pair set as ;in, For training sample pair indexes, The total number of sample pairs to be trained; For location proximity labels; when the first When the corresponding "login device" and "mobile phone" of a sample pair are within a preset proximity range, ;otherwise Register tags for mobile terminals; when the first When the sample group corresponds to the "phone number" used for registration, ;otherwise For the first Group sample pairs In weight The Euclidean distance is given below; at this point, the loss function of the proximity recognizer is... for in, A preset interval threshold is used to define the lower limit of the feature distance for "non-locationally proximate" sample pairs; when the sample pair is provided by the registered terminal device in a locationally proximate state, then... The loss function will minimize This makes the two sets of feature vectors as close as possible; when the sample pair comes from a non-registered terminal device or a registered terminal device that is not in a location proximity state. The loss function will minimize That is, the distance between two sets of feature vectors must be no less than [a certain value]. To achieve effective separation; Step 4.3: Device Identifier: The network structure of the registered terminal device identifier consists of two fully connected layers: the first layer has 512 neurons, used for high-dimensional mapping of the compressed features output by the feature extractor; the second layer has 2 neurons, corresponding to the two categories of labels: "registered terminal device" and "unregistered terminal device"; its training process is as follows: during the training phase, acoustic samples collected by unregistered terminal devices are introduced as negative samples, which together with the registered terminal device samples submitted by the user (i.e., positive samples) constitute the training set; the login device samples output by the feature extractor... Features of a sample The input registered terminal device identifier is mapped through a fully connected layer to obtain a 2D feature vector, which is then converted into a class prediction probability vector through a softmax function. ; will the first The true label for each sample is That is, converting registered terminal devices or unregistered terminal devices into one-hot encoded vectors. The cross-entropy loss function is used to calculate the deviation between the predicted and actual values. Defined as: in For the total number of tamed samples, For training sample index, and They represent the first One-hot encoded vectors corresponding to each sample With category prediction probability vector The The probability corresponding to the dimension; Step 4.4: Joint Model Training To achieve coordinated optimization of location proximity recognition and registered terminal device recognition, this step constructs a weighted joint loss function. The formal definition is as follows: and These are the weight coefficients, used to balance the training priorities of the two tasks, set to 0.65 and 0.35 respectively; The sample collection process for model training is as follows: First, users place their registered mobile devices in the "nearby login devices" area. "and non-proximity login devices" At the location of "", multiple sets of acoustic sample pairs are collected to form a positive sample set; then, the service provider provides multiple unregistered mobile terminals to collect samples from the "nearby login devices". "and non-proximity login devices" Multiple acoustic sample pairs are collected at the location to form a negative sample set. After training, the optimized feature extractor, location proximity recognizer and registered terminal device recognizer are bound to the user account and stored in the authentication model database for subsequent device identity authentication.

6. The method according to claim 1, characterized in that: Step 5: Device Identification In the two-factor authentication process, once the first authentication credentials, namely account and password verification, are successful, the second authentication, namely device authentication, is automatically initiated. First, the legitimate user places the mobile terminal used during the registration phase in a preset proximity position to the login device. The mobile terminal's speaker plays a preset acoustic sensing signal, and the login device and the mobile terminal's microphones simultaneously record this signal, forming a multipath acoustic sample pair. The collected multipath acoustic sample pairs are then processed sequentially using step 3, namely noise cancellation and signal segmentation, to obtain the corresponding target acoustic signal segments. Based on the user account that has passed the first authentication, the authentication model corresponding to the user is retrieved from the authentication model database. The authentication model includes a feature extractor, a registration device identifier, and a location proximity identifier. After the segmented acoustic signal segments are converted into time-spectral features, they are input into the retrieved feature extractor to obtain a compressed feature representation. Then, the obtained compressed feature representation is input into the registered device identifier to obtain the predicted probability of the "registered device" category. If the predicted probability is less than a certain threshold, the device authentication is directly rejected. Otherwise, the obtained compressed feature representation is input into the location proximity identifier to obtain the predicted probability of the "location proximity" relationship. If the predicted probability is greater than a certain threshold, the device authentication is successful. Otherwise, the device authentication fails.

Images