An implicit identity authentication method based on multi-modal data fusion and related devices

Abstract

The application provides a kind of implicit identity authentication method and related device based on multi-modal data fusion, belong to identity authentication technical field.The application carries out segmented processing to several types of motion sensor data after preprocessing, obtains several fixed length data segments;Respectively carry out feature extraction to one touch screen sensor data and several fixed length data segments after abnormal data processing, obtain space-time feature and several touch screen sensor features;The space-time feature and several touch screen sensor features are input into the improved Transform network, and the segment representation of the fused behavior semantics is obtained;The segment representation of the fused behavior semantics is input into the improved SVDD network, and the implicit identity authentication result is obtained.The application solves the problem that the accuracy of implicit identity authentication is not high.

Description

Technical Field

[0001] This invention belongs to the field of identity authentication technology, specifically relating to an implicit identity authentication method and related apparatus based on multimodal data fusion. Background Technology

[0002] With the rapid development of internet technology, smartphones have become an important tool for interactive communication, enabling users to transmit various multimedia information (such as text, audio, video, and entertainment content) and provide services such as online shopping, information browsing, and financial transactions. In providing these services, users generate a large amount of confidential and personal data, which is stored on their smartphones. However, because smartphones are vulnerable to unauthorized access or theft, the security and protection of this sensitive information and data have become particularly important.

[0003] Currently, smartphone privacy protection primarily relies on explicit authentication mechanisms, including knowledge-based methods (such as passwords, PINs, and pattern unlocking) and biometric-based methods (such as fingerprints and facial recognition). However, these authentication methods have many limitations, such as single-point-of-sight authentication, heavy user input burden, and vulnerability to shoulder-peeping attacks. These issues not only affect user experience but may also lead to serious privacy breaches.

[0004] To address the aforementioned issues, implicit authentication has gained increasing attention. This technology transparently and implicitly detects whether a logged-in user is the true owner of the smartphone by capturing unique behavioral biometrics of the user's interactions with the smartphone. Implicit authentication offers several advantages: First, even if an attacker has obtained the user's smartphone password (e.g., PIN code) and has physical access, this method can still perform transparent authentication by using behavioral patterns during the session, thus preventing attackers from accessing information and data stored on the phone. Second, behavioral biometrics are difficult to replicate, significantly reducing the risk of identity theft compared to traditional PIN code authentication. However, current implicit authentication methods still have limitations in handling unimodal and multimodal heterogeneous data.

[0005] Single-modal authentication methods typically utilize data collected from different sensors (such as motion sensors and touchscreen sensors) to model user behavior. Among these, authentication methods based on motion sensors (such as accelerometers and gyroscopes) can capture the smartphone's movement state through continuous monitoring of motion sensor data, enabling authentication whether or not a touchscreen is present. However, in real-world environments, motion sensor data is easily affected by various factors, such as user activity state (e.g., walking), body posture (e.g., sitting and lying down), and psychophysiological state (e.g., tension, relaxation), making it difficult to construct robust behavior patterns. To improve authentication accuracy, some researchers have imposed strict constraints on the usage environment (e.g., desktop holding, sitting / walking, and dynamic / semi-static / static scenarios), but these limitations make it difficult for existing authentication systems to provide stable identity authentication in practical applications. On the other hand, authentication methods based on touchscreen sensor data also have two main drawbacks. First, in short-duration touchscreen operations (e.g., clicking), there is limited touch feature information, making it difficult to accurately model user behavior and thus limiting authentication performance. Secondly, this method only performs authentication when a touch screen event occurs, making it difficult to cope with scenarios where attackers reduce touch screen operations, thus affecting the accuracy of identity authentication.

[0006] To address the aforementioned issues, several identity authentication methods based on multimodal data have emerged in recent years. These methods typically extract features from data across different modalities and simply concatenate them into a feature vector for authentication. However, current multimodal data fusion methods often employ simple concatenation or static weighting strategies, failing to fully consider the heterogeneity and misalignment between different modalities in sampling frequency, triggering mechanisms, and behavioral semantics. In particular, motion sensor data exhibits high-frequency continuous characteristics, while touchscreen sensor data is low-frequency, sparse, event-driven data. The two differ significantly in time scale and behavioral granularity; direct concatenation often leads to semantic misalignment and information redundancy, limiting the discriminative power of the fusion model and affecting the accuracy of implicit identity authentication. Summary of the Invention

[0007] The purpose of this invention is to provide an implicit identity authentication method and related apparatus based on multimodal data fusion, which solves the problem of low accuracy of implicit identity authentication in the prior art.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides an implicit identity authentication method based on multimodal data fusion, comprising the following steps:

[0010] Acquire data from several types of motion sensors and one type of touchscreen sensor;

[0011] The obtained motion sensor data of several types are preprocessed to obtain preprocessed motion sensor data of several types; the preprocessing refers to first performing noise reduction and filtering, and then performing standardization processing after noise reduction and filtering.

[0012] Anomaly processing is performed on the obtained touch screen sensor data to obtain anomaly-processed touch screen sensor data.

[0013] The preprocessed motion sensor data of several types is segmented to obtain several data segments of fixed length;

[0014] Feature extraction is performed on a type of touchscreen sensor data and several fixed-length data segments after abnormal data processing to obtain spatiotemporal features and several touchscreen sensor features.

[0015] Spatiotemporal features and features from several touchscreen sensors are input into an improved Transformer network to obtain a fragment representation that fuses behavioral semantics.

[0016] The improved Transformer network includes:

[0017] An improvement to the computational method of the attention mechanism in Transformer networks is based on decaying kernel functions and behavioral event-guided scores.

[0018] By inputting the fragment representation of the fused behavioral semantics into the improved SVDD (Support Vector Data Description) network, implicit identity authentication results are obtained.

[0019] The improved SVDD network includes:

[0020] Deep neural networks are used to nonlinearly map raw user behavior data to a high-dimensional feature space.

[0021] Construct a minimum envelope hypersphere in a high-dimensional feature space;

[0022] Update the parameters of the deep neural network and the hypersphere parameters.

[0023] A further improvement of the present invention is that the various types of motion sensor data include accelerometer sensor data, gyroscope sensor data, and magnetometer sensor data.

[0024] A further improvement of the present invention is that the feature extraction is specifically performed as follows: a convolutional neural network is used to extract features from several data segments of fixed length to obtain spatiotemporal features, and a statistical method is used to extract features from a touch screen sensor data after abnormal data processing to obtain several touch screen sensor features.

[0025] A further improvement of the present invention is that the touch screen sensor features include position features, length features, speed features, angle features, and pressure features.

[0026] A further improvement of this invention is that the calculation formula for the fragment representation of fused behavioral semantics is as follows:

[0027]

[0028] Where h represents the fragment representation of the final fused multimodal behavioral data, and W :,k Let V be the attention weight vector, representing the contribution of the k-th segment to the overall representation. k Let K represent the behavioral features of the k-th segment, where K is the total number of segments;

[0029] Attention weight vector W :,k The calculation formula is:

[0030]

[0031] in, An attention score matrix to incorporate behavioral event-guided scores;

[0032] Attention score matrix with behavioral event guidance scores The calculation formula is:

[0033]

[0034] Among them, A :,k Let α be the attention score for the k-th segment in the standard Transformer. k Scores are assigned to behavioral events;

[0035] Behavioral event guidance score α k The calculation formula is:

[0036]

[0037] Where, ω α,r For learnable event weights, b α As a bias term, G r,j ∈[0,1]^(2×K) is the event guidance matrix, representing the degree of influence of the r-th event on the j-th segment, and K r (k,j) is the decaying kernel function;

[0038] decaying kernel function K r The expression for (k,j) is:

[0039]

[0040] Where k is the current segment index, j is the segment index used for correlation modeling, and R r The decay radius is the radius of the r-th type of behavior, used to control the rate at which the correlation decays with the segment spacing. App refers to App events and Touch refers to touch events.

[0041] A further improvement of this invention is that the formula for calculating the minimum envelope hypersphere is:

[0042]

[0043] Among them, h i Let F(h) be the i-th input sample, i = 1, ..., N, where N is the total number of training samples. i W) is the feature mapping function of the deep neural network, defined by the parameter set W = {W} k} control, input sample h i Mapped to a high-dimensional feature space, c is the hypersphere center vector, used to describe the cluster centers of the data in the high-dimensional feature space, |F(h i ;W)-c| 2 W is the squared Euclidean distance from the i-th sample to the center of the hypersphere, used to measure the deviation of the sample from the center of the hypersphere. λ is the regularization coefficient, used to balance the fitting error term and the regularization term. k | 2 For network parameters W k The square of the second norm is used as a regularization constraint.

[0044] A further improvement of the present invention is that the updating of the parameters of the deep neural network and the hypersphere parameters is specifically achieved by using an alternating optimization strategy to update the parameters of the deep neural network and the hypersphere parameters.

[0045] Secondly, the present invention provides an implicit identity authentication system based on multimodal data fusion, comprising:

[0046] The data acquisition module is used to acquire data from several types of motion sensors and data from a touch screen sensor.

[0047] The preprocessing module is used to preprocess the obtained motion sensor data of several types to obtain preprocessed motion sensor data of several types; the preprocessing refers to first performing noise reduction and filtering, and then performing standardization processing after noise reduction and filtering.

[0048] An abnormal data processing module is used to process the obtained touch screen sensor data to obtain abnormal touch screen sensor data after abnormal data processing.

[0049] The segmentation processing module is used to segment the pre-processed motion sensor data of several types to obtain several data segments of fixed length.

[0050] The feature extraction module is used to extract features from a type of touch screen sensor data and several fixed-length data segments after abnormal data processing, to obtain spatiotemporal features and several touch screen sensor features.

[0051] The feature input module is used to input spatiotemporal features and features from several touchscreen sensors into an improved Transformer network to obtain a fragment representation that fuses behavioral semantics.

[0052] The improved Transformer network includes:

[0053] An improvement to the computational method of the attention mechanism in Transformer networks is based on decaying kernel functions and behavioral event-guided scores.

[0054] The implicit identity authentication module is used to input the fragment representation of fused behavioral semantics into the improved SVDD network to obtain the implicit identity authentication result;

[0055] The improved SVDD network includes:

[0056] Deep neural networks are used to nonlinearly map raw user behavior data to a high-dimensional feature space.

[0057] Construct a minimum envelope hypersphere in a high-dimensional feature space;

[0058] Update the parameters of the deep neural network and the hypersphere parameters.

[0059] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the implicit identity authentication method based on multimodal data fusion described above.

[0060] Fourthly, the present invention provides a storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the implicit identity authentication method based on multimodal data fusion described above.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] The implicit identity authentication method proposed in this invention based on multimodal data fusion improves the calculation method of the Transformer network attention mechanism by using a decaying kernel function and behavioral event guidance scores. This operation can make the user behavior modeling for a period of time during each authentication input more accurate. On the other hand, it uses a deep neural network to nonlinearly map the user's original behavior data to a high-dimensional feature space, constructs a minimum envelope hypersphere in the high-dimensional feature space, and updates the parameters of the deep neural network and the hypersphere parameters, thereby improving the SVDD network. This operation can make the overall user behavior modeling more accurate, thus effectively solving the problem of low accuracy of implicit identity authentication in the prior art. Attached Figure Description

[0063] Figure 1 This is a flowchart of the implicit identity authentication method based on multimodal data fusion according to the present invention;

[0064] Figure 2 This is a schematic diagram of the implicit identity authentication system based on multimodal data fusion according to the present invention;

[0065] Figure 3 This is a flowchart of the implicit identity authentication method based on multimodal data fusion in Embodiment 3 of the present invention;

[0066] Figure 4 This is a schematic diagram of the structure of the electronic device of the present invention. Detailed Implementation

[0067] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0068] Example 1:

[0069] The flowchart of the implicit identity authentication method based on multimodal data fusion of this invention is as follows: Figure 1 As shown, the implicit identity authentication method based on multimodal data fusion of the present invention includes the following steps:

[0070] S1. Acquire data from several types of motion sensors and one type of touchscreen sensor;

[0071] S2. Preprocess the obtained motion sensor data of several types to obtain preprocessed motion sensor data of several types; the preprocessing refers to first performing noise reduction and filtering, and then performing standardization processing after noise reduction and filtering;

[0072] S3. Perform abnormal data processing on the obtained touch screen sensor data to obtain abnormal data processed touch screen sensor data.

[0073] S4. The preprocessed motion sensor data of several types is segmented to obtain several data segments of fixed length;

[0074] S5. After processing the abnormal data, perform feature extraction on one type of touch screen sensor data and several fixed-length data segments to obtain spatiotemporal features and several touch screen sensor features;

[0075] S6. Input the spatiotemporal features and several touchscreen sensor features into the improved Transformer network to obtain a fragment representation of fused behavioral semantics;

[0076] The improved Transformer network includes:

[0077] An improvement to the computational method of the attention mechanism in Transformer networks is based on decaying kernel functions and behavioral event-guided scores.

[0078] S7. Input the fragment representation of the fused behavioral semantics into the improved SVDD network to obtain the implicit identity authentication result;

[0079] The improved SVDD network includes:

[0080] Deep neural networks are used to nonlinearly map raw user behavior data to a high-dimensional feature space.

[0081] Construct a minimum envelope hypersphere in a high-dimensional feature space;

[0082] Update the parameters of the deep neural network and the hypersphere parameters.

[0083] In step S6 of this invention, the calculation method of the attention mechanism of the Transformer network based on the decaying kernel function and the behavior event guidance score is improved. This operation can make the user behavior modeling for a period of time for each authentication input more accurate. In step S7, the user's original behavior data is nonlinearly mapped to a high-dimensional feature space using a deep neural network; a minimum envelope hypersphere is constructed in the high-dimensional feature space; the parameters of the deep neural network and the hypersphere parameters are updated to improve the SVDD network. This operation can make the overall user behavior modeling more accurate.

[0084] Example 2:

[0085] A schematic diagram of the implicit identity authentication system based on multimodal data fusion of this invention is shown below. Figure 2 As shown, the implicit identity authentication system based on multimodal data fusion of the present invention includes:

[0086] The data acquisition module is used to acquire data from several types of motion sensors and data from a touch screen sensor.

[0087] The preprocessing module is used to preprocess the obtained motion sensor data of several types to obtain preprocessed motion sensor data of several types; the preprocessing refers to first performing noise reduction and filtering, and then performing standardization processing after noise reduction and filtering.

[0088] An abnormal data processing module is used to process the obtained touch screen sensor data to obtain abnormal touch screen sensor data after abnormal data processing.

[0089] The segmentation processing module is used to segment the pre-processed motion sensor data of several types to obtain several data segments of fixed length.

[0090] The feature extraction module is used to extract features from a type of touch screen sensor data and several fixed-length data segments after abnormal data processing, to obtain spatiotemporal features and several touch screen sensor features.

[0091] The feature input module is used to input spatiotemporal features and features from several touchscreen sensors into an improved Transformer network to obtain a fragment representation that fuses behavioral semantics.

[0092] The improved Transformer network includes:

[0093] An improvement to the computational method of the attention mechanism in Transformer networks is based on decaying kernel functions and behavioral event-guided scores.

[0094] The implicit identity authentication module is used to input the fragment representation of fused behavioral semantics into the improved SVDD network to obtain the implicit identity authentication result;

[0095] The improved SVDD network includes:

[0096] Deep neural networks are used to nonlinearly map raw user behavior data to a high-dimensional feature space.

[0097] Construct a minimum envelope hypersphere in a high-dimensional feature space;

[0098] Update the parameters of the deep neural network and the hypersphere parameters.

[0099] Example 3:

[0100] The architecture diagram of the implicit identity authentication method based on multimodal data fusion of this invention is as follows: Figure 3 As shown, the implicit identity authentication method based on multimodal data fusion of the present invention includes the following steps:

[0101] This embodiment specifically uses implicit identity authentication of smart terminal users (such as mobile phones and smart bands) as an example.

[0102] S1. Acquire data from several types of motion sensors and one type of touchscreen sensor.

[0103] First, acquire data from several types of motion sensors and one type of touchscreen sensor.

[0104] Several types of motion sensor data include accelerometer sensor data, gyroscope sensor data, and magnetometer sensor data.

[0105] This step may also include several types of motion sensor data, including rotation vector meter sensor data, which can be selected according to actual needs.

[0106] S2. Preprocess the obtained motion sensor data of several types to obtain preprocessed motion sensor data of several types.

[0107] Preprocessing refers to denoising and filtering first, followed by standardization.

[0108] Specifically, wavelet denoising (to eliminate environmental vibration interference) and Kalman filtering (to suppress sensor drift) are first applied to the obtained motion sensor data of several types and touch screen sensor data. After wavelet denoising and Kalman filtering, normalization processing (Z-score normalization) is performed.

[0109] S3. Perform abnormal data processing on the obtained touch screen sensor data to obtain abnormal data processed touch screen sensor data.

[0110] Specifically, if any of the collected touchscreen sensor data is missing, then that touchscreen sensor data will be removed.

[0111] S4. The preprocessed motion sensor data of several types is segmented to obtain several data segments of fixed length.

[0112] Specifically, the overlapping sliding window method is used to divide the continuous sensor data sequence (preprocessed motion sensor data of several types) into equal-length time periods (several fixed-length data segments).

[0113] Each data segment consists of 100 data points, which is equivalent to 2 seconds of data at a sampling rate of 50 Hz.

[0114] S5. After processing the abnormal data, perform feature extraction on one type of touch screen sensor data and several fixed-length data segments to obtain spatiotemporal features and several touch screen sensor features;

[0115] This step specifically uses a convolutional neural network (one-dimensional convolutional neural network) to extract features from several fixed-length data segments. Figure 3 (Using encoding to represent), we obtain the spatiotemporal characteristics.

[0116] Convolutional neural networks (CNNs) have multiple convolutional layers and max-pooling layers, which can effectively capture the spatial relationships and temporal patterns of motion sensor data. After processing by the convolutional layers, the motion sensor data is transformed into low-dimensional embedding vectors, representing the motion features within that time period.

[0117] This step specifically employs statistical methods to extract features from the processed abnormal data of a touchscreen sensor, resulting in several touchscreen sensor features. These features include position, length, velocity, angle, and pressure characteristics. The position, length, velocity, angle, and pressure characteristics are explained in detail below:

[0118] Positional features refer to the spatial positional information generated when a user interacts with a touchscreen. Assume the starting coordinates of a touch operation are (x...). start ,y start The termination coordinate is (x) end ,y end The formula for calculating location features is:

[0119] h loc =[x start ,y start ,x end ,y end ]

[0120] Among them, h loc This refers to location features.

[0121] Length features are used to measure the degree and direction of finger movement during a single touch operation, reflecting the stability and style differences of the user's operation.

[0122] This step extracts two key length features: the movement length and the displacement length.

[0123] Movement length represents the total length of the actual movement trajectory of a user's finger on the screen during a single swipe, reflecting the physical length of the entire swipe path. Displacement length refers to the Euclidean distance between the starting and ending points of the swipe operation, characterizing the directional intensity or "straightness" of the overall swipe.

[0124] Assume a single touch trajectory consists of a series of consecutive touch points, denoted as {(x1,y1),(x2,y2),…,(x... n ,y nThe formulas for calculating the translation length and displacement length are as follows:

[0125]

[0126] L disp =(x n -x1) 2 +(y n -y1) 2

[0127] Among them, L move L represents the length of the movement, reflecting the degree of tortuosity of the entire trajectory. disp The displacement length reflects the overall direction and coverage of the touch operation. Combining these two characteristics effectively characterizes differences in user behavior during swipes. For example, users accustomed to continuous, rapid swipes exhibit a larger ratio of movement length to displacement length, while users performing stable, straight-line swipes show a similar ratio. These length features enhance the ability of subsequent SVDD networks to recognize personalized swipe behaviors without increasing dimensionality, demonstrating good discriminative and interpretable capabilities.

[0128] Speed ​​features are used to describe the speed and rhythm of a user's finger swiping during touch operations, reflecting an individual's operating habits and behavioral rhythm during interaction. To comprehensively model the user's swiping speed pattern, this embodiment extracts speed features from two levels: first, statistical features of the speed sequence based on the swiping trajectory; and second, global speed features based on the overall swiping process.

[0129] First, assume the touch trajectory consists of n consecutive touch points with timestamps t1, t2, ..., t3. n The coordinates are {(x1,y1),(x2,y2),…,(x... n ,y n The formula for calculating the instantaneous velocity between each pair of adjacent contacts is: This yields the velocity sequence {v1, v2, ..., v n-1 In this embodiment, several statistical features (such as mean, standard deviation, maximum and minimum values) are calculated for the sequence as velocity features describing the change pattern of finger sliding speed.

[0130] Secondly, in order to better capture the overall speed of a complete sliding operation, this embodiment introduces two global speed metrics:

[0131] Average movement speed (based on total distance traveled):

[0132] Average displacement velocity (based on starting and ending point displacements):

[0133] By combining the volatility of instantaneous velocity sequences with the stability of global average velocity, the extracted velocity features can comprehensively capture the dynamic features during user operations, which helps to enhance the discriminativeness and robustness in identity recognition or behavior modeling.

[0134] Angular features are used to characterize the direction and trajectory deflection of a user's finger movement on a touchscreen, effectively reflecting the directional characteristics and behavioral trajectory patterns of the operation. This embodiment extracts two angle sequences as feature bases: phase angle sequences and diagonal sequences, and performs statistical feature extraction on them.

[0135] First, the phase angle sequence refers to the angle between the line connecting the screen coordinate origin (usually set to the lower left corner) to each touch point location and the horizontal direction. For touch sample points (x... i ,y i ), its phase angle The calculation method is as follows: This angle reflects the finger's position relative to the entire screen and has stable global orientation characteristics.

[0136] Secondly, the diagonal sequence is used to describe the angle between the line connecting adjacent touch points and the horizontal direction (X-axis), which can reflect the directional change during the sliding process. For two consecutive touch points (x... i ,y i ) and (x i+1 ,y i+1 ), which are diagonally opposite The calculation formula is: This sequence depicts local directional changes, and is particularly effective at depicting tortuous trajectories and habitual sliding directions.

[0137] Finally, this embodiment extracts statistical features from the two types of angle sequences, including mean, maximum, minimum, and standard deviation, to form the final angle feature vector. These features not only reflect the overall trend of the sliding angle but also capture its range of variation and instability, which helps to improve the representation ability of user operation behavior and the discrimination performance of the SVDD network.

[0138] Pressure features are used to describe the differences in physical pressure applied by users when operating on a touchscreen, reflecting an individual's touchscreen habits and force control methods during mobile phone use. Different users often apply significantly different pressure values ​​when performing operations such as swiping, clicking, or long-pressing, which makes pressure features highly discriminative in individual identification and behavior modeling.

[0139] This embodiment extracts the pressure sequence recorded during each touch operation, i.e., the set of pressure values ​​corresponding to each touch point. To more comprehensively describe the overall characteristics of the user's force application behavior, this embodiment calculates several statistical indicators for this sequence, including the mean, maximum, minimum, and standard deviation of the pressure. The mean reflects the user's overall force level. The maximum value represents the maximum pressure applied in that operation, usually related to actions such as forceful sliding or long pressing. The minimum value describes the pressure value of the lightest touch. The standard deviation measures the degree of fluctuation in the pressure applied by the user during the operation, reflecting whether their operation is stable and uniform. These statistical features can effectively capture individual differences in the user's touch interaction process. Combined with dynamic features such as position and speed, they can further improve the performance of the SVDD network in identity recognition and behavior analysis.

[0140] S6. Input the spatiotemporal features and several touchscreen sensor features into the improved Transformer network to obtain a fragment representation of fused behavioral semantics.

[0141] The improved Transformer network in this step ( Figure 3 (represented by UBFormer in Chinese), including:

[0142] An improvement to the computational method of the attention mechanism in Transformer networks is proposed based on decaying kernel functions and behavioral event-guided scores.

[0143] The formula for calculating the fragment representation of fused behavioral semantics is as follows:

[0144]

[0145] Where h represents the fragment representation of the final fused multimodal behavioral data, and W :,k Let V be the attention weight vector, representing the contribution of the k-th segment to the overall representation. k Let K be the behavioral feature representation of the k-th segment, where K is the total number of segments.

[0146] Attention weight vector W :,k The calculation formula is:

[0147]

[0148] in, An attention score matrix to incorporate behavioral event-guided scores.

[0149] Attention score matrix with behavioral event guidance scores The calculation formula is:

[0150]

[0151] Among them, A:,k Let α be the attention score for the k-th segment in the standard Transformer. k Scores are assigned to behavioral events.

[0152] For all k, the formula for calculating the attention score matrix A in the standard Transformer is:

[0153]

[0154] Where Q, K, and V are the query, key, and value matrices obtained by linear transformation of the input fragment, respectively, and b is the vector dimension of the input fragment.

[0155] Behavioral event guidance score α k The calculation formula is:

[0156]

[0157] Where, ω α,r For learnable event weights, b α As a bias term, G r,j ∈[0,1]^(2×K) is the event guidance matrix, representing the degree of influence of the r-th event on the j-th segment, and K r (k,j) is the decay kernel function.

[0158] The event guidance matrix is ​​used to represent information about two types of behavioral events, with the first row being G. 1,k Indicates the importance of the k-th segment related to the App event, line 2 G 2,k Indicates the degree of relevance to the touch event.

[0159] decaying kernel function K r The expression for (k,j) is:

[0160]

[0161] Where k is the current segment index, j is the segment index used for correlation modeling, and R r The decay radius is the radius of the r-th type of behavior, used to control the rate at which the correlation decays with the segment spacing. App refers to App events and Touch refers to touch events.

[0162] S7. Input the fragment representation of the fused behavioral semantics into the improved SVDD network to obtain the implicit identity authentication result.

[0163] The improved SVDD network in this step includes:

[0164] A. Using a deep neural network F(h) i W) Nonlinearly maps raw user behavior data to a high-dimensional feature space;

[0165] B. Construct a minimum envelope hypersphere in a high-dimensional feature space;

[0166] C. Update the parameters of the deep neural network and the hypersphere parameters.

[0167] In step A, a deep neural network F(h) is used. i W) Nonlinearly maps the user's original behavior data to a high-dimensional feature space, resulting in the feature representation F(X) = [F(h1;W), F(h2;W), ..., F(h... N ;W)] T .

[0168] This feature mapping process fully leverages the multi-layered structure and non-linear activation functions (such as ReLU activation) of deep neural networks to extract deep behavioral features from interactive data such as touch gestures. Key hyperparameters of the deep neural network (such as the number of hidden layers, the number of neurons, and the learning rate) are optimized through random search to improve the performance of the SVDD network in identity authentication tasks.

[0169] The formula for calculating the minimum envelope hypersphere in step B is:

[0170]

[0171] Among them, h i Let be the i-th input sample (composed of multimodal sensor data), i = 1, ..., N, where N is the total number of training samples, F(h i W) is the feature mapping function of the deep neural network, defined by the parameter set W = {W} k} control, input sample h i Mapped to a high-dimensional feature space, c is the hypersphere center vector, used to describe the cluster centers of the data in the high-dimensional feature space, |F(h i ;W)-c| 2 Let |W| be the squared Euclidean distance from the i-th sample to the center of the hypersphere, used to measure the deviation of the sample from the center of the hypersphere, and λ be the regularization coefficient used to balance the fitting error term and the regularization term, thereby preventing overfitting of the deep neural network. k | 2 For network parameters W k The square of the second norm is used as a regularization constraint.

[0172] To simultaneously optimize the deep neural network parameters (weights W) and the hypersphere parameters (hypersphere center c), this embodiment employs an alternating optimization strategy for updates:

[0173] a. With the center of the hypersphere fixed, minimize the objective function (the formula for calculating the minimum envelope hypersphere) to update the parameters W of the deep neural network, so that the feature mapping result can be as close as possible to the current hypersphere boundary;

[0174] b. With the parameters W of the deep neural network fixed, recalculate and update the hypersphere center c so that it can better represent the center position of the current feature distribution;

[0175] c. Repeat the above two processes until the objective function converges.

[0176] By using an alternating optimization approach, it is possible to effectively ensure that the deep SVDD network can learn discriminative high-dimensional feature representations and obtain stable and reasonable hypersphere centers during training, thereby improving the accuracy and robustness of implicit identity authentication.

[0177] To prevent hypersphere collapse during SVDD network training (where all outputs converge to the center point, causing the hypersphere radius to approach zero), the SVDD network sets the hypersphere center *c* to the mean of the outputs mapped from a subset of training samples during the initialization phase. It also removes the bias term from the activation function to ensure the network output has a distributed nature. Furthermore, non-saturating activation functions such as ReLU are used to enhance the SVDD network's nonlinear modeling capabilities.

[0178] Example 4:

[0179] Please see Figure 4 As shown, the present invention also provides an electronic device 100 based on an implicit identity authentication method of multimodal data fusion; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0180] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the implicit identity authentication method based on multimodal data fusion described in Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0181] The at least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor. The processor 102 is the control center of the electronic device 100, connecting various parts of the electronic device 100 via various interfaces and lines.

[0182] The memory 101 in the electronic device 100 stores multiple instructions to implement an implicit authentication method based on multimodal data fusion, and the processor 102 can execute the multiple instructions to achieve the following:

[0183] Acquire data from several types of motion sensors and one type of touchscreen sensor;

[0184] The obtained motion sensor data of several types are preprocessed to obtain preprocessed motion sensor data of several types; the preprocessing refers to first performing noise reduction and filtering, and then performing standardization processing after noise reduction and filtering.

[0185] Anomaly processing is performed on the obtained touch screen sensor data to obtain anomaly-processed touch screen sensor data.

[0186] The preprocessed motion sensor data of several types is segmented to obtain several data segments of fixed length;

[0187] Feature extraction is performed on a type of touchscreen sensor data and several fixed-length data segments after abnormal data processing to obtain spatiotemporal features and several touchscreen sensor features.

[0188] Spatiotemporal features and features from several touchscreen sensors are input into an improved Transformer network to obtain a fragment representation that fuses behavioral semantics.

[0189] The improved Transformer network includes:

[0190] An improvement to the computational method of the attention mechanism in Transformer networks is based on decaying kernel functions and behavioral event-guided scores.

[0191] By inputting the fragment representation of fused behavioral semantics into the improved SVDD network, implicit identity authentication results are obtained.

[0192] The improved SVDD network includes:

[0193] Deep neural networks are used to nonlinearly map raw user behavior data to a high-dimensional feature space.

[0194] Construct a minimum envelope hypersphere in a high-dimensional feature space;

[0195] Update the parameters of the deep neural network and the hypersphere parameters.

[0196] Example 5:

[0197] If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0198] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0200] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0201] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0202] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An implicit identity authentication method based on multimodal data fusion, characterized in that, Includes the following steps: Acquire data from several types of motion sensors and one type of touchscreen sensor; The obtained motion sensor data of several types are preprocessed to obtain preprocessed motion sensor data of several types; the preprocessing refers to first performing noise reduction and filtering, and then performing standardization processing after noise reduction and filtering. Anomaly processing is performed on the obtained touch screen sensor data to obtain anomaly-processed touch screen sensor data. The preprocessed motion sensor data of several types is segmented to obtain several data segments of fixed length; Feature extraction is performed on a type of touchscreen sensor data and several fixed-length data segments after abnormal data processing to obtain spatiotemporal features and several touchscreen sensor features. Spatiotemporal features and features from several touchscreen sensors are input into an improved Transformer network to obtain a fragment representation that fuses behavioral semantics. The improved Transformer network includes: An improvement to the computational method of the attention mechanism in Transformer networks is based on decaying kernel functions and behavioral event-guided scores. By inputting the fragment representation of fused behavioral semantics into the improved SVDD network, implicit identity authentication results are obtained. The improved SVDD network includes: Deep neural networks are used to nonlinearly map raw user behavior data to a high-dimensional feature space. Construct a minimum envelope hypersphere in a high-dimensional feature space; Update the parameters of the deep neural network and the hypersphere parameters.

2. The implicit identity authentication method based on multimodal data fusion according to claim 1, characterized in that, The various types of motion sensor data include accelerometer sensor data, gyroscope sensor data, and magnetometer sensor data.

3. The implicit identity authentication method based on multimodal data fusion according to claim 1, characterized in that, The feature extraction specifically involves: using a convolutional neural network to extract features from several fixed-length data segments to obtain spatiotemporal features; and using statistical methods to extract features from a type of touchscreen sensor data after abnormal data processing to obtain several touchscreen sensor features.

4. The implicit identity authentication method based on multimodal data fusion according to claim 1, characterized in that, The touchscreen sensor features include position features, length features, speed features, angle features, and pressure features.

5. The implicit identity authentication method based on multimodal data fusion according to claim 1, characterized in that, The formula for calculating the fragment representation of fused behavioral semantics is as follows: Where h represents the fragment representation of the final fused multimodal behavioral data, and W :,k Let V be the attention weight vector, representing the contribution of the k-th segment to the overall representation. k Let K represent the behavioral features of the k-th segment, where K is the total number of segments; Attention weight vector W :,k The calculation formula is: in, An attention score matrix to incorporate behavioral event-guided scores; Attention score matrix with behavioral event guidance scores The calculation formula is: Among them, A :,k Let α be the attention score for the k-th segment in the standard Transformer. k Scores are assigned to behavioral events; Behavioral event guidance score α k The calculation formula is: Where, ω α,r For learnable event weights, b α As a bias term, G r,j ∈[0,1]^(2×K) is the event guidance matrix, representing the degree of influence of the r-th event on the j-th segment, and K r (k,j) is the decaying kernel function; decaying kernel function K r The expression for (k,j) is: Where k is the current segment index, j is the segment index used for correlation modeling, and R r The decay radius is the radius of the r-th type of behavior, used to control the rate at which the correlation decays with the segment spacing. App refers to App events and Touch refers to touch events.

6. The implicit identity authentication method based on multimodal data fusion according to claim 1, characterized in that, The formula for calculating the minimum envelope hypersphere is: Among them, h i Let F(h) be the i-th input sample, i = 1, ..., N, where N is the total number of training samples. i W) is the feature mapping function of the deep neural network, defined by the parameter set W = {W} k } control, input sample h i Mapped to a high-dimensional feature space, c is the hypersphere center vector, used to describe the cluster centers of the data in the high-dimensional feature space, |F(h i ;W)-c| 2 W is the squared Euclidean distance from the i-th sample to the center of the hypersphere, used to measure the deviation of the sample from the center of the hypersphere. λ is the regularization coefficient, used to balance the fitting error term and the regularization term. k | 2 For network parameters W k The square of the second norm is used as a regularization constraint.

7. The implicit identity authentication method based on multimodal data fusion according to claim 1, characterized in that, The updating of the parameters of the deep neural network and the hypersphere parameters specifically involves using an alternating optimization strategy to update the parameters of the deep neural network and the hypersphere parameters.

8. An implicit identity authentication system based on multimodal data fusion, characterized in that, include: The data acquisition module is used to acquire data from several types of motion sensors and data from a touch screen sensor. The preprocessing module is used to preprocess the obtained motion sensor data of several types to obtain preprocessed motion sensor data of several types; the preprocessing refers to first performing noise reduction and filtering, and then performing standardization processing after noise reduction and filtering. An abnormal data processing module is used to process the obtained touch screen sensor data to obtain abnormal touch screen sensor data after abnormal data processing. The segmentation processing module is used to segment the pre-processed motion sensor data of several types to obtain several data segments of fixed length. The feature extraction module is used to extract features from a type of touch screen sensor data and several fixed-length data segments after abnormal data processing, to obtain spatiotemporal features and several touch screen sensor features. The feature input module is used to input spatiotemporal features and features from several touchscreen sensors into an improved Transformer network to obtain a fragment representation that fuses behavioral semantics. The improved Transformer network includes: An improvement to the computational method of the attention mechanism in Transformer networks is based on decaying kernel functions and behavioral event-guided scores. The implicit identity authentication module is used to input the fragment representation of fused behavioral semantics into the improved SVDD network to obtain the implicit identity authentication result; The improved SVDD network includes: Deep neural networks are used to nonlinearly map raw user behavior data to a high-dimensional feature space. Construct a minimum envelope hypersphere in a high-dimensional feature space; Update the parameters of the deep neural network and the hypersphere parameters.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the implicit identity authentication method based on multimodal data fusion as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the implicit identity authentication method based on multimodal data fusion as described in any one of claims 1 to 7.

Images