A method, apparatus, device, and medium for continuous identity authentication based on multimodal user behavior fusion.

Abstract

This application discloses a continuous authentication method, apparatus, device, and medium based on multimodal user behavior fusion, relating to the field of computer technology. It includes: acquiring current user behavior data and performing feature processing on the behavior data to obtain a corresponding target behavior dataset; the behavior data includes contextual behavior data, mouse behavior data, and information interaction behavior data; performing data augmentation processing on the target behavior dataset, and inputting the augmented target behavior dataset into corresponding pre-trained anomaly detection models to obtain anomaly detection results for the current user; stacking the anomaly detection results to obtain target meta-feature data, and inputting the target meta-feature data into a preset meta-model to obtain authentication results for the current user, thereby achieving access control for the current user. This enables continuous detection of user behavior, ensuring that each user access complies with security policies.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for continuous identity authentication based on multimodal user behavior fusion. Background Technology

[0002] Authentication is the core means of verifying a user's identity and is widely used in information security and access control scenarios. Traditional authentication methods typically rely on credentials (such as passwords, authentication cards, and fingerprints) for one-time verification during initial login, assuming that the user's behavior is trustworthy throughout the session. However, this method cannot cope with security threats such as session hijacking and privilege abuse. Existing authentication methods mainly include two categories: static authentication and dynamic authentication.

[0003] Static authentication relies on fixed information provided by the user, commonly including passwords, PIN codes, pattern unlocks, and digital certificates. The verification process typically involves the user entering pre-set information, which the system compares to encrypted data stored in the database; a match results in successful verification. While static authentication is simple to implement and widely used, passwords are easily leaked and stolen. To improve security, dynamic authentication has emerged. This method generates unique verification information for each login, ensuring the independence and timeliness of each verification. Typical methods include one-time passwords (OTP), time-synchronized verification, and challenge-response mechanisms. One-time passwords are generated based on timestamps or event triggers; users must enter a randomly generated password within a specified time, often using SMS, email, mobile apps, or hardware tokens. Time-synchronized verification ensures that dynamic passwords are always short-lived and effective through time synchronization between the server and client devices, preventing password interception and replay attacks. Challenge-response mechanisms generate random challenge information; users must complete identity verification based on the challenge value, thus increasing the unpredictability of the verification. Although dynamic authentication effectively improves login security, it is still essentially a one-time authentication. After a user successfully logs in, the system cannot continuously monitor their actions, making it vulnerable to security vulnerabilities due to risks such as session hijacking and device theft.

[0004] As can be seen from the above, how to continuously monitor user behavior and ensure that every user access complies with security policies is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a continuous authentication method, apparatus, device, and medium based on multimodal user behavior fusion, which can continuously detect user behavior and ensure that each user access complies with security policies. The specific solution is as follows:

[0006] Firstly, this application provides a continuous identity authentication method based on multimodal user behavior fusion, including:

[0007] The system acquires the current user's behavior data and performs feature processing on the behavior data to obtain a corresponding target behavior dataset. The behavior data includes contextual behavior data, mouse behavior data, and information interaction behavior data. The target behavior dataset includes a first target behavior dataset corresponding to the contextual behavior data, a second target behavior dataset corresponding to the mouse behavior data, and a third target behavior dataset corresponding to the information interaction behavior data.

[0008] The target behavior dataset is augmented, and the augmented target behavior dataset is input into the corresponding pre-trained anomaly detection model to obtain anomaly detection results for the current user; the pre-trained anomaly detection model includes a first anomaly detection model corresponding to the contextual behavior data, a second anomaly detection model corresponding to the mouse behavior data, and a third anomaly detection model corresponding to the information interaction behavior data.

[0009] The anomaly detection results are stacked to obtain target meta-feature data, and the target meta-feature data is input into a preset meta-model to obtain the authentication result for the current user, so as to realize the permission control of the current user.

[0010] Optionally, if the behavioral data is contextual behavioral data, then obtaining the current user's behavioral data includes:

[0011] If a user interaction is triggered, the current user's contextual behavior data will be captured based on the system logs.

[0012] The contextual behavior data includes log entry identifiers, user identifiers, usernames, timestamps, IP addresses, operating systems, access methods, accessed page names, target URLs, behavior URLs, and network protocol data.

[0013] Optionally, if the behavior data is mouse behavior data, then obtaining the current user's behavior data includes:

[0014] The mouse behavior data of the current user is monitored based on a preset monitoring code, and the mouse behavior data is recorded based on a preset sampling frequency.

[0015] The mouse behavior data includes user ID, username, timestamp, operation data, and mouse coordinate data.

[0016] Optionally, if the behavioral data is information interaction behavioral data, then obtaining the current user's behavioral data includes:

[0017] If a user interaction is triggered, the preset recording module will be used to identify and record the current user's information interaction data.

[0018] The information interaction behavior data includes user ID, username, timestamp, session ID, event type, visited page, and page dwell time.

[0019] Optionally, the data augmentation process on the target behavior dataset includes:

[0020] The target behavior dataset is augmented using a pre-defined generative adversarial network to obtain an augmented target behavior dataset with a consistent time scale.

[0021] Optionally, the step of inputting the enhanced target behavior dataset into the corresponding pre-trained anomaly detection model to obtain anomaly detection results for the current user includes:

[0022] The enhanced first target behavior dataset is input into the pre-trained first anomaly detection model to obtain the first anomaly detection result corresponding to the scenario behavior data;

[0023] The enhanced second target behavior dataset is input into the pre-trained second anomaly detection model to obtain the second anomaly detection result corresponding to the scenario behavior data;

[0024] The enhanced third target behavior dataset is input into the pre-trained third anomaly detection model to obtain the third anomaly detection result corresponding to the scenario behavior data;

[0025] The pre-trained anomaly detection model is an anomaly detection model constructed based on an autoencoder of a long short-term memory network unit, including an encoder and a decoder; the anomaly detection result includes a threshold, reconstruction error, and predicted binary label, wherein the threshold is a critical value used to distinguish between legitimate user behavior and illegitimate user behavior.

[0026] Optionally, the preset meta-model is a machine learning model based on the random forest algorithm.

[0027] Secondly, this application provides a continuous identity authentication device based on multimodal user behavior fusion, comprising:

[0028] The behavior data acquisition module is used to acquire the current user's behavior data and perform feature processing on the behavior data to obtain a corresponding target behavior dataset; the behavior data includes contextual behavior data, mouse behavior data, and information interaction behavior data; the target behavior dataset includes a first target behavior dataset corresponding to the contextual behavior data, a second target behavior dataset corresponding to the mouse behavior data, and a third target behavior dataset corresponding to the information interaction behavior data;

[0029] An anomaly detection result generation module is used to perform data augmentation processing on the target behavior dataset, and input the augmented target behavior dataset into the corresponding pre-trained anomaly detection model to obtain anomaly detection results for the current user; the pre-trained anomaly detection model includes a first anomaly detection model corresponding to the contextual behavior data, a second anomaly detection model corresponding to the mouse behavior data, and a third anomaly detection model corresponding to the information interaction behavior data.

[0030] The authentication result generation module is used to stack the anomaly detection results to obtain target meta-feature data, and input the target meta-feature data into a preset meta-model to obtain the authentication result for the current user, so as to realize the permission control of the current user.

[0031] Thirdly, this application provides an electronic device, comprising:

[0032] Memory, used to store computer programs;

[0033] A processor is used to execute the computer program to implement the aforementioned continuous identity authentication method based on multimodal user behavior fusion.

[0034] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned continuous identity authentication method based on multimodal user behavior fusion.

[0035] This application provides a continuous authentication method based on multimodal user behavior fusion. First, it acquires the current user's behavior data and performs feature processing on the behavior data to obtain a corresponding target behavior dataset. The behavior data includes contextual behavior data, mouse behavior data, and information interaction behavior data. Then, it performs data augmentation processing on the target behavior dataset and inputs the augmented target behavior dataset into a pre-trained anomaly detection model to obtain anomaly detection results for the current user. Finally, it stacks the anomaly detection results to obtain target meta-feature data, and inputs the target meta-feature data into a preset meta-model to obtain the authentication result for the current user, thereby achieving access control for the current user.

[0036] As can be seen from the above, this application combines multiple behavioral features, such as contextual behavioral data, mouse behavior data, and information interaction behavior data, and utilizes a deep learning model to comprehensively analyze multimodal data. Compared to single behavioral feature verification, multimodal fusion can effectively avoid false positives caused by fluctuations in individual features, while further improving anti-attack capabilities and verification stability. This enables continuous detection of user behavior, ensuring that every user access complies with security policies. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This application discloses a flowchart of a continuous identity authentication method based on multimodal user behavior fusion.

[0039] Figure 2 This application discloses a specific flowchart of a continuous identity authentication method based on multimodal user behavior fusion.

[0040] Figure 3 This is a schematic diagram of a continuous identity authentication device based on multimodal user behavior fusion disclosed in this application;

[0041] Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Existing authentication methods mainly fall into two categories: static authentication and dynamic authentication. Static authentication relies on fixed information provided by the user, with common methods including passwords, PIN codes, pattern unlocks, and digital certificates. The verification process typically involves the user entering pre-set information, which the system compares to encrypted data stored in the database; a match results in successful authentication. While static authentication is simple to implement and widely used, passwords are easily leaked or stolen. To improve security, dynamic authentication has emerged. This method generates unique verification information for each login, ensuring the independence and timeliness of each verification. Typical methods include dynamic passwords (OTP), time-synchronized authentication, and challenge-response mechanisms. Dynamic passwords are generated based on timestamps or event triggering mechanisms; users must enter a randomly generated password within a specified time, often using SMS, email, mobile apps, or hardware tokens. Time-synchronized authentication ensures that dynamic passwords are always short-lived and effective through time synchronization between the server and client devices, preventing password interception and replay attacks. Challenge-response mechanisms generate random challenge information; users must complete identity verification based on this challenge value, thus increasing the unpredictability of the authentication process. While dynamic authentication effectively improves login security, it is essentially a one-time verification. After a user successfully logs in, the system cannot continuously monitor their actions, making it vulnerable to security vulnerabilities due to risks such as session hijacking and device theft. Therefore, this application provides a continuous authentication scheme based on multimodal user behavior fusion, which can continuously detect user behavior and ensure that each user access complies with security policies.

[0044] See Figure 1 As shown in the figure, this application discloses a continuous identity authentication method based on multimodal user behavior fusion, including:

[0045] Step S11: Obtain the current user's behavior data and perform feature processing on the behavior data to obtain the corresponding target behavior dataset.

[0046] In this embodiment, the behavioral data includes contextual behavioral data, mouse behavioral data, and information interaction behavioral data. Contextual behavioral data reflects how users respond to different interaction scenarios; mouse behavioral data is used to analyze mouse usage patterns; and information interaction behavioral data is used to determine how users process information. Furthermore, different data collection methods are used depending on the type of behavioral data. Specifically, if the behavioral data is contextual behavioral data, obtaining the current user's behavioral data may include: if a user interaction is triggered, capturing the current user's contextual behavioral data based on system logs; wherein the contextual behavioral data includes log entry identifiers, user identifiers, usernames, timestamps, IP addresses, operating systems, access methods, accessed page names, target URLs, behavioral URLs, and network protocol data. If the behavioral data is mouse behavioral data, obtaining the current user's behavioral data may include: monitoring the current user's mouse behavioral data based on preset monitoring code, and recording the mouse behavioral data based on a preset sampling frequency; wherein the mouse behavioral data includes user ID, username, timestamp, operation data, and mouse coordinate data. If the behavioral data is information interaction behavior data, then obtaining the current user's behavioral data may include: if a user interaction behavior is triggered, identifying and recording the current user's information interaction behavior data using a preset recording module; the information interaction behavior data includes user ID, username, timestamp, session ID, event type, visited page, and page dwell time data. That is, by obtaining multiple types of behavioral data, the accuracy of identity verification is improved, resulting in higher accuracy and a lower error rate compared to methods based on a single modality.

[0047] Step S12: Perform data augmentation on the target behavior dataset, and input the augmented target behavior dataset into the corresponding pre-trained anomaly detection model to obtain anomaly detection results for the current user.

[0048] In this embodiment, due to the differences in sampling frequencies among the three types of behavioral data and the time-series nature of user behavior, deviations may occur during multimodal fusion, thus affecting the accuracy of authentication. Therefore, data augmentation processing is required to ensure alignment between the various data sets. Specifically, the data augmentation processing of the target behavior dataset may include: using a preset generative adversarial network to augment the target behavior dataset to obtain an augmented target behavior dataset with a consistent time scale; wherein, the target behavior dataset includes a first target behavior dataset corresponding to the contextual behavior data, a second target behavior dataset corresponding to the mouse behavior data, and a third target behavior dataset corresponding to the information interaction behavior data.

[0049] Furthermore, an unsupervised anomaly detection of time-series behavior is performed based on an autoencoder using Long Short-Term Memory (LSTM), and continuous authentication is achieved for each modality. Specifically, the step of inputting the enhanced target behavior dataset into the corresponding pre-trained anomaly detection model to obtain anomaly detection results for the current user can include: inputting the enhanced first target behavior dataset into the pre-trained first anomaly detection model to obtain a first anomaly detection result corresponding to the contextual behavior data; inputting the enhanced second target behavior dataset into the pre-trained second anomaly detection model to obtain a second anomaly detection result corresponding to the contextual behavior data; and inputting the enhanced third target behavior dataset into the pre-trained third anomaly detection model to obtain a third anomaly detection result corresponding to the contextual behavior data. The pre-trained anomaly detection model is an anomaly detection model constructed based on an autoencoder using Long Short-Term Memory network units, including an encoder and a decoder. The anomaly detection result includes a threshold, reconstruction error, and predicted binary label, wherein the threshold is a critical value used to distinguish between legitimate and illegitimate user behavior.

[0050] The encoder module consists of two LSTM layers and one fully connected layer. The LSTM layers capture the temporal dependencies of the action sequence, while the fully connected layer transforms the sequence format to suit the decoder. The decoder's task is to reconstruct the original sequence using the output sequence from the encoder. The decoder structure is similar to the encoder structure, containing two LSTM layers and one fully connected layer. The LSTM layers in the decoder expand the feature space, and the fully connected layer maps the sequence back to the original space through a linear transformation. This process reverses the transformation performed by the encoder, ultimately generating the reconstructed sequence. The training objective of the above anomaly detection model is to minimize the reconstruction error between the input sequence and the corresponding reconstructed sequence. If the reconstruction error exceeds a predefined threshold, the sequence is marked as an anomaly. It is worth noting that the specific method for calculating the reconstruction error can be determined according to the actual application. For example, in some specific implementations, mean squared error can be used to calculate the reconstruction error.

[0051] Step S13: Stack the anomaly detection results to obtain target meta-feature data, and input the target meta-feature data into a preset meta-model to obtain the authentication result for the current user, so as to realize the permission control of the current user.

[0052] In this embodiment, to ensure information integrity, three types of behavioral data are integrated using a stacking approach. Specifically, this includes the difference between the reconstruction error and the corresponding threshold (DET) and the sum of predicted labels (SP). DET represents the degree of anomaly; a higher value indicates a greater likelihood that the behavior belongs to an imposter. SP is the comprehensive detection result from all three modalities, ranging from 0 to 3. A value of 0 indicates that all modalities classify the behavior as imposter behavior, while a value of 3 indicates that all modalities classify it as legitimate user behavior. Values ​​of 1 and 2 indicate inconsistencies between modalities, meaning that most or a few modalities consider the behavior as dangerous user behavior. Furthermore, the meta-model uses these meta-features for training. The algorithm for implementing the meta-model is selected based on the specific needs of continuous authentication, such as a machine learning algorithm like random forest. It is worth mentioning that during training, the parameters of the meta-model are optimized to learn and process the relationships between the three modalities and identify the behavior of legitimate or dangerous users. That is, the three types of behavioral data are processed through their respective long short-term memory autoencoder models. Meta-features are then generated and fed into a trained meta-model, which evaluates the model and makes a final authentication decision.

[0053] As can be seen from the above, the embodiments of this application combine multiple behavioral features, such as contextual behavioral data, mouse behavior data, and information interaction behavior data, and utilize deep learning models to comprehensively analyze multimodal data. Compared to single behavioral feature verification, multimodal fusion can effectively avoid false alarms caused by fluctuations in individual features, while further improving anti-attack capabilities and verification stability. Furthermore, multimodal authentication technology can also combine the continuous learning and self-evolution capabilities of AI models to accurately identify abnormal behavioral deviations, achieving seamless and continuous verification. This enables continuous detection of user behavior, ensuring that every user access complies with security policies.

[0054] See Figure 2 As shown in the figure, this application discloses a specific continuous identity authentication method based on multimodal user behavior fusion, including:

[0055] This embodiment integrates three modalities: contextual behavior, mouse behavior, and information interaction behavior. Contextual behavior reflects how users respond to different interaction scenarios, mouse behavior analyzes mouse usage patterns, and information interaction behavior corresponds to how users process information. First, data from these three modalities is collected through historical users or testers of the router. Then, behavioral feature sets are created for each modality. Next, an improved Generative Adversarial Network (GAN) method is used to enhance the datasets of the three modalities, enabling them to perform data augmentation tasks to ensure data alignment across the three modalities. Subsequently, the scheme uses an autoencoder based on Long Short-Term Memory (LSTM) to perform unsupervised anomaly detection of time-series behavior and implements continuous authentication for each modality. Finally, a meta-model using a stacked generalization method is used to achieve multimodal fusion, resulting in the final decision scheme for determining continuous authentication. Compared to single-modal methods, this approach offers higher accuracy and a lower error rate, enhancing authentication security while providing a better user experience.

[0056] In this embodiment, system logs are used to capture contextual information about user interactions within the data space. The system logs capture information across multiple dimensions, including log number, user ID, username, timestamp, IP address, operating system, access method, title, target URL, behavior URL, and protocol. The user ID is a unique identifier associated with the username; users log in to the website using their usernames, ensuring that each recorded interaction is accurately attributed to the corresponding user. It's important to note that the sampling rate is event-triggered; that is, a data sample is recorded whenever a significant user interaction occurs (such as submitting a form or navigating a page).

[0057] In this embodiment, specific JavaScript code is embedded in each webpage to monitor user mouse activity. Recorded information includes user ID, username, timestamp, operation (mouse action event, such as movement, click, etc.), X-coordinate of the click, and Y-coordinate of the click. It should be noted that the specific sampling rate can be determined based on actual conditions and is not specifically limited here. It is recommended to set the sampling rate to 10 Hz and continuously record samples during user interaction with the webpage.

[0058] In this embodiment, a dedicated JavaScript module is used in the website backend to identify and record user interactions with content, and then the data is transmitted to the server. The recorded data includes information across multiple dimensions, such as user ID, username, timestamp, session ID, event type, visited page, and page dwell time. It's important to note that the sampling rate is behavior-triggered; that is, samples are recorded whenever significant user behavior occurs (such as page scrolling).

[0059] Furthermore, to alleviate the problems of insufficient sample quantity or uneven temporal distribution in the target behavior dataset, a predefined Generative Adversarial Network (PGAN) is introduced to augment the target behavior data. This network learns the temporal features and patterns of target behavior to generate synthetic samples that are consistent with the original data on a temporal scale, thereby expanding the scale and diversity of the dataset. Specifically, the PGAN model includes a generator and a discriminator: the generator, receiving random noise and a temporal conditional vector as input, outputs a simulated behavior sequence; the discriminator determines whether the input sequence originates from the real dataset, further driving the generator to learn and generate high-quality samples that conform to the temporal distribution. Through adversarial training, the generated samples not only maintain consistency with the real data in behavioral features but also exhibit consistent sampling intervals, behavior frequencies, and evolutionary trends over time. Finally, these high-quality, numerous, and temporally aligned synthetic samples are fused with the original dataset to construct an augmented target behavior dataset, providing more comprehensive temporal information support for subsequent modeling and analysis.

[0060] In this embodiment, three sets of long short-term memory autoencoder models are used to process three behavioral modalities: contextual behavior, mouse behavior, and information interaction behavior. These models capture the temporal patterns of user behavior and detect anomalies. The output of each model includes: a threshold, a critical value used to distinguish between legitimate user behavior and imposter behavior; a reconstruction error, used to determine the numerical difference between the reconstructed output and the original input; and a prediction label, a binary label indicating whether the behavior belongs to a legitimate user (1) or a dangerous user (0).

[0061] Furthermore, a stacked approach is used to integrate the three behavioral modalities. Meta-features record the outputs of the long short-term memory autoencoder model for the three modalities and their combinations. These include the difference between the reconstruction error and the corresponding threshold (DET) and the sum of predicted labels (SP). DET indicates the degree of anomalousness; a higher value indicates a greater likelihood that the behavior belongs to an imposter. SP is the combined detection result from all three modalities, ranging from 0 to 3. A value of 0 indicates that all modalities classify the behavior as imposter behavior, while a value of 3 indicates that all modalities classify it as legitimate user behavior. Values ​​of 1 and 2 indicate inconsistencies between modalities, i.e., most or a few modalities consider the behavior as dangerous user behavior. The meta-model is trained using these meta-features. The algorithm implementing the meta-model is chosen based on the specific needs of continuous authentication, such as a machine learning algorithm like random forest. During training, the parameters of the meta-model are optimized to learn and process the relationships between the three modalities and identify the behavior of legitimate or dangerous users. During the authentication phase, new behavioral data from the three modalities are processed through their respective long short-term memory autoencoder models. Meta-features are then generated and fed into the trained meta-model. The meta-model evaluates the features and makes the final authentication decision.

[0062] As can be seen from the above, the embodiments of this application combine multiple behavioral features, such as input patterns, mouse trajectories, and device movement patterns, and utilize deep learning models to comprehensively analyze multimodal data. Compared to single behavioral feature verification, multimodal fusion can effectively avoid false alarms caused by fluctuations in individual features, while further improving anti-attack capabilities and verification stability. Furthermore, multimodal authentication technology can also combine the continuous learning and self-evolution capabilities of AI models to accurately identify abnormal behavioral deviations, achieving seamless and continuous verification. In high-risk scenarios, multimodal dynamic behavioral authentication, with its more accurate behavior recognition and dynamic permission adjustment mechanism, significantly improves the security and flexibility of identity verification.

[0063] See Figure 3 As shown in the figure, this application discloses a continuous identity authentication device based on multimodal user behavior fusion, including:

[0064] The behavior data acquisition module 11 is used to acquire the behavior data of the current user and perform feature processing on the behavior data to obtain a corresponding target behavior dataset; the behavior data includes contextual behavior data, mouse behavior data and information interaction behavior data; the target behavior dataset includes a first target behavior dataset corresponding to the contextual behavior data, a second target behavior dataset corresponding to the mouse behavior data and a third target behavior dataset corresponding to the information interaction behavior data;

[0065] The anomaly detection result generation module 12 is used to perform data augmentation processing on the target behavior dataset and input the augmented target behavior dataset into the corresponding pre-trained anomaly detection model to obtain the anomaly detection result for the current user; the pre-trained anomaly detection model includes a first anomaly detection model corresponding to the contextual behavior data, a second anomaly detection model corresponding to the mouse behavior data, and a third anomaly detection model corresponding to the information interaction behavior data.

[0066] The identity verification result generation module 13 is used to stack the anomaly detection results to obtain target meta-feature data, and input the target meta-feature data into a preset meta-model to obtain the identity verification result for the current user, so as to realize the permission control of the current user; the preset meta-model is a machine learning model based on the random forest algorithm.

[0067] As can be seen from the above, the embodiments of this application combine multiple behavioral features, such as contextual behavioral data, mouse behavior data, and information interaction behavior data, and utilize deep learning models to comprehensively analyze multimodal data. Compared to single behavioral feature verification, multimodal fusion can effectively avoid false alarms caused by fluctuations in individual features, while further improving anti-attack capabilities and verification stability. This enables continuous detection of user behavior, ensuring that each user access complies with security policies.

[0068] In some specific embodiments, the behavior data acquisition module 11 may specifically include:

[0069] The contextual behavior data acquisition unit is used to capture the current user's contextual behavior data based on the system log if a user interaction behavior is triggered; wherein, the contextual behavior data includes log entry identifier, user identifier, username, timestamp, IP address, operating system, access method, accessed page name, target URL, behavior URL, and network protocol data;

[0070] The mouse behavior data acquisition unit is used to monitor the current user's mouse behavior data based on a preset monitoring code, and record the mouse behavior data based on a preset sampling frequency; wherein, the mouse behavior data includes user ID, username, timestamp, operation data, and mouse coordinate data;

[0071] The information interaction behavior data acquisition unit is used to identify and record the current user's information interaction behavior data using a preset recording module if a user interaction behavior is triggered; the information interaction behavior data includes user ID, username, timestamp, session ID, event type, accessed page, and page dwell time data.

[0072] In some specific embodiments, the anomaly detection result generation module 12 may specifically include:

[0073] The data augmentation processing unit is used to perform data augmentation processing on the target behavior dataset using a preset generative adversarial network to obtain an augmented target behavior dataset with a consistent time scale.

[0074] The first anomaly detection result generation unit is used to input the enhanced first target behavior dataset into the pre-trained first anomaly detection model to obtain the first anomaly detection result corresponding to the scenario behavior data.

[0075] The second anomaly detection result generation unit is used to input the enhanced second target behavior dataset into the pre-trained second anomaly detection model to obtain the second anomaly detection result corresponding to the scenario behavior data.

[0076] The third anomaly detection result generation unit is used to input the enhanced third target behavior dataset into the pre-trained third anomaly detection model to obtain the third anomaly detection result corresponding to the scenario behavior data.

[0077] The pre-trained anomaly detection model is an anomaly detection model constructed based on an autoencoder of a long short-term memory network unit, including an encoder and a decoder; the anomaly detection result includes a threshold, reconstruction error, and predicted binary label, wherein the threshold is a critical value used to distinguish between legitimate user behavior and illegitimate user behavior.

[0078] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the continuous identity authentication method based on multimodal user behavior fusion disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0079] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0080] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0081] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the continuous identity authentication method based on multimodal user behavior fusion executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0082] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed continuous identity authentication method based on multimodal user behavior fusion. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0083] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0084] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0085] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0086] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0087] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for continuous identity authentication based on multi-modal user behavior fusion, characterized in that, The method comprises the following steps: obtaining behavior data of a current user, and performing feature processing on the behavior data to obtain a corresponding target behavior data set; the behavior data comprises scene behavior data, mouse behavior data and information interaction behavior data; the target behavior data set comprises a first target behavior data set corresponding to the scene behavior data, a second target behavior data set corresponding to the mouse behavior data and a third target behavior data set corresponding to the information interaction behavior data; performing data enhancement processing on the target behavior data set, and inputting the enhanced target behavior data set into a corresponding pre-trained anomaly detection model to obtain an anomaly detection result for the current user; the pre-trained anomaly detection model comprises a first anomaly detection model corresponding to the scene behavior data, a second anomaly detection model corresponding to the mouse behavior data and a third anomaly detection model corresponding to the information interaction behavior data; the anomaly detection result for the current user comprises a first anomaly detection result corresponding to the scene behavior data, a second anomaly detection result corresponding to the scene behavior data and a third anomaly detection result corresponding to the scene behavior data; the pre-trained anomaly detection model is an anomaly detection model constructed based on a long short-term memory network unit, comprising an encoder and a decoder; the anomaly detection result comprises a threshold value, a reconstruction error and a predicted binary label, wherein the threshold value is a critical value for distinguishing between legal user behavior and illegal user behavior; stacking the anomaly detection results to obtain target meta-feature data, and inputting the target meta-feature data into a preset meta-model to obtain an identity verification result for the current user, so as to realize permission management of the current user; wherein the data enhancement processing on the target behavior data set comprises: performing data enhancement processing on the target behavior data set by using a preset generative adversarial network to obtain enhanced target behavior data sets with consistent time scales; the inputting of the enhanced target behavior data set into a corresponding pre-trained anomaly detection model to obtain an anomaly detection result for the current user comprises: inputting the enhanced target behavior data set into a corresponding pre-trained anomaly detection model, so as to capture the time dependence of the behavior sequence in the target behavior data set by using the encoder in the anomaly detection model, and convert the time dependence into an output sequence suitable for the decoder; expanding the feature space by using the decoder, and mapping the output sequence back to the original space through linear transformation to generate a reconstruction sequence; generating a reconstruction error based on the reconstruction sequence, and generating the predicted binary label by judging whether the reconstruction error exceeds the threshold value.

2. The method for continuous identity authentication based on multi-modal user behavior fusion according to claim 1, characterized in that, If the behavior data is scene behavior data, the obtaining of the behavior data of the current user comprises: if a user interaction behavior is triggered, capturing the scene behavior data of the current user based on system logs; The scenario behavior data includes a log record identifier, a user identifier, a user name, a timestamp, an IP address, an operating system, an access method, an access page name, a target website address, behavior website address, and network protocol data. 3.The method of claim 1, wherein, If the behavior data is mouse behavior data, the behavior data of the current user is obtained by: monitoring mouse behavior data of the current user based on a preset monitoring code, and recording the mouse behavior data based on a preset sampling frequency; The mouse behavior data includes a user number, a user name, a timestamp, operation data, and mouse coordinate data.

4. The method for continuous identity authentication based on multi-modal user behavior fusion according to claim 1, characterized in that, If the behavior data is information interaction behavior data, the behavior data of the current user is obtained by: If a user interaction behavior is triggered, a preset recording module is used to identify and record information interaction behavior data of the current user. The information interaction behavior data includes a user number, a user name, a timestamp, a session number, an event type, an access page, and page dwell time data.

5. The method of claim 1 to 4, wherein, The preset meta-model is a machine learning model based on a random forest algorithm.

6. A device for continuous identity authentication based on multi-modal user behavior fusion, characterized in that, It includes: a behavior data acquisition module for acquiring behavior data of a current user and performing feature processing on the behavior data to obtain a corresponding target behavior data set; The behavior data includes scenario behavior data, mouse behavior data, and information interaction behavior data. The target behavior data set includes a first target behavior data set corresponding to the scenario behavior data, a second target behavior data set corresponding to the mouse behavior data, and a third target behavior data set corresponding to the information interaction behavior data; an anomaly detection result generation module for performing data augmentation processing on the target behavior data set and inputting the augmented target behavior data set into a corresponding pre-trained anomaly detection model to obtain an anomaly detection result for the current user; the pre-trained anomaly detection model includes a first anomaly detection model corresponding to the scenario behavior data, a second anomaly detection model corresponding to the mouse behavior data, and a third anomaly detection model corresponding to the information interaction behavior data; the anomaly detection result for the current user includes a first anomaly detection result corresponding to the scenario behavior data, a second anomaly detection result corresponding to the scenario behavior data, and a third anomaly detection result corresponding to the scenario behavior data; the pre-trained anomaly detection model is an anomaly detection model constructed based on a long short-term memory network unit, including an encoder and a decoder; the anomaly detection result includes a threshold, a reconstruction error, and a prediction binary label, wherein the threshold is a critical value for distinguishing between legal user behavior and illegal user behavior; an identity verification result generation module for stacking the anomaly detection result to obtain target meta-feature data, and inputting the target meta-feature data into a preset meta-model to obtain an identity verification result for the current user, so as to realize permission management of the current user; The anomaly detection result generation module includes: The data augmentation processing unit is configured to perform data augmentation processing on the target behavior dataset by using a preset generative adversarial network, to obtain an augmented target behavior dataset with a consistent time scale. The data encoding unit is configured to input the augmented target behavior dataset into a corresponding pre-trained anomaly detection model, so as to capture a time-dependent relationship of a behavior sequence in the target behavior dataset by using an encoder in the anomaly detection model, and convert the time-dependent relationship into an output sequence suitable for a decoder. The data decoding unit is configured to expand a feature space by using the decoder, and map the output sequence back to an original space by linear transformation, to generate a reconstructed sequence. The reconstruction error generation unit is configured to generate a reconstruction error based on the reconstructed sequence, and generate the prediction binary label by judging whether the reconstruction error exceeds the threshold.

7. An electronic device, comprising: The computer program is stored in the memory and executed by the processor to implement the method for continuous identity authentication based on multi-modal user behavior fusion according to any one of claims 1 to 5. The computer program is stored in the memory and executed by the processor to implement the method for continuous identity authentication based on multi-modal user behavior fusion according to any one of claims 1 to 5. ​ 8. A computer-readable storage medium, characterized in that, ​

Images