A User Desktop Behavior Auditing Method and System Based on YOLO and LSTM

Abstract

This invention relates to the field of auditing systems, and particularly to a user desktop behavior auditing method and system based on YOLO and LSTM. The method includes: acquiring desktop video data to create a training dataset; training a user desktop behavior auditing model using the data-augmented dataset; acquiring real-time video data and analyzing the real-time video data frame by frame using the trained user desktop behavior auditing model; predicting user operation behavior trends based on the analyzed real-time video data; validating the prediction results; and converting the validated prediction results into structured logs. This invention uses a Long Short-Term Memory network to model time-series data and combines it with the YOLOv8 deep learning framework, enabling the system to identify the specific software used by the user and infer the type of task performed by the user, thereby significantly improving the accuracy of behavior recognition and auditing efficiency.

Description

Technical Field

[0001] This invention relates to the field of auditing systems, and more specifically to a user desktop behavior auditing method and system based on YOLO and LSTM. Background Technology

[0002] In the context of rapid development in information technology, the auditing and management of desktop user behavior has gradually become a crucial requirement for enterprise information security and productivity management. Especially with the increasing prevalence of remote and hybrid work models, effectively acquiring user behavior information on computers and using this information for risk assessment and productivity analysis has become paramount. However, existing methods for auditing user desktop behavior largely rely on simple log recording and keyboard and mouse operation tracking, which have many limitations. With the continuous advancement of deep learning technology, its superiority in areas such as image recognition and behavior analysis is becoming increasingly apparent, providing new solutions for desktop behavior auditing.

[0003] Deep learning-based user desktop behavior auditing methods can perform in-depth analysis of user actions using screen recording data, providing more detailed and intelligent behavioral understanding. However, research in this area is currently lacking, with mature methods and publicly available datasets. Traditional behavior auditing methods typically only provide relatively simple operation records, such as software startup time and keystroke counts, failing to intuitively reflect the complex operations performed by users on the screen. These methods lack a deep semantic understanding of user behavior, making it difficult to determine what specific tasks the user is performing or how they are using the software. Traditional methods often cannot effectively handle diverse and unstructured data, resulting in significant information gaps in user operation records, hindering further intelligent analysis and prediction.

[0004] Therefore, existing technologies have significant shortcomings in terms of data integrity and in-depth analysis, and cannot meet the behavioral auditing needs in complex application scenarios. Summary of the Invention

[0005] In view of this, the present invention aims to solve the problems of insufficient depth of operation auditing and limited ability to understand behavior in the prior art, so as to effectively improve the visualization management capability of user operations.

[0006] The user desktop behavior auditing method based on YOLO and LSTM described in this invention includes:

[0007] Step 1. Obtain desktop video data to create a dataset, and perform annotation, partitioning, and data augmentation operations on the dataset to obtain the training set and validation set;

[0008] Step 2. Train the user desktop behavior audit model using the training set and evaluate the user desktop behavior audit model using the validation set. The user desktop behavior audit model uses a YOLOv8 neural network to obtain the trained user desktop behavior audit model.

[0009] Step 3. Acquire real-time video data. Use the trained user desktop behavior auditing model to perform image recognition on each frame of the real-time video data to obtain information O from the real-time video data. i O i This includes information about the window's type, position, and status.

[0010] Step 4. Place O i The data is input into an LSTM network for time series analysis to predict user behavior trends; the prediction results are then validated.

[0011] Step 5. Convert the verified prediction results into structured logs, add warning information to the operation records in the structured logs, perform tagging and encryption, and store the structured logs and tag data.

[0012] A user desktop behavior auditing system based on YOLO and LSTM is disclosed. The system adopts a distributed architecture, and the modules interact with each other through API interfaces. The system includes a video acquisition terminal, a behavior analysis server, and a management terminal connected in sequence.

[0013] Video capture terminal: Acquires user desktop video data via HDMI loop-out encoder and backup encoder, and transmits the video data to behavior analysis server; Behavior analysis server: Processes the data and performs deep learning analysis to generate user behavior audit reports; Management terminal: Provides a graphical user interface to display detailed behavior audit results, set early warning rules, and export audit reports.

[0014] The beneficial effects of this invention are:

[0015] By focusing on fine-grained identification of user behavior, the system uses Long Short-Term Memory (LSTM) networks to model time-series data and further integrates the YOLOv8 deep learning framework. This enables the system to identify the specific software used by users and infer the types of tasks they perform, thereby effectively improving the accuracy of behavior identification and thus enhancing audit efficiency. By performing fine-grained classification and labeling of user behavior and setting alarm mechanisms according to specific rules, the system greatly improves the comprehensiveness of behavior monitoring and the management value of the system. Attached Figure Description

[0016] Figure 1 This is an operational schematic diagram of the video acquisition terminal described in this invention;

[0017] Figure 2This is a flowchart illustrating the data set creation and deep learning model training module described in this invention.

[0018] Figure 3 This is a flowchart illustrating the implementation of the reasoning and behavior recognition module described in this invention;

[0019] Figure 4 This is a flowchart illustrating the behavior auditing and logging module described in this invention.

[0020] Figure 5 This is a schematic diagram illustrating the operation of the deep learning-based user desktop behavior auditing system described in this invention. Detailed Implementation

[0021] To make the objectives, technical solutions, features, and advantages of the present invention clearer, the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0022] Example 1:

[0023] This invention provides a user desktop behavior auditing method based on YOLO and LSTM, comprising:

[0024] Step 1. Obtain desktop video data to create a dataset. Perform annotation, partitioning, and data augmentation operations on the dataset to obtain the training set and validation set.

[0025] Specifically, the dataset creation process includes: loading and decoding screen video data using automated scripts to obtain raw video frames; using OpenCV to read the video and extract image data frame by frame, converting each frame into a standard RGB image format for subsequent processing; extracting information-rich and representative keyframes frame by frame to reduce data redundancy and retain the core content of user operations; the keyframe extraction is based on a threshold k for the changes in image content between frames: when the changes between video frames exceed the threshold k, that frame is retained as a keyframe, k is preferably 50, and can be adjusted according to actual needs; the conditions for keyframe extraction are defined as follows:

[0026]

[0027] Wherein, D(f) i ,f i+1 ) represents frame f i and frame f i+1 The average pixel difference between them, denoted by N, Indicates frame f i The value of the middle pixel (x,y) when the average pixel difference D(f i ,f i+1 When the value is greater than k, extract the keyframe f. i+1 .

[0028] The annotation process includes: using the CVAT tool to annotate the extracted keyframes, including window type, operation objects, user interaction behaviors, etc.; the CVAT tool supports semi-automatic annotation based on existing models to reduce the workload of manual annotation and improve annotation consistency and efficiency; the annotation process combines automated tools and manual verification to ensure the accuracy of annotation and a high degree of consistency in data quality.

[0029] The partitioning process includes: dividing the original dataset into a training set, a validation set, and sometimes a test set; the training set is used to train the model, allowing it to learn the characteristics and patterns of desktop operation behavior, and gradually adjusting the model's parameters so that the model can perform well on the training data; the validation set is used to evaluate the model's performance during training; after the model has completed several training cycles, the validation set is used to evaluate the model's performance to determine whether there are overfitting, underfitting, or other issues; the data in the validation set is not used for training, but it reflects the model's performance on new data, thereby helping to adjust the model parameters.

[0030] Data augmentation includes rotating, scaling, and adjusting the contrast of images to obtain an augmented dataset.

[0031] Furthermore, data augmentation is a core strategy for improving the generalization ability of a model. By performing various transformations on the original data to generate diverse data samples, the risk of model overfitting can be effectively reduced. The data augmentation process can be represented as:

[0032] I′=T(I)

[0033] Where I represents the original image, and T represents the set of data augmentation transformation operations, including rotation R(θ), scaling S(s), contrast adjustment C(c), etc.; the transformed image I′ can be used to enrich the data sample space and improve the robustness of the model; the probability distribution P(I′) after data augmentation can be expressed as:

[0034] P(I′)=P(T)P(I)dT

[0035] Where P(T) represents the probability distribution of the data augmentation transformation T, and P(I) represents the probability distribution of the original image I. By optimizing the probability distribution P(T) of different transformation operations, the model can ultimately have better adaptability and generalization performance.

[0036] Furthermore, P(T) optimization includes: in data augmentation, selecting various types of image transformation operations, such as rotation, translation, scaling, cropping, flipping, brightness adjustment, and noise addition; optimizing P(T) determines the probability of each transformation operation being used, and adjusting it through the AutoAugment algorithm, an automated data augmentation strategy, to ensure that each transformation appears appropriately during training, thereby enriching the diversity of the dataset; for example, if it is found that the model does not perform well in handling illumination changes, the probability of illumination change transformations is increased, so that the model obtains more illumination change data during training;

[0037] Specifically, through reinforcement learning, the optimal augmentation operation and its corresponding probability distribution P(T) are selected to maximize the model's performance on the validation set, thereby improving the model's generalization performance. The optimization of data augmentation can be achieved through feedback from the model on the validation set. Specifically, an initial augmentation operation probability P(T) can be set, and then the probability can be adjusted by observing the model's performance on the validation set. For example, if certain transformations (such as rotation) can significantly improve the accuracy on the validation set, their probability can be increased, and vice versa.

[0038] Step 2. Train the user desktop behavior audit model using the training set and evaluate the user desktop behavior audit model using the validation set. The user desktop behavior audit model uses a YOLOv8 neural network to obtain the trained user desktop behavior audit model.

[0039] Specifically, during training, the dataset is trained in batches, and the model and data are loaded onto the GPU for computation to improve training speed.

[0040] Batch training refers to dividing the entire dataset into several smaller batches and training the model batch by batch, unlike feeding all the data into the model at once. Assuming the dataset has N samples, it can be divided into several batches, each batch being B (called the batch size). In each iteration, the model processes only B samples. Batch training addresses the limitation of computational resources and accelerates model convergence. Processing large datasets in batches reduces computational overhead, avoids memory shortages, and allows model training to be completed in less memory. YOLOv8 is an efficient convolutional neural network, used in this invention to recognize application windows and user interaction objects in desktop operations. The loss function L during training can be expressed as:

[0041] L = L cls +L obj +L bbox

[0042] Among them, L cls For classification loss, Lobj For the target confidence loss, L bbox The bounding box regression loss is used; by continuously minimizing this loss function, the model learns to accurately locate and identify software UI elements; during model training, the optimization problem can be formalized as:

[0043]

[0044] Where, θ * Let f represent the optimized model parameters, θ represent the model parameters, and x represent the model parameters. i Indicates the input data, y i Let θ be the corresponding label, L be the loss function, and m be the number of training samples. The parameters θ are iteratively updated using the gradient descent algorithm to gradually minimize the loss function and obtain the optimal model.

[0045] Furthermore, the trained model is validated and evaluated. If the evaluation criteria are not met, training continues. If the evaluation criteria are met, the user desktop behavior audit model is considered to have been trained successfully.

[0046] Specifically, the model is validated and evaluated to ensure it meets the expected standards in terms of recognition accuracy and generalization ability. The validation phase employs cross-validation. Specifically, the dataset is divided into k parts, with k-1 parts used as the training set each time, and the remaining parts as the validation set. This process is repeated k times, and the average of each validation iteration is used as the evaluation result. The loss value of the validation set and the loss value of the training set are calculated. By repeatedly dividing the dataset in different ways, the model can be fully trained and validated on all data, resulting in more stable and reliable evaluation results. The validation set is used to evaluate the model's performance on unseen data. The evaluation conditions are: precision not less than 90%, recall not less than 85%, F1 score not less than 88%, the difference between the loss value of the validation set and the loss value of the training set not exceeding 5%, and the model's generalization error not exceeding 0.1. If all conditions are met, the evaluation is considered successful.

[0047]

[0048]

[0049] E gen =E val -E train ≤0.1

[0050] Where TP represents the number of correctly predicted positive examples, FP represents the number of incorrectly predicted positive examples, and FN represents the number of incorrectly predicted negative examples; E genE represents the generalization error of the model, which reflects the model's ability to adapt to new data. Ideally, the generalization error should be as small as possible. val E represents the verification error. train The training error is represented by the model's average error on the validation set. After each training epoch, the user desktop behavior auditing model is used to perform inference on the validation set. The prediction results of the validation set are compared with the actual labels, and the average loss of the validation set is calculated using the loss function to obtain the validation error. The training error is represented by the model's average error on the training set. After each training epoch, the average loss on the training set is calculated using the same loss function as the validation set. If the model's performance in the validation phase does not meet the evaluation criteria, the dataset or training parameters are adjusted accordingly until the model's performance meets the expected standards.

[0051] Step 3. Acquire real-time video data. Use the trained user desktop behavior auditing model to perform image recognition on each frame of the real-time video data to obtain information O from the real-time video data. i O i This includes information about the window's type, location, and status.

[0052] Specifically, the system receives input video stream data V = {f1, f2, ..., f...} T}, where f i Let represent a frame of image captured at time point i; each frame of image undergoes object detection using a user desktop behavior auditing model to identify the currently active software window on the user's screen and its state; define the object detection function: g:f i →O i O i Indicates in frame f i The set of window objects identified in the image; the results of object detection include the window type, position, and specific state (e.g., open, minimized, etc.).

[0053] Step 4. Place O i The data is input into an LSTM network for time series analysis to predict user behavior trends; the prediction results are then validated.

[0054] Specifically, time series analysis includes: analyzing the target set {O1, O2, ..., O...} output from the user desktop behavior audit model. T As input, use an LSTM network to process O i Perform time series analysis and establish video data O i Time-dependent model h t To predict user behavior trends:

[0055] h t =LSTM(Ot ,h t-1 )

[0056] Among them, h t The hidden state at time point t represents the user's operational behavior characteristics at the current time point and its historical context. The historical context contains the behavioral characteristics and operational information of previous time steps, which helps the model capture the continuity of the user's behavior over a period of time. The user's past operational behaviors, as well as the duration and repetition frequency of specific operations, are represented by the hidden state h of the behavioral sequence. t-1 It is retained in the LSTM network.

[0057] Furthermore, LSTM accumulates user operation patterns by continuously updating hidden states; by capturing user operation patterns and performing time series analysis, it determines whether the user is following specific operating habits; where operation patterns are the user's operations exhibiting a specific workflow over a period of time, such as alternating between operations in an editor and a browser.

[0058] Validating the predictions includes defining a validity check function v:O for each identified action. t →{0,1}; further, v(O t ) = 1 indicates that the recognition result is valid, v(O) t ) = 0 indicates that the recognition result is invalid; during the verification process, if v(O) = 0, the recognition result is invalid. t If ) = 0, the system marks this frame of data as abnormal, judges the abnormal data, and if the judgment result is an image recognition error, the data is re-inputted into the user desktop behavior audit model for inference; otherwise, it jumps to the step of setting O i The input is fed into an LSTM network for time series analysis and re-prediction. The purpose of the verification mechanism is to ensure that the identified behavior conforms to the expected logic, such as that there should not be a large number of non-work-related application switching during office hours, and that frequent changes in user behavior should not deviate from normal operating habits. The valid verification result is denoted as R = {r1, r2, ..., r...}. T}, where r t This indicates the operation behavior identified at time point t, including information such as window name, operation type, and usage duration.

[0059] Step 5. Convert the verified prediction results into structured logs, add warning information to the operation records in the structured logs, perform tagging and encryption, and store the structured logs and tag data.

[0060] Specifically, the predicted behavioral results are obtained as a set R; the transformation process uses a function f to convert the prediction results into structured log entries L.

[0061] L = {l1, l2, ..., lT}

[0062] Wherein, the function f:R→L represents the log formatting function l t , l t Defined as:

[0063] l t =f(r t )=(A t ,T t C t )

[0064] Among them, A t For the application that the user operates at time point t, T t For the duration of the operation, C t This refers to the specific category of the operation; the structured recording method can describe the user's operational behavior within each time period.

[0065] Furthermore, adding warning information includes: setting a set of warning rules P:

[0066] P = {p1, p2, ..., p} n}

[0067] Where each rule p i A class of behaviors that need to be monitored is defined for log entries l t If condition p is satisfied i (l t If ) = 1, then an alarm w is generated. t This indicates that a violation was detected at time point t; for example, for non-work software (such as QQ), the rule is defined as follows:

[0068]

[0069] Where, if p i (l t If ) = 1, the system generates alarm information w. t This alerts users to any violations of operating procedures. The purpose of these alert rules is to ensure that user actions comply with established work guidelines and to address violations promptly.

[0070] Furthermore, the tagging process includes: tagging each operation record by behavior and labeling the operation category; defining a tag set C = {c1, c2, ..., c...} m}, each log entry l t Mapped to a specific category label c i This is achieved through the label function g:L→C:

[0071] c t =g(l t)

[0072] Specifically, define A t For office software (such as Word, Excel), the tag c is used. t For "office work"; if A t For a video player, the tag c t For "entertainment" purposes; let the log file be F. enc It is encrypted using the encryption function E to prevent unauthorized access:

[0073] F enc =E(F,K)

[0074] Where K represents the encryption key, F enc The encrypted log file uses a symmetric encryption algorithm (such as AES) to ensure data security during transmission and storage. Even if the encrypted log file is intercepted, it cannot be read directly, thus ensuring the confidentiality and integrity of user behavior data.

[0075] Example 2:

[0076] This invention provides a user desktop behavior auditing system based on YOLO and LSTM, comprising: a video acquisition terminal, a behavior analysis server, and a management terminal; the system flow is as follows: Figure 5 As shown.

[0077] Video capture terminal: such as Figure 1 As shown, the user's desktop video data is acquired through the HDMI loop-out encoder and the backup encoder, and the video data is transmitted to the behavior analysis server.

[0078] Specifically, the user's computer connects to the loop-out encoder via HDMI to ensure that screen data can be captured in real time and without loss. The HDMI loop-out encoder is responsible for high-fidelity acquisition of the user's computer's screen output signal and forwarding the signal to the monitor, ensuring that the user can use the computer normally without interference from the monitoring process. To ensure stable acquisition of video data, the HDMI interface uses electrical isolation to minimize the impact of external electromagnetic interference on signal quality and ensure the reliability and consistency of data transmission. The backup encoder acquires video signals to ensure that data is not lost in the event of a failure of the main device; this is called synchronous acquisition by the parallel recording device. The parallel recording device works in conjunction with the main encoder (HDMI loop-out encoder) and uses a redundancy mechanism to ensure data security and high system reliability. The redundant device (backup encoder) receives the same video signal as the main device in real time and performs synchronous recording to provide complete backup data in the event of a failure of the main device or network interruption, improving the robustness of the system. Real-time data synchronization is achieved through a high-bandwidth internal bus, thereby providing seamless failover capability and ensuring the continuity and integrity of video data under any circumstances.

[0079] Furthermore, the backup encoder and the HDMI loop-out encoder convert the captured screen content into a video stream and perform preliminary compression. The compression process uses efficient video coding technologies, such as H.264 or H.265. The compression operation can significantly reduce the data size while maintaining high video quality, enabling the system to perform real-time video transmission under bandwidth constraints. The encoder integrates a hardware compression unit to achieve synchronous video capture and compression, thereby minimizing video processing latency and ensuring the real-time performance and efficiency of subsequent processing steps.

[0080] Furthermore, the video data collection module monitors the system's CPU usage, memory status, and network bandwidth in real time. When system resources are sufficient, it increases the acquisition frequency to obtain higher resolution and more detailed video data; when system resources are scarce or network bandwidth is limited, it reduces the acquisition frequency to alleviate the system load. Video stream data is stored in local storage media. The system periodically cleans up the storage content, retaining only keyframes and important operation segments related to behavior auditing. When video data needs to be transmitted to subsequent processes, it is encrypted using secure transmission protocols (such as TLS / SSL) to ensure data security during transmission.

[0081] Behavior analysis server: processes data and performs deep learning analysis to generate user behavior audit reports; further, it includes: dataset creation and deep learning model training modules, real-time inference and behavior recognition modules, and behavior auditing and log recording modules.

[0082] Furthermore, the dataset creation and deep learning model training modules, such as Figure 2 As shown, it includes: acquiring user desktop video data, creating a training dataset based on the user desktop video data, and using the dataset to train a deep learning model; a real-time inference and behavior recognition module, as shown... Figure 3 As shown, it includes: acquiring real-time user desktop video data, using a trained deep learning model to infer from the real-time user desktop video data, predicting the software used by the user and the user's specific operational behavior in different time periods, and obtaining the prediction results; the behavior auditing and log recording module is as follows. Figure 4 As shown, this includes: converting prediction results into logs, adding alerts, tagging, and encrypting logs.

[0083] Management Terminal: Provides a graphical user interface to display detailed behavior audit results, set early warning rules, and export audit reports.

[0084] Finally, it should be noted that the above description is only one embodiment of the present invention. For those skilled in the art, various changes, modifications, substitutions and variations can be conceived of these embodiments without departing from the principles and spirit of the present invention. The scope of protection of the present invention is defined by the appended claims and their equivalents, and all the above-mentioned behaviors should be covered within the scope of protection of the present invention.

Claims

1. A user desktop behavior auditing method based on YOLO and LSTM, characterized in that, include: Step 1. Obtain desktop video data to create a dataset; The dataset is labeled, partitioned, and augmented to obtain the training set and validation set. Data annotation includes: labeling the data window type, the object being manipulated, and the user interaction behavior; Step 2. Train the user desktop behavior audit model using the training set and evaluate the user desktop behavior audit model using the validation set to obtain the trained user desktop behavior audit model; the user desktop behavior audit model uses a YOLOv8 neural network; the optimization problem during training of the user desktop behavior audit model is: ； ； in, This represents the updated model parameters. Indicates model parameters, Indicates input data, For the corresponding tags, The number of training samples, The model represents the input data. The predicted input results For loss function, Represents classification loss, Indicates the target confidence loss. This represents the bounding box regression loss; Step 3. Acquire real-time video data. Use the trained user desktop behavior auditing model to perform image recognition on each frame of the real-time video data to obtain information from the real-time video data. , This includes information about the window's type, position, and status. Step 4. The data is input into an LSTM network for time series analysis to predict user behavior trends; the prediction results are then validated; the time series analysis includes: processing the target set output by the user desktop behavior audit model. As input, an LSTM network is used to... Perform time series analysis and establish Time-dependent model To predict user behavior trends: ； in, Indicates at a point in time The hidden state captures the user's operational behavior characteristics at the current point in time and its historical context; the historical context contains the behavioral characteristics and operational information of previous time steps, which helps the model capture the continuity of the user's behavior over a period of time; the user's past operational behaviors, as well as the duration and repetition frequency of specific operations, are represented by the hidden state of the behavioral sequence. Retained in the LSTM network; Step 5. Convert the verified prediction results into structured logs, add warning information to the operation records in the structured logs, perform tagging and encryption, and store the structured logs and tag data.

2. The user desktop behavior auditing method based on YOLO and LSTM according to claim 1, characterized in that, Creating the dataset involves: setting a threshold k=50, extracting keyframes from the collected video data based on the threshold, and the set of keyframes constitutes the dataset; the formula for keyframe extraction is: ； in, Representing a frame and frame The average pixel difference between the two images, where N represents the average pixel difference between the two images. and The sum of the number of all pixels in the array. For frames medium pixel The value when the average pixel difference When the value is greater than k, extract keyframes. .

3. The user desktop behavior auditing method based on YOLO and LSTM according to claim 1, characterized in that, The evaluation of the user desktop behavior auditing model using the validation set includes: calculating the precision, recall, F1 score, loss value of the validation set, loss value of the training set, and generalization error of the model; the precision is greater than or equal to 90%, the recall is greater than or equal to 85%, the F1 score is greater than or equal to 88%, the difference between the loss value of the validation set and the loss value of the training set is less than or equal to 5%, and the generalization error of the model is less than or equal to 0.

1. Training ends when all evaluation conditions are met.

4. The user desktop behavior auditing method based on YOLO and LSTM according to claim 1, characterized in that, Validating the prediction results includes defining a validity check function. The system outputs the prediction results that are verified as valid, marks the frame data corresponding to the prediction results that are verified as invalid as abnormal frame data, determines the source of the abnormality of the frame data, and performs image recognition or time series analysis again based on the determination result.

5. A user desktop behavior auditing system based on YOLO and LSTM, the system being used to execute the user desktop behavior auditing method based on YOLO and LSTM as described in any one of claims 1 to 4, characterized in that, The system includes: video acquisition terminals, behavior analysis servers, and management terminals; Video capture terminal: Acquires user desktop video data through HDMI loop-out encoder and backup encoder, and transmits the video data to behavior analysis server; Behavior analysis server: processes data and performs deep learning analysis to generate user behavior audit reports; Management Terminal: Provides a graphical user interface to display detailed behavior audit results, set early warning rules, and export audit reports.

6. The user desktop behavior auditing system based on YOLO and LSTM according to claim 5, characterized in that, The behavior analysis server includes: a dataset creation and deep learning model training module, a real-time inference and behavior recognition module, and a behavior auditing and logging module; Dataset creation and deep learning model training module: acquire user desktop video data, create a training dataset based on the user desktop video data, and use the dataset to train a deep learning model. Real-time inference and behavior recognition module: acquires real-time user desktop video data, uses a trained deep learning model to infer the real-time user desktop video data, predicts the software used by the user and the user's specific operation behavior in different time periods, and obtains the prediction results. Behavior auditing and logging module: Converts prediction results into logs, adds alerts, handles tagging, and encrypts logs.

7. The user desktop behavior auditing system based on YOLO and LSTM according to claim 5, characterized in that, The system adopts a distributed architecture, and the terminal and the server interact with each other through API interfaces.

Images