User desktop behavior auditing method and system based on YOLO and LSTM

Abstract

The invention relates to the field of auditing systems, in particular to a user desktop behavior auditing method and system based on YOLO and LSTM, and the method comprises the steps: obtaining desktop video data to make a training data set; training a user desktop behavior auditing model by adopting the data set after data enhancement; acquiring real-time video data, and analyzing the real-time video data frame by frame by using the trained user desktop behavior auditing model; predicting an operation behavior trend of the user according to the analyzed real-time video data; the prediction result is verified; and converting the verified prediction result into a structured log. According to the method, modeling is carried out on the time sequence data by using the long-short-term memory network, and a YOLOv8 deep learning framework is combined, so that the system can identify specific software used by a user, and can deduce the task type executed by the user, thereby remarkably improving the behavior identification accuracy and improving the auditing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of audit systems, and in particular to a user desktop behavior audit method and system based on YOLO and LSTM. Background Art

[0002] In the context of the rapid development of information technology, the audit and management of desktop operation behaviors has gradually become one of the important needs of enterprise information security and productivity management. Especially with the increasing popularity of remote office and hybrid office mode, how to effectively obtain user operation behavior information on the computer and conduct risk assessment, productivity analysis, etc. has become particularly important. However, the existing user desktop behavior auditing methods mostly rely on simple logging, keyboard and mouse operation tracking, which have many limitations. With the continuous advancement of deep learning technology, its superiority in image recognition, behavior analysis and other fields has gradually emerged, which provides a new solution for desktop behavior auditing.

[0003] The user desktop behavior audit method based on deep learning can conduct in-depth analysis of user operation behaviors through screen recording data, providing a more detailed and intelligent understanding of behavior. However, research in this field is still blank, lacking mature methods and public data set support. Traditional behavior audit methods can usually only provide relatively simple operation records, such as software startup time, keyboard tapping times, etc., and cannot intuitively reflect the complex operations performed by users on the screen. These methods lack in-depth semantic understanding of user behavior, making it difficult to determine what specific tasks users are performing or how they are using the software. Traditional methods are often unable to effectively deal with diverse and unstructured data, resulting in a large lack of information in the records of user operations, making it difficult to conduct further intelligent analysis and prediction.

[0004] Therefore, existing technologies have obvious deficiencies in data integrity and in-depth analysis, and cannot meet the behavioral auditing needs in complex application scenarios. Summary of the invention

[0005] In view of this, the present invention aims to solve the problems of insufficient depth of operation auditing and limited behavior understanding ability in the prior art, so as to effectively improve the visual management ability of user operations.

[0006] The user desktop behavior audit method based on YOLO and LSTM of the present invention includes:

[0007] Step 1. Obtain desktop video data to create a data set, annotate, divide, and perform data enhancement operations on the data set to obtain a training set and a validation set;

[0008] Step 2. Use the training set to train the user desktop behavior audit model, and use the validation set to evaluate the user desktop behavior audit model. The user desktop behavior audit model uses the YOLOv8 neural network to obtain a trained user desktop behavior audit model.

[0009] Step 3. Obtain real-time video data, use the trained user desktop behavior audit model to perform image recognition on the real-time video data frame by frame, and obtain the information of the real-time video data. i , O i Including window type, location and status information;

[0010] Step 4. i Input into the LSTM network for time series analysis to predict the user's operation behavior trend; verify the prediction results;

[0011] Step 5. Convert the verified prediction results into structured logs, add warning information to the operation records in the structured logs, perform labeling and encryption, and store the structured logs and label data.

[0012] A user desktop behavior audit system based on YOLO and LSTM, the system adopts a distributed architecture, and each module interacts with each other through an API interface, including: a video acquisition terminal, a behavior analysis server, and a management terminal connected in sequence.

[0013] Video acquisition terminal: obtains user desktop video data through HDMI loop-out encoder and backup encoder, and transmits the video data to the behavior analysis server; behavior analysis server: processes and performs deep learning analysis on the data to generate user behavior audit reports; management terminal: provides a graphical user interface to display detailed behavior audit results, set warning rules, and export audit reports.

[0014] The beneficial effects of the present invention are:

[0015] By focusing on the fine-grained recognition of user operation behaviors, using long short-term memory networks (LSTM) to model time series data, and further combining the YOLOv8 deep learning framework, the system can identify the specific software used by users and infer the types of tasks performed by users, thereby effectively improving the accuracy of behavior recognition and thus improving audit efficiency; by performing fine-grained classification and labeling of user behaviors and setting up alarm mechanisms based on specific rules, the comprehensiveness of behavior monitoring and the management value of the system are greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the operation of the video acquisition terminal of the present invention;

[0017] Figure 2A schematic diagram of the process of data set preparation and deep learning model training module described in the present invention;

[0018] Figure 3 A schematic diagram of the process of implementing the reasoning and behavior recognition module of the present invention;

[0019] Figure 4 This is a flow chart of the behavior audit and log recording module of the present invention;

[0020] Figure 5 This is a schematic diagram of the operations included in the deep learning-based user desktop behavior audit system of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution, characteristics and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0022] Embodiment 1:

[0023] The present invention provides a user desktop behavior audit method based on YOLO and LSTM, comprising:

[0024] Step 1. Obtain desktop video data to create a data set, label, divide, and perform data enhancement operations on the data set to obtain a training set and a validation set.

[0025] Specifically, the data set preparation includes: loading and decoding the screen video data through an automated script to obtain the original video frame, using OpenCV to read the video and extract the image data frame by frame, and each frame is converted into a standard RGB image format for subsequent processing; extracting key frames that are rich in information and representative frame by frame to reduce data redundancy and retain the core content of user operations; wherein, the extraction of key frames is based on the setting of the threshold k of the image content change between frames: when the change between video frames exceeds the threshold k, the frame is retained as a key frame, and k is preferably 50, which can be adjusted according to actual needs; the conditions for key frame extraction are defined as follows:

[0026]

[0027] Among them, D(f i ,f i+1 ) represents frame f i and frame f i+1 The average pixel difference between Represents frame f i The value of the pixel (x, y) in the i ,f i+1 ) is greater than k, extract the key frame f i+1 .

[0028] The annotation operations include: using the CVAT tool to annotate the extracted key frames, and the annotation content includes window type, operation object, user interaction behavior, etc.; the CVAT tool supports semi-automatic annotation based on existing models to reduce the workload of manual annotation and improve annotation consistency and efficiency; the annotation process combines automated tools and manual verification to ensure the accuracy of annotation and the high consistency of data quality.

[0029] The partitioning operation includes: partitioning the original data set into a training set, a validation set, and sometimes a test set; the training set is used to train the model, that is, to let the model learn the characteristics and patterns of desktop operation behaviors, and gradually adjust the parameters of the model so that the model can perform well on the training data; the validation set is used to evaluate the performance of the model during the training process; when the model completes several training cycles, the validation set is used to evaluate the performance of the model to determine whether there is overfitting, underfitting, etc.; the data in the validation set is not used for training, and the validation set can reflect the performance of the model on new data, thereby helping to adjust the model parameters.

[0030] Data enhancement includes: rotating, scaling, and adjusting contrast of the image to obtain a data-enhanced dataset;

[0031] Furthermore, data enhancement is the core strategy to improve the generalization ability of the model. It generates diversified data samples by performing various transformations on the original data, thereby effectively reducing the risk of model overfitting. The data enhancement process can be expressed as:

[0032] I′=T(I)

[0033] Where I is the original image, T is the data enhancement transformation operation set, including rotation R(θ), scaling S(s), contrast adjustment C(c), etc.; the transformed image I′ can be used to enrich the data sample space and improve the robustness of the model; the probability distribution P(I′) after data enhancement can be expressed as:

[0034] P(I′)=P(T)P(I)dT

[0035] Among them, P(T) represents the probability distribution of the data augmentation transformation T, and P(I) represents the probability distribution of the original image I. By optimizing the probability distribution P(T) of different transformation operations, the model finally has better adaptability and generalization performance.

[0036] Furthermore, the optimization of P(T) includes: selecting multiple types of image transformation operations in data enhancement, such as rotation, translation, scaling, cropping, flipping, brightness adjustment, and adding noise; optimizing P(T) to determine the probability of each transformation operation being used, and adjusting it through the automatic search data enhancement strategy AutoAugment algorithm to ensure that each transformation can appear appropriately during the training process, thereby enriching the diversity of the data set; for example, if it is found that the model does not perform well in handling illumination changes, the probability of illumination change transformation is increased so that the model can obtain more illumination change data during training;

[0037] Specifically, through reinforcement learning, the best enhancement operation and the corresponding probability distribution P(T) are selected to maximize the model performance on the validation set, so that the model has better generalization performance; the optimization of data enhancement can be tuned through the feedback of the model on the validation set; specifically, the initial enhancement operation probability P(T) can be set, and then the probability can be adjusted by observing the performance of the model on the validation set; for example, certain specific transformations (such as rotation) can significantly improve the accuracy of the validation set, then its probability can be increased, otherwise it can be reduced.

[0038] Step 2. Use the training set to train the user desktop behavior audit model, and use the validation set to evaluate the user desktop behavior audit model. The user desktop behavior audit model uses the YOLOv8 neural network to obtain a trained user desktop behavior audit model.

[0039] Specifically, during the training process, the data set is trained in batches, and during the training process, the model and data are loaded into the GPU for calculation to increase the training speed.

[0040] Among them, batch training refers to dividing the entire data set into several smaller batches and training the model batch by batch, which is different from sending all the data to the model at once. Assuming that the data set has N samples, it can be divided into several batches, each batch size is B (called Batch Size), and the model only processes B samples in each iteration; the batch training method is used to solve the problem of limited computing resources and accelerate the convergence speed of the model; batch processing of large data sets can reduce computing overhead, avoid insufficient memory problems, and allow model training to be completed in a smaller memory; YOLOv8 is an efficient convolutional neural network, which is used in the present invention to identify application windows and user interaction objects in desktop operations; the loss function L in the training process can be expressed as:

[0041] L=L cls +L obj +L bbox

[0042] Among them, L cls is the classification loss, Lobj is the target confidence loss, L bbox is the bounding box regression loss. By continuously minimizing this loss function, the model learns to accurately locate and identify software UI elements. During the model training process, the optimization problem can be formalized as:

[0043]

[0044] Among them, θ * represents the optimized model parameters, f represents, θ represents the model parameters, x i Represents input data, y i is the corresponding label, L is the loss function, and m is the number of training samples. The parameter θ is iteratively updated through the gradient descent algorithm, and the loss function is gradually minimized to obtain the optimal model.

[0045] Furthermore, the trained model is verified and evaluated. If the evaluation conditions are not met, the training continues. If the evaluation conditions are met, the user desktop behavior audit model training is considered complete.

[0046] Specifically, the model is verified and evaluated to ensure that it meets the expected standards in terms of recognition accuracy and generalization ability. The cross-validation method is used in the verification stage. Specifically, the data set is divided into k parts, and k-1 parts are used as training sets each time, and the remaining parts are used as verification sets. This is repeated k times, and the average value of each verification is taken as the evaluation result. The loss value of the verification set and the loss value of the training set are calculated. By repeatedly dividing the data set into different parts, the model can be fully trained and verified on all data, thereby obtaining more stable and reliable evaluation results. The verification set is used to evaluate the performance of the model on unseen data, where the evaluation conditions are: precision (Precision) is not less than 90%, recall rate (Recall) is not less than 85%, F1 score (F1 Score) is not less than 88%, the difference between the loss value of the verification set and the loss value of the training set does not exceed 5%, and the generalization error of the model does not exceed 0.1. If all conditions are met, the evaluation is considered to have passed.

[0047]

[0048]

[0049] E gen =E val -E train ≤0.1

[0050] Among them, TP represents the number of correctly predicted positive examples, FP represents the number of incorrectly predicted positive examples, and FN represents the number of incorrectly predicted negative examples; E genIt represents the generalization error of the model. The generalization error reflects the adaptability of the model to new data. Ideally, the generalization error should be as small as possible. val represents the verification error, E train Represents the training error; the validation error is the average error of the model on the validation set. After each round of training, the validation set is used to infer the user desktop behavior audit model, and the prediction results of the validation set are compared with the actual labels. The loss function is used to calculate the average loss of the validation set to obtain the validation error; the training error is the average error of the model on the training set. After each training epoch, the average loss on the training set is calculated using the same loss function as the validation set. If the performance of the model in the validation phase does not meet the evaluation conditions, the data set or training parameters are adjusted accordingly until the performance of the model meets the expected standards.

[0051] Step 3. Obtain real-time video data, use the trained user desktop behavior audit model to perform image recognition on the real-time video data frame by frame, and obtain the information of the real-time video data. i , O i Includes window type, location, and status information.

[0052] Specifically, the system receives input video stream data V = {f1, f2, ..., f T}, where f i represents a frame of image captured at time point i; each frame of image is subjected to target detection through the user desktop behavior audit model to identify the currently active software window and its status on the user's screen; define the target detection function: g:f i →O i , where O i Indicates that in frame f i The result of target detection includes the type, location and specific status of the window (such as open, minimized, etc.).

[0053] Step 4. i Input into the LSTM network for time series analysis to predict the user's operation behavior trend; verify the prediction results.

[0054] Specifically, the time series analysis includes: converting the target set {O1, O2, …, O T} as input, use LSTM network to train O i Perform time series analysis and build video data i Time-dependent model h t , to predict the user's operation behavior trend:

[0055] h t =LSTM(Ot ,h t-1 )

[0056] Among them, h t represents the hidden state at time point t, capturing the user's operation behavior characteristics and historical context at the current time point; the historical context contains the behavior characteristics and operation information of the previous time step, which is used to help the model capture the continuity of the user's behavior over a period of time; the user's past operation behavior and the duration and repetition frequency of specific operations are hidden in the behavior sequence h t-1 Retained in the LSTM network.

[0057] Furthermore, LSTM accumulates the user's operation mode by continuously updating the hidden state; by capturing the user's operation mode, time series analysis is performed to determine whether the user is following specific operation habits; where the operation mode is the user's operation over a period of time that exhibits a specific workflow, such as alternating operations between the editor and the browser.

[0058] Verifying the prediction includes: for each identified operation behavior, defining a validity verification function v: t →{0,1}; further, v(O t )=1 means the recognition result is legal, v(O t )=0 means the recognition result is illegal; during the verification process, if v(O t )=0, the system marks this frame data as abnormal, judges the abnormal data, and when the judgment result is an image recognition error, re-enters the data into the user desktop behavior audit model for reasoning; if not, jumps to O i The input is sent to the LSTM network for time series analysis and re-prediction. The purpose of the verification mechanism is to ensure that the identified behavior conforms to the expected logic. For example, there should not be a large number of non-work-related application switches during office hours, and the frequent changes in user behavior should not deviate from normal operating habits. The legal verification result is recorded as R = {r1, r2, …, r T}, where r t Indicates the operation behavior recognized at time point t, including information such as window name, operation type, and usage duration.

[0059] Step 5. Convert the verified prediction results into structured logs, add warning information to the operation records in the structured logs, perform labeling and encryption, and store the structured logs and label data.

[0060] Specifically, the predicted behavior results are obtained as a set R; the conversion process converts the prediction results into structured log entries L through the function f:

[0061] L={l1,l2,…,lT}

[0062] Among them, the function f:R→L represents the log formatting function l t , l t Defined as:

[0063] l t =f(r t )=(A t ,T t ,C t )

[0064] Among them, A t is the application operated by the user at time point t, T t is the operation duration, C t It is a specific category of operation; the structured recording method can describe the user's operation behavior in each time period.

[0065] Furthermore, adding warning information includes: setting a set of warning rule sets P:

[0066] P={p1,p2,…,p n}

[0067] Among them, each rule p i Defines a class of behaviors that need to be monitored. For log entries l t , if the condition p is satisfied i (l t )=1, then generate alarm w t , indicating that a violation is detected at time point t; for example, for non-working software (such as QQ), define the rule:

[0068]

[0069] Among them, if p i (l t )=1, the system generates an alarm message w t , prompting users of illegal operations; the setting of early warning rules is to ensure that users' operating behaviors comply with established work specifications and to deal with illegal operations in a timely manner when they occur.

[0070] Furthermore, labeling includes: labeling each operation record, marking the operation category; defining a label set C = {c1, c2, ..., c m}, each log entry l t is mapped to a specific class label c i , which is achieved through the label function g:L→C:

[0071] c t =g(l t)

[0072] Specifically, define A t For office software (such as Word, Excel), label c t is "office"; if A t For a video player, label c t is "entertainment"; let the log file be F enc , use the encryption function E to encrypt it to prevent unauthorized access:

[0073] F enc =E(F,K)

[0074] Where K represents the encryption key, F enc It is an encrypted log file. The encryption process uses a symmetric encryption algorithm (such as AES) to ensure the security of data during transmission and storage. Even if the encrypted log file is intercepted, it cannot be directly read, thereby ensuring the confidentiality and integrity of user behavior data.

[0075] Embodiment 2:

[0076] The present invention provides a user desktop behavior audit system based on YOLO and LSTM, including: a video acquisition terminal, a behavior analysis server, and a management terminal; the system process is as follows Figure 5 shown.

[0077] Video acquisition terminal: Figure 1 As shown, the user's desktop video data is obtained through the HDMI loop-out encoder and the backup encoder, and the video data is transmitted to the behavior analysis server.

[0078] Specifically, the user computer is connected to the loop-out encoder via HDMI to ensure that the screen data can be captured in real time and losslessly; the HDMI loop-out encoder is responsible for high-fidelity acquisition of the screen output signal of the user computer and forwarding the signal to the display to ensure that the user can use the computer normally without being disturbed by the monitoring process; in order to ensure the stable acquisition of video data, the HDMI interface adopts electrical isolation to minimize the impact of external electromagnetic interference on signal quality and ensure the reliability and consistency of data transmission; the backup encoder obtains the video signal to ensure that the data will not be lost when the main device fails, which is called the synchronous acquisition of the parallel recording device; the parallel recording device works in conjunction with the main encoder (HDMI loop-out encoder) and adopts a redundancy mechanism to ensure data security and high reliability of the system; the redundant device (backup encoder) receives the same video signal as the main device in real time and performs synchronous recording to provide complete backup data when the main device fails or the network is interrupted, thereby improving the robustness of the system; real-time data synchronization is achieved through a high-bandwidth internal bus, thereby providing seamless fault switching capabilities to ensure the continuity and integrity of video data under any circumstances.

[0079] Furthermore, the backup encoder and HDMI loop-out encoder convert the captured screen content into a video stream and perform preliminary compression; the compression process adopts high-efficiency video encoding technology, such as H.264 or H.265. The compression operation can maintain high video quality while significantly reducing the data volume, allowing the system to perform real-time video transmission under bandwidth constraints; the encoder integrates a hardware compression unit to achieve simultaneous video capture and compression, thereby minimizing the delay in video processing and ensuring the real-time and efficiency of subsequent processing steps.

[0080] Furthermore, the video data collection module monitors the system's CPU usage, memory status and network bandwidth in real time. When system resources are sufficient, the acquisition frequency is increased to obtain higher resolution and more detailed video data; when system resources are tight or network bandwidth is limited, the acquisition frequency is reduced to reduce the system burden; video stream data is stored in local storage media; the system will regularly clean up the storage content, retaining only key frames and important operation fragments related to behavioral auditing; when video data needs to be transmitted to subsequent processes, it is encrypted through a secure transmission protocol (such as TLS / SSL) to ensure data security during transmission.

[0081] Behavior analysis server: processes and performs deep learning analysis on data to generate user behavior audit reports; further, it includes: data set creation and deep learning model training module, real-time reasoning and behavior recognition module, behavior audit and log recording module.

[0082] Furthermore, dataset preparation and deep learning model training modules are Figure 2 As shown, it includes: obtaining user desktop video data, making a training data set based on the user desktop video data, and using the data set to train a deep learning model; real-time reasoning and behavior recognition module as shown Figure 3 As shown in the figure, it includes: obtaining real-time user desktop video data, using the trained deep learning model to infer the real-time user desktop video data, predicting the software used by the user in different time periods and the user's specific operation behavior, and obtaining the prediction results; the behavior audit and log recording module is as shown in the figure. Figure 4 As shown, it includes: converting prediction results into logs, adding warnings, label processing, and log encryption operations.

[0083] Management terminal: provides a graphical user interface to display detailed behavior audit results, set warning rules, and export audit reports.

[0084] Finally, it should be noted that the above only describes one embodiment of the present invention. For those skilled in the art, it is conceivable that various changes, modifications, substitutions and deformations may be made to these embodiments without departing from the principle and spirit of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents, and the above-mentioned actions should be covered within the scope of protection of the present invention.

Claims

1. A user desktop behavior audit method based on YOLO and LSTM, characterized in that: include: Step 1. Obtain desktop video data to create a data set; label, divide, and perform data enhancement operations on the data set to obtain a training set and a validation set; Step 2. Use the training set to train the user desktop behavior audit model, use the validation set to evaluate the user desktop behavior audit model, and obtain a trained user desktop behavior audit model; the user desktop behavior audit model uses the YOLOv8 neural network; Step 3. Obtain real-time video data, use the trained user desktop behavior audit model to perform image recognition on the real-time video data frame by frame, and obtain the information of the real-time video data. i , O i Including window type, location and status information; Step 4. i Input into the LSTM network for time series analysis to predict the user's operation behavior trend; verify the prediction results; Step 5. Convert the verified prediction results into structured logs, add warning information to the operation records in the structured logs, perform labeling and encryption, and store the structured logs and label data.

2. The user desktop behavior audit method based on YOLO and LSTM according to claim 1 is characterized in that: The data set preparation includes: setting a threshold value k=50, extracting key frames from the collected video data according to the threshold value, and the set of key frames constitutes the data set; the key frame extraction condition formula is: Among them, D(f i ,f i+1 ) represents frame f i and frame f i+1 The average pixel difference between the two images, N represents the image and The sum of all pixel points in For frame f i The value of the pixel (x, y) in the i ,f i+1 ) is greater than k, extract the key frame f i+1 .

3. The user desktop behavior audit method based on YOLO and LSTM according to claim 1 is characterized in that: Data labeling includes: marking data window type, operation object, and user interaction behavior.

4. The user desktop behavior audit method based on YOLO and LSTM according to claim 1 is characterized in that: When training the user desktop behavior audit model, the optimization problem of training is: L=L cls +L obj +L bbox Among them, θ * represents the updated model parameters, θ represents the model parameters, x i Represents the input data, y i is the corresponding label, m is the number of training samples, f(x i ; θ) represents the model's response to the input data x i The predicted input result, L is the loss function, L cls represents the classification loss, L obj represents the target confidence loss, L bbox represents the bounding box regression loss.

5. The user desktop behavior audit method based on YOLO and LSTM according to claim 1 is characterized in that: Using the validation set to evaluate the user desktop behavior audit model includes: calculating the accuracy, recall rate, F1 score, loss value of the validation set, loss value of the training set, and generalization error of the model; the accuracy is greater than or equal to 90%, the recall rate is greater than or equal to 85%, the F1 score is greater than or equal to 88%, the difference between the loss value of the validation set and the loss value of the training set is less than or equal to 5%, and the generalization error of the model is less than or equal to 0.

1. Training ends when all evaluation conditions are met.

6. The user desktop behavior audit method based on YOLO and LSTM according to claim 1 is characterized in that: Verifying the prediction results includes: defining a legality verification function v, outputting the prediction results that are verified to be legal, marking the frame data corresponding to the prediction results that are verified to be illegal as abnormal frame data, determining the abnormal source of the frame data, and re-performing image recognition or time series analysis based on the determination result.

7. A user desktop behavior audit system based on YOLO and LSTM, the system is used to execute the user desktop behavior audit method based on YOLO and LSTM according to any one of claims 1 to 6, characterized in that: The system includes: video acquisition terminal, behavior analysis server, and management terminal; Video acquisition terminal: obtains user desktop video data through HDMI loop-out encoder and backup encoder, and transmits the video data to the behavior analysis server; Behavior analysis server: processes and performs deep learning analysis on data to generate user behavior audit reports; Management terminal: provides a graphical user interface to display detailed behavior audit results, set warning rules, and export audit reports.

8. The user desktop behavior audit system based on YOLO and LSTM according to claim 7 is characterized in that: The behavior analysis server includes: data set creation and deep learning model training module, real-time reasoning and behavior recognition module, behavior audit and log recording module; Dataset creation and deep learning model training module: obtain user desktop video data, create a training dataset based on the user desktop video data, and use the dataset to train the deep learning model; Real-time reasoning and behavior recognition module: obtains real-time user desktop video data, uses the trained deep learning model to reason about the real-time user desktop video data, predicts the software used by the user in different time periods and the user's specific operation behavior, and obtains the prediction results; Behavior auditing and logging module: convert prediction results into logs, add warnings, process labels, and encrypt logs.

9. The user desktop behavior audit system based on YOLO and LSTM according to claim 7, characterized in that: The system adopts a distributed architecture, and the terminal and server interact with each other through the API interface.

Images