An AI model-based periodic screenshot analysis method and computer system

Abstract

The application relates to the field of artificial intelligence analysis, in particular to a periodic screenshot analysis method based on an AI model and a computer system. The method generates a screenshot task in response to the triggering of a preset screenshot module, calls a preset acquisition module to acquire images of a target window of an edge node according to the screenshot task, obtains screenshot data and a benchmark interface screenshot, inputs the screenshot data and the benchmark interface screenshot into a pre-trained cloud AI analysis module, calculates analysis processing data, adopts an AI model based on a twin neural network architecture, calls a preset anomaly detection module to model and predict time series of the analysis processing data, calculates an anomaly prediction value, and finally judges whether a difference between an actual observation value and the anomaly prediction value is greater than an error threshold value, and if yes, triggers an alarm. Compared with the prior art, the method solves the problems of low screenshot processing efficiency and inaccurate analysis and prediction results.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence analysis. More particularly, the present application relates to an AI model-based periodic screenshot analysis method and computer system. BACKGROUND

[0002] With the increasing richness of mobile terminal and desktop software applications and the acceleration of product iteration speed, the interactive interface and dynamic message of these software change more and more frequently. In order to realize real-time monitoring of the above-mentioned software interactive interface and dynamic message, in the traditional technology, the information uploading and manual analysis comparison are mainly realized by means of log, event reporting or manual screenshot. This method is time-consuming, has high misjudgment rate and high cost. Especially for screenshot analysis, only single event or manual triggering action is often focused on, which is difficult to meet the continuous monitoring needs of large-scale and distributed systems.

[0003] In view of the above technical problems related to screenshot analysis, the Chinese application file with publication number CN119516564A discloses an intelligent task management method based on OCR and AI technology. It mainly analyzes image data by intercepting task message screenshots through shortcut keys. This process relies on OCR technology for analysis, which has the following defects:

[0004] Firstly, the degree of automation is not high, and the triggering screenshot action still needs manual assistance.

[0005] Secondly, OCR technology, i.e. Optical Character Recognition, is a technology that converts the text information in images into editable and searchable text format. This technology is limited by its underlying algorithm kernel, and in the real-time processing of a large number of screenshots, the processing efficiency is low and the analysis prediction result is inaccurate.

[0006] Therefore, the existing technology mainly has the problems of low screenshot processing efficiency and inaccurate analysis prediction result. SUMMARY

[0007] To solve the above technical problems of low screenshot processing efficiency and inaccurate analysis prediction result, the present application discloses an AI model-based periodic screenshot analysis method and computer system.

[0008] In a first aspect, the present application discloses an AI model-based periodic screenshot analysis method, comprising:

[0009] In response to the triggering of a preset screenshot module, a screenshot task is generated;

[0010] According to the screenshot task, a preset acquisition module is called to perform image acquisition on the target window of the edge node, so as to obtain screenshot data and a reference interface screenshot.

[0011] The screenshot data and the benchmark interface screenshot are input into a pre-trained cloud AI analysis module to obtain analysis processing data through calculation;

[0012] A preset anomaly detection module is called to model and predict the time sequence of the analysis processing data to obtain an anomaly prediction value through calculation;

[0013] It is determined whether the difference between the actual observation value and the anomaly prediction value is greater than an error threshold value, and if so, an alarm is triggered.

[0014] Beneficial effects: The method of the present application first generates a screenshot task of an edge node through a screenshot module, and then calls a collection module to obtain image data of a target window of the edge node to obtain screenshot data and a benchmark interface screenshot, thereby realizing automatic interception of the screenshot action and solving the problem of low automation degree in the prior art. The screenshot data and the benchmark interface screenshot are input into an AI analysis module in the cloud to obtain analysis processing data through calculation, and an anomaly detection module is called to model and predict the time sequence of the analysis processing data to obtain an anomaly prediction value through calculation. This method fully utilizes the data processing kernel of the twin network architecture, improves the overall data processing efficiency, and solves the problem of low screenshot processing efficiency in the prior art. By modeling and predicting the time sequence of the analysis processing data through the anomaly detection module, a more accurate anomaly prediction value can be obtained. Finally, the difference between the actual observation value and the accurate anomaly prediction value is obtained to obtain a more accurate analysis prediction result, and the threshold value is relied on to trigger an alarm, further improving the automation degree and accuracy of the method of the present application.

[0015] Preferably, the anomaly detection module includes an LSTM (Long Short-Term Memory) model and an ARIMA (Autoregressive Integrated Moving Average) model; the analysis processing data is modeled and predicted in time sequence to obtain an anomaly prediction value, specifically:

[0016] The LSTM model is called to predict the similarity value of the analysis processing data at the next time, and a non-linear prediction result is generated according to a plurality of similarity values;

[0017] The ARIMA model is called to statistically predict the similarity sequence of the analysis processing data to generate a linear prediction result;

[0018] The non-linear prediction result and the linear prediction result are weighted and fused to obtain the anomaly prediction value.

[0019] Beneficial effects: The anomaly detection module integrates an LSTM model and an ARIMA model, wherein the LSTM model is suitable for processing data with complex nonlinear relationships, and the ARIMA model is suitable for processing data with linear relationships, and the two models have advantages and disadvantages, and the method combines the two models organically in a weighted fusion manner, so that the advantages of the LSTM model and the ARIMA model are maximized, and the calculated anomaly prediction value is more accurate.

[0020] Preferably, if the number of anomaly prediction values is greater than or equal to a preset threshold, the method further comprises:

[0021] adding the anomaly prediction value to the sample training set, and updating the model weight of the anomaly detection module based on an incremental learning mechanism or a transfer learning mechanism.

[0022] Beneficial effects: The method uses incremental training or transfer learning to update the model weight, which reduces the cost of full retraining and ensures the long-term stability and efficiency of the model.

[0023] Preferably, the calculation expression for updating the model weight of the anomaly detection module is:

[0024]

[0025] In the formula, w i represents the weight coefficient of the i-th model, represents the average reconstruction error of the i-th model in the new sample training set, and δ represents a constant greater than 0, represents the summation calculation from j=1 to j=K, and K represents the total number of models.

[0026] Preferably, the acquisition module includes a virtualization interface, and a preset acquisition module is called to acquire images of the target window of the edge node to obtain screenshot data, including:

[0027] The virtualization interface is called to acquire data of the target window of the edge node to obtain pixel data.

[0028] The pixel data is compressed and encrypted to obtain the screenshot data.

[0029] Beneficial effects: Compressing and encrypting the pixel data can optimize the data storage and transmission efficiency while enhancing the security of data transmission.

[0030] Preferably, the screenshot data and the reference interface screenshot are input into a pre-trained cloud AI analysis module to calculate analysis processing data, including:

[0031] extracting visual features of the reference interface screenshot;

[0032] The visual features and the screenshot data are input into a pre-trained cloud AI analysis module to obtain analysis processing data.

[0033] Preferably, before the screenshot data and the reference interface screenshot are input into the pre-trained cloud AI analysis module, the method further comprises:

[0034] The screenshot data and the reference interface screenshot located at the edge node are transmitted to the cloud AI analysis module through a preset secure channel.

[0035] Preferably, before the screenshot data and the reference interface screenshot are input into the pre-trained cloud AI analysis module, the method further comprises:

[0036] In the edge node, the reference interface screenshot is subjected to image preprocessing.

[0037] Preferably, the screenshot module adopts a program script based on a Cron expression.

[0038] In a second aspect, the application discloses a computer system comprising a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the AI model-based periodic screenshot analysis method of the first aspect is realized.

[0039] The application has the following beneficial effects:

[0040] (1) Compared with the prior art, the method fully utilizes the data processing kernel of the twin network architecture, improves the overall data processing efficiency, and solves the problem of low screenshot processing efficiency in the prior art. On this basis, the method further models and predicts the time sequence of the analysis processing data through the anomaly detection module to obtain an anomaly prediction value. This way can obtain more accurate analysis and prediction results.

[0041] (2) Compared with the prior art, the method realizes the organic combination of the two models through the weighted fusion manner, maximizes the advantages of the LSTM model and the ARIMA model, and further makes the anomaly prediction value more accurate.

[0042] (3) Compared with the prior art, the method adopts the incremental training or transfer learning manner to update the model weight, which reduces the cost of full retraining, and guarantees the long-term stability and high efficiency of the model. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a flowchart of the AI model-based periodic screenshot analysis method in the first embodiment of the application;

[0044] Figure 2 is a structural schematic diagram of the computer system in the second embodiment of the application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.

[0046] The specific embodiments of the present application will be described in detail below with reference to the drawings.

[0047] Embodiment one

[0048] As shown in the figure, the present embodiment discloses an AI model-based periodic screenshot analysis method, which comprises: Figure 1 S10: In response to the triggering of a preset screenshot module, a screenshot task is generated.

[0049] In the present embodiment, the screenshot module is a program script configured in the edge node and based on Cron expression. When the edge node is running, the screenshot module is automatically loaded to generate a screenshot task.

[0050] S20: According to the screenshot task, a preset acquisition module is called to perform image acquisition on the target window of the edge node to obtain screenshot data and a reference interface screenshot.

[0051] In the present embodiment, the screenshot data refers to the edge node screen pixel data after preliminary compression and encryption processing. The edge node can be a smartphone or other industrial control terminal. The screenshot data belongs to real-time screenshot data, and the reference interface screenshot belongs to a comparison reference image.

[0052] S30: The screenshot data and the reference interface screenshot are input into a pre-trained cloud AI analysis module to calculate analysis processing data.

[0053] Among them, the cloud AI analysis module adopts an AI model based on a twin neural network architecture. The cloud AI analysis module is configured in a cloud analysis server (cloud server).

[0054] S40: A preset anomaly detection module is called to model and predict the time series of the analysis processing data to calculate an anomaly prediction value.

[0055] S50: It is judged whether the difference between the actual observation value and the anomaly prediction value is greater than the error threshold value. If yes, an alarm is triggered.

[0056] In the present embodiment, the actual observation value can be taken from the reference interface screenshot, which is generally used as a reference parameter.

[0057] In the present embodiment, the actual observation value can be taken from the reference interface screenshot, which is generally used as a reference parameter.

[0058] Through the above steps S10-S50, the method fully utilizes the data processing kernel of the twin network architecture, improves the overall data processing efficiency, and solves the problem of low screenshot processing efficiency in the prior art. On this basis, the method further models and predicts the time sequence of the analysis and processing data through the anomaly detection module, and calculates the anomaly prediction value. This way can obtain more accurate analysis and prediction results.

[0059] Not based on this, relying on the edge-cloud collaborative computing framework built by the embodiment method, the screenshot collection and preliminary processing can be deployed on the edge node, and the complex AI inference and large-scale data analysis can be deployed on the cloud. The edge node collects and pre-processes data in real time, and the cloud is responsible for deep model calculation and data storage. The two work together. This architecture not only guarantees low-latency response, but also takes advantage of the powerful computing power of the cloud, achieving efficient resource utilization and system scalability.

[0060] For example, if the SCADA (Supervisory Control And Data Acquisition) system interface of a certain factory site is covered by an unauthorized pop-up window, it may cause misoperation. The application background is based on the "normal operation full screen" as the benchmark interface screenshot, and the screenshot data is extracted from the current interface. The above cloud AI analysis module combined with the anomaly detection module can successfully intercept 47 times of abnormal interface operation, and the average alarm response time is less than 3s, which greatly reduces the cost of manual inspection and significantly improves the automation level.

[0061] Preferably, before step S10, the method further comprises:

[0062] S100: The user configures the monitoring target and the scheduling strategy.

[0063] S101: Set up a screenshot module according to the monitoring target and the scheduling strategy.

[0064] In this embodiment, the user can pre-configure the monitoring target and the scheduling strategy to generate a program script based on the Cron expression. For example, the above program script can be:

[0065]

[0066]

[0067] It needs to be explained that the above program script (screenshot module) represents that the screenshot action is triggered automatically once a day at zero o'clock for the monitoring target, and the latest 7 screenshots are retained. The above program script also supports the enhanced syntax of Cron expression, such as @midnight+30min (execution 30 minutes after midnight), which can automatically avoid system peak period. The screenshot module can be set to more than two, one of which is used for real-time screenshot, and the other is used for reference interface screenshot.

[0068] Preferably, the method of the embodiment can also configure a "anti-shake" strategy in the screenshot module. When the current task fails due to network jitter, the exponential backoff retry is automatically performed within an interval of 5-15 minutes to ensure that the screenshot task is completed within the available window.

[0069] Compared with the prior art, the method of the embodiment relies on program script to automatically perform image capture on the monitoring target, solving the problem of low automation degree in the prior art.

[0070] Further, the acquisition module includes a virtualization interface, and the step S20 includes:

[0071] S21: calling the virtualization interface to acquire data of the target window of the edge node to obtain pixel data.

[0072] S22: compressing and encrypting the pixel data to obtain screenshot data.

[0073] In the embodiment, the virtualization interface can use any one of DirectX, OpenGL, RDP or screen capture API. The selection of the specific virtualization interface is determined according to the type of the edge node.

[0074] In other embodiments, the virtualization interface can also use the operating system provided by the edge node.

[0075] By compressing the pixel data through the steps S21-S22, the data storage and transmission efficiency can be optimized, and the security of the pixel data in the transmission process can be greatly improved by encrypting the compressed pixel data.

[0076] Further, before the step S30, the method of the embodiment further includes:

[0077] In the edge node, the reference interface screenshot is preprocessed.

[0078] In the embodiment, the image preprocessing includes resolution normalization, color space conversion, and sensitive area desensitization. The sensitive area desensitization includes keyword identification and ID card identification desensitization. Through image preprocessing on the reference interface screenshot, the performance of the cloud AI analysis module can be improved, and privacy can be protected and data format can be unified.

[0079] Further, in the process of the above step S30, in order to further improve the security of data transmission, the method further includes:

[0080] The screenshot data located at the edge node and the reference interface screenshot are transmitted to the cloud AI analysis module through a preset secure channel.

[0081] Preferably, the secure channel adopts TLS (Transport Layer Security). TLS is a secure communication protocol based on encryption technology, which is used to establish an end-to-end encrypted channel on the Internet to protect the confidentiality, integrity and identity authentication of data during transmission.

[0082] Specifically, the above step S30 includes:

[0083] S31: Extracting visual features of the reference interface screenshot.

[0084] In the embodiment, the visual features include a gray histogram and texture features.

[0085] S32: Inputting the visual features and the screenshot data into a pre-trained cloud AI analysis module to calculate analysis processing data.

[0086] It should be explained that the cloud AI analysis module belongs to a Siamese network, which simultaneously processes two inputs (visual features and screenshot data) using a twin neural network structure. The twin network is composed of two convolutional neural network branches with the same structure and shared parameters. Each branch includes a plurality of convolutional layers, pooling layers and fully connected layers, which are used to encode the input image into a fixed-dimensional feature vector. For example, each branch can adopt a 4-layer convolutional network to output a 128-dimensional feature embedding. The feature vectors output by the two branches are calculated by an L2 norm or a cosine similarity function to obtain a similarity score. The higher the score, the more similar the interfaces, and the lower the score, the more significant changes. The cloud AI analysis module outputs the similarity value of the current interface (screenshot data) and the reference interface (visual features) and the change area heat map and other information in real time, which are used for subsequent judgment and visualization.

[0087] Exemplarily, the network architecture of the cloud AI analysis module is as follows:

[0088] Convolutional network:

[0089] Conv1: 32 3x3 convolution kernels + BatchNorm + ReLU + 2x2 max pooling;

[0090] Conv2: 64 3x3 convolution kernels + BatchNorm + ReLU + 2x2 max pooling;

[0091] Conv3: 128 3x3 convolution kernels + BatchNorm + ReLU + 2x2 max pooling;

[0092] Conv4: 256 3x3 convolution kernels + BatchNorm + ReLU + 2x2 max pooling;

[0093] Embedding layer: flatten the pooled features, pass through two fully connected layers (FC1: 512→256, FC2: 256→128), and output a 128-dimensional vector.

[0094] As for the cloud AI analysis module loss function described above, the embodiment method adopts a contrastive loss (ContrastiveLoss) and combines a cosine similarity regularization term to strengthen the ability to distinguish small amplitude interface changes (such as text color, shadow) and structural changes (such as button position).

[0095] During pre-training, “hard negative samples” (high similarity but actually changed) and “hard positive samples” (low similarity but the interface has not really changed) can be automatically sampled according to the similarity score output by the last model, thereby accelerating the convergence of the model.

[0096] More specifically, the analysis processing data described above includes a highlighted difference region heat map and multi-scale fusion information. The embodiment method adopts Grad-CAM (Gradient-weighted Class Activation Mapping) technology to compare target channels in the two branches respectively to generate a highlighted difference region heat map. As for multi-scale fusion information, the embodiment method extracts feature vectors in the Conv2 and Conv4 layers described above respectively, and obtains them through adaptive weight fusion. This way not only retains global layout information but also captures local detail differences.

[0097] For example, a bank releases a new version of APP every week, and needs to automatically verify the layout consistency of the “transfer” and “bill” pages. Take the “last stable version image released after the last release” as the benchmark interface screenshot, and get the screenshot data from the “current release candidate version”. The cloud AI analysis module described above is used to verify 5000 pairs of interface samples, with an accuracy of 98.7% and a recall rate of 97.5%. Compared with existing technologies, the pixel difference is improved by about 12%.

[0098] In other embodiments, the AI analysis module can also use other convolutional network structures as feature encoders, such as ResNet or MobileNet, etc., to adapt to different hardware platforms. The similarity calculation can select cosine similarity or dot product weighting and other different metrics.

[0099] Further, the anomaly detection module includes an LSTM model and an ARIMA model, and the step S40 includes:

[0100] S41: calling the LSTM model, predicting the similarity value of the analysis processing data at the next time, and generating a nonlinear prediction result according to the plurality of similarity values.

[0101] S42: calling the ARIMA model, statistically predicting the similarity sequence of the analysis processing data, and generating a linear prediction result.

[0102] S43: weighting and fusing the nonlinear prediction result and the linear prediction result to calculate an anomaly prediction value.

[0103] It needs to be explained that the LSTM model is a special recurrent neural network (RNN) designed to solve the gradient vanishing and gradient explosion problems encountered by traditional RNN when processing long sequence data, and can more effectively capture long-term dependencies in data. The ARIMA model is a classic time series prediction method suitable for analyzing non-stationary time series data. The embodiment method forms "complementary advantages" by weighting and fusing the two models.

[0104] Through the steps S41-S43 and the mechanism of judging anomalies, the calculated anomaly prediction value is more accurate and the labor cost is lower.

[0105] In other embodiments, the anomaly detection module can replace or fuse other time series models (such as Transformer or GKSLSTM), or add an attention mechanism after LSTM to enhance the sequence feature capture capability. ARIMA can be replaced by Prophet or TBATS prediction model to adapt to multi-period or non-stationary sequences.

[0106] Further, if the number of anomaly prediction values is greater than or equal to a preset threshold, the embodiment method further includes:

[0107] adding the anomaly prediction value to the sample training set, and updating the model weight of the anomaly detection module based on an incremental learning mechanism or a transfer learning mechanism.

[0108] In this embodiment, in order to cope with the long-term evolution of interface content, an online incremental learning strategy is introduced, and the screenshot samples and model weights are updated by system regularly or event-driven. When the abnormal samples or new scene samples with reliable labels are detected, they are added to the training set, and the LSTM weights are updated by incremental training or transfer learning. Specifically, an incremental LSTM autoencoder set is maintained, each autoencoder is fine-tuned on new data, and the final result is obtained by weighted combination according to performance. This incremental learning mechanism can make the model continuously adapt to new data distribution, reduce the cost of full retraining, and ensure the long-term stability and efficiency of the model.

[0109] Preferably, the specific steps of implementing the incremental learning mechanism include:

[0110] S400: Predefine a sliding window sample cache.

[0111] For example, a "candidate sample pool" with a size of W (for example, W=1,000) is maintained in the cloud, and the latest screenshots and their similarity labels reported by the edge nodes are received in real time. Only the last W normal / abnormal labeled samples are saved, and when the candidate sample pool is full, it is updated according to the "oldest sample elimination" or "low information sample replacement" strategy.

[0112] S401: Execute a set of weak learners (LSTM autoencoders).

[0113] Specifically, the Kth autoencoder model is initialized, each autoencoder model is composed of a double-layer LSTM encoder and a decoder, and is used to reconstruct the similarity time series. When the ith model is online, all autoencoder models are pre-trained on the offline historical data, and the initial validation error of each autoencoder model is recorded.

[0114] S402: When the number of abnormal prediction values in the candidate sample pool reaches or exceeds the threshold, a fine-tuning period is started.

[0115] Specifically, for each autoencoder model, T-step incremental training is performed on the latest candidate samples, and only the weights of the outermost fully connected layer are updated. The calculation expression for updating the model weights of the abnormal detection module is as follows:

[0116]

[0117] In the formula, w i represents the weight coefficient of the ith model, represents the average reconstruction error of the ith model in the new sample training set, and δ represents a constant greater than 0, represents the summation calculation from j=1 to j=K, and K represents the total number of models.

[0118] Further, the abnormality discrimination formula of step S50 is:

[0119]

[0120] In the formula, s t represents the actual observation value at time t, which can be 1 if taken from the reference interface screenshot, represents the time series similarity prediction value after weighted fusion, i.e., the abnormality prediction value, with a value range of 0-1, θ alert represents the error threshold.

[0121] The calculation formula of the abnormality prediction value is:

[0122]

[0123] In the formula, K represents the total number of the autoencoder models, represents the abnormality prediction value of the i-th model.

[0124] Further, if the weight w i of a certain autoencoder model is at the lowest level for multiple fine-tuning periods, the model is eliminated, and a newly initialized autoencoder is retrained with all samples to replace it, so as to prevent the degradation of the integrated model.

[0125] Through the above technical solution description, the method of the embodiment has at least the following three advantages:

[0126] 1. The method of the embodiment introduces a twin neural network to compare the interface similarity between different time screenshots, realizing quantitative description of interface changes. This deep feature-based comparison method is more robust than traditional pixel difference or static template matching, can automatically ignore the dynamic content difference in the interface, and improves the recognition accuracy.

[0127] 2. The method of the embodiment effectively combines the LSTM network with the ARIMA statistical model, and fuses the prediction results of the two by residual weighting, improving the accuracy and reliability of interface change trend prediction. This method takes into account the advantages of deep learning and classical algorithms, and improves the detection performance of complex time series anomalies.

[0128] 3. In a frequently changing application environment, the method of the embodiment uses an incremental learning strategy to continuously optimize the model with new data. Through incremental integration or transfer learning mechanisms, the model continuously captures the dynamic characteristics of the system. This mechanism reduces the retraining overhead, enabling the system to run online for a long time and maintain high detection accuracy.

[0129] Embodiment Two

[0130] For example, Figure 2As shown, based on the embodiment one, the embodiment discloses a computer system, comprising a processor and a memory, the memory stores computer program instructions, when the computer program instructions are executed by the processor, the AI model-based periodic screenshot analysis method recorded in the embodiment one is realized.

Claims

1. An AI model-based periodic screenshot analysis method, characterized by, The method comprises the following steps: in response to the triggering of the preset screenshot module, a screenshot task is generated; according to the screenshot task, a preset acquisition module is called to perform image acquisition on a target window of the edge node to obtain screenshot data and a baseline interface screenshot; the screenshot data and the baseline interface screenshot are input into a pre-trained cloud AI analysis module to calculate analysis processing data; wherein the cloud AI analysis module adopts an AI model based on a twin neural network architecture; a preset anomaly detection module is called to model and predict the time sequence of the analysis processing data to calculate an anomaly prediction value; specifically, the anomaly detection module comprises an LSTM model and an ARIMA model; the LSTM model is called to predict the similarity value of the analysis processing data at the next time, and a nonlinear prediction result is generated according to a plurality of similarity values; the ARIMA model is called to statistically predict the similarity sequence of the analysis processing data to generate a linear prediction result; the nonlinear prediction result and the linear prediction result are weighted and fused to calculate the anomaly prediction value; it is judged whether the difference between the actual observation value and the anomaly prediction value is greater than an error threshold value; if yes, an alarm is triggered; the screenshot module is a program script configured in the edge node and based on a Cron expression; when the edge node is running, the screenshot module is automatically loaded to generate a screenshot task. 2.The AI model-based periodic screenshot analysis method of claim 1, wherein, if the number of anomaly prediction values is greater than or equal to a preset threshold value, the method further comprises: the anomaly prediction values are added to a sample training set, and the model weight of the anomaly detection module is updated based on an incremental learning mechanism or a transfer learning mechanism. 3.The AI model-based periodic screenshot analysis method of claim 2, wherein, the calculation expression for updating the model weight of the anomaly detection module is: wherein represents the weight coefficient of the th model, represents the average reconstruction error of the th model in the new sample training set, represents a constant greater than 0, represents the summation calculation from to represents the total number of models.​ 4.The AI model-based periodic screenshot analysis method of claim 1, wherein, the acquisition module comprises a virtualization interface; the preset acquisition module is called to perform image acquisition on the target window of the edge node to obtain the screenshot data, which comprises the following steps: the virtualization interface is called to perform data acquisition on the target window of the edge node to obtain pixel data; the pixel data is compressed and encrypted to obtain the screenshot data. 5.The AI model-based periodic screenshot analysis method of claim 1, wherein, the screenshot data and the baseline interface screenshot are input into a pre-trained cloud AI analysis module to calculate analysis processing data, which comprises the following steps: the visual features of the baseline interface screenshot are extracted; the visual features and the screenshot data are input into the pre-trained cloud AI analysis module to calculate the analysis processing data. 6.The AI model-based periodic screenshot analysis method of claim 1, wherein, in the step of inputting the screenshot data and the baseline interface screenshot into the pre-trained cloud AI analysis module, the method further comprises: the screenshot data and the baseline interface screenshot located in the edge node are transported to the cloud AI analysis module through a preset secure channel. 7.The AI model-based periodic screenshot analysis method of claim 1, wherein, before the step of inputting the screenshot data and the baseline interface screenshot into the pre-trained cloud AI analysis module, the method further comprises: in the edge node, the baseline interface screenshot is subjected to image preprocessing.

8. A computer system, characterized by the device comprises a processor and a memory, and the memory stores computer program instructions; when the computer program instructions are executed by the processor, the computer program instructions realize the method for periodic screenshot analysis based on an AI model according to any one of claims 1-7.

Images