Regular screen capture analysis method based on AI model and computer system

Abstract

The invention relates to the field of artificial intelligence analysis, in particular to a regular screenshot analysis method based on an AI model and a computer system.The method comprises the steps that a screenshot task is generated in response to triggering of a preset screenshot module, and then a preset collection module is called according to the screenshot task to conduct image acquisition on a target window of an edge node; the method comprises the steps that firstly, screen capture data and a reference interface screenshot are obtained, then the screen capture data and the reference interface screenshot are input into a pre-trained cloud AI analysis module, and analysis processing data are obtained through calculation; wherein the cloud AI analysis module adopts an AI model based on a twin neural network architecture, then calls a preset anomaly detection module, performs time sequence modeling and prediction on analysis processing data, calculates an anomaly prediction value, finally judges whether a difference value between an actual observation value and the anomaly prediction value is greater than an error threshold value or not, and triggers an alarm if the difference value between the actual observation value and the anomaly prediction value is greater than the error threshold value. Compared with the prior art, the method provided by the invention solves the problems of low screenshot processing efficiency and inaccurate analysis prediction result.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence analysis. More specifically, the present invention relates to a method and computer system for periodic screenshot analysis based on an AI model. Background Art

[0002] With the increasing diversity of mobile and desktop software applications and the accelerating pace of product iteration, the interactive interfaces and data dynamic messages of these software applications are changing increasingly frequently. To achieve real-time monitoring of these software interactive interfaces and dynamic messages, traditional technologies rely on logs, event reports, or manual screenshots to upload information and conduct manual analysis and comparison. This method is time-consuming, has a high error rate, and is costly. In particular, screenshot analysis often focuses only on single events or manually triggered actions, making it difficult to meet the continuous monitoring needs of large-scale, distributed systems.

[0003] To address the technical issues related to screenshot analysis, Chinese application publication number CN119516564A discloses an intelligent task management method based on OCR and AI technologies. This method primarily analyzes image data from screenshots of task messages captured by users using shortcut keys. This process relies on OCR technology for analysis, which has the following drawbacks:

[0004] First, the degree of automation is not high, and triggering the screenshot action still requires manual assistance.

[0005] Secondly, OCR (Optical Character Recognition) is a technology that converts text in images into editable and searchable text. However, this technology is limited by its underlying algorithm core. When processing large numbers of screenshots in real time, it suffers from low processing efficiency and inaccurate prediction results.

[0006] Therefore, the existing technology mainly has the problems of low screenshot processing efficiency and inaccurate analysis and prediction results. Summary of the Invention

[0007] In order to solve the above-mentioned technical problems of low screenshot processing efficiency and inaccurate analysis and prediction results, the present invention discloses a regular screenshot analysis method and computer system based on an AI model.

[0008] In a first aspect, the present invention discloses a method for periodic screenshot analysis based on an AI model, comprising:

[0009] In response to the triggering of the preset screenshot module, a screenshot task is generated;

[0010] According to the screenshot task, the preset acquisition module is called to acquire the image of the target window of the edge node to obtain the screenshot data and the baseline interface screenshot;

[0011] Input the screenshot data and the baseline interface screenshot into the pre-trained cloud-based AI analysis module to calculate and process the analysis data. The cloud-based AI analysis module uses an AI model based on a twin neural network architecture.

[0012] Call the preset anomaly detection module to perform time series modeling and prediction on the analyzed and processed data, and calculate the anomaly prediction value;

[0013] Determine whether the difference between the actual observation value and the abnormal prediction value is greater than the error threshold. If so, trigger an alarm.

[0014] Beneficial effects: The method of the present invention first generates a screenshot task of the edge node through the screenshot module, and then calls the acquisition module to obtain the image of the target window of the edge node to obtain the screenshot data and the baseline interface screenshot, thereby realizing the automatic interception of the screenshot action, solving the problem of low automation of the existing technology; the screenshot data and the baseline interface screenshot are input into the AI ​​analysis module in the cloud, the analysis and processing data are calculated, and the anomaly detection module is called to perform time series modeling and prediction on the analysis and processing data, and the anomaly prediction value is calculated. This method makes full use of the data processing core of the twin network architecture, improves the overall data processing efficiency, and solves the problem of low efficiency of screenshot processing in the existing technology. The analysis and processing data are modeled and predicted in time series by the anomaly detection module, and the anomaly prediction value is calculated. This method can obtain a more accurate anomaly prediction value. Finally, the actual observation value is subtracted from the accurate anomaly prediction value to obtain a more accurate analytical prediction result, and the threshold judgment is relied on to trigger the alarm, which further improves the automation and accuracy of the method of the present invention.

[0015] Preferably, the anomaly detection module includes an LSTM (Long Short-Term Memory Network) model and an ARIMA (Autoregressive Integrated Moving Average) model; time series modeling and prediction are performed on the analyzed and processed data, and an anomaly prediction value is calculated, specifically:

[0016] Call the LSTM model to predict the similarity value of the analyzed data at the next moment, and generate nonlinear prediction results based on multiple similarity values;

[0017] Call the ARIMA model to perform statistical prediction on the similarity sequence of the analyzed and processed data and generate linear prediction results;

[0018] The nonlinear prediction results and the linear prediction results are weighted and fused to calculate the abnormal prediction value.

[0019] Beneficial effect: The anomaly detection module integrates the LSTM model and the ARIMA model. The LSTM model is suitable for processing data with complex nonlinear relationships, while the ARIMA model is suitable for processing data with linear relationships. The two models have their own advantages and disadvantages. The method of the present invention realizes the organic combination of the two models through weighted fusion, so that the advantages of the LSTM model and the ARIMA model are maximized, thereby making the calculated anomaly prediction value more accurate.

[0020] Preferably, if the number of abnormal prediction values ​​is greater than or equal to a preset threshold, the method of the present invention further comprises:

[0021] The anomaly prediction value is added to the sample training set, and the model weight of the anomaly detection module is updated based on the incremental learning mechanism or the transfer learning mechanism.

[0022] Beneficial effect: The method of the present invention adopts incremental training or transfer learning to update the model weights, which reduces the cost of full retraining and ensures the long-term stability and efficiency of the model.

[0023] Preferably, the calculation expression for updating the model weight of the anomaly detection module is:

[0024]

[0025] Where w i represents the weight coefficient of the i-th model, represents the average reconstruction error of the i-th model in the new sample training set, δ represents a constant greater than 0, It indicates that the summation is performed from j=1 to j=K, where K represents the total number of models.

[0026] Preferably, the acquisition module includes a virtualization interface, calling a preset acquisition module to acquire an image of a target window of an edge node to obtain screenshot data, including:

[0027] Call the virtualization interface to obtain data from the target window of the edge node to obtain pixel data;

[0028] The pixel data is compressed and encrypted to obtain the screenshot data.

[0029] Beneficial effect: Compressing and encrypting pixel data can optimize data storage and transmission efficiency while enhancing data transmission security.

[0030] Preferably, the screenshot data and the baseline interface screenshot are input into a pre-trained cloud AI analysis module to calculate and obtain analysis and processing data, including:

[0031] Extract visual features of baseline interface screenshots;

[0032] The visual features and screenshot data are input into the pre-trained cloud-based AI analysis module to calculate and obtain the analysis and processing data.

[0033] Preferably, before inputting the screenshot data and the baseline interface screenshot into the pre-trained cloud AI analysis module, the method of the present invention further includes:

[0034] The screenshot data and baseline interface screenshots at the edge node are transmitted to the cloud AI analysis module through a preset secure channel.

[0035] Preferably, before inputting the screenshot data and the baseline interface screenshot into the pre-trained cloud AI analysis module, the method further includes:

[0036] In the edge node, image preprocessing is performed on the baseline interface screenshot.

[0037] Preferably, the screenshot module adopts a program script based on Cron expression.

[0038] In a second aspect, the present invention discloses a computer system comprising a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, the periodic screenshot analysis method based on the AI ​​model recorded in the first aspect is implemented.

[0039] The beneficial effects of the present invention are:

[0040] (1) Compared with the prior art, the method of the present invention fully utilizes the data processing core of the twin network architecture, improves the overall data processing efficiency, and solves the problem of low efficiency of screenshot processing in the prior art. On this basis, the method of the present invention also uses the anomaly detection module to model and predict the time series of the analyzed and processed data, and calculates the anomaly prediction value. This method can obtain more accurate analytical prediction results.

[0041] (2) Compared with the existing technology, the method of the present invention realizes the organic combination of the two models through weighted fusion, so that the advantages of the LSTM model and the ARIMA model are maximized, thereby making the calculated abnormal prediction value more accurate.

[0042] (3) Compared with the existing technology, the method of the present invention adopts incremental training or transfer learning to update the model weights, which reduces the cost of full retraining and ensures the long-term stability and efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flowchart of the regular screenshot analysis method based on the AI ​​model in Example 1 of the present invention;

[0044] Figure 2 It is a structural diagram of the computer system in the second embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0046] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] Example 1

[0048] like Figure 1 As shown, this embodiment discloses a periodic screenshot analysis method based on an AI model, including:

[0049] S10: In response to the triggering of the preset screenshot module, a screenshot task is generated.

[0050] In this embodiment, the screenshot module is a program script configured on the edge node and based on the Cron expression. When the edge node is running, the screenshot module is automatically loaded and a screenshot task is generated.

[0051] S20: According to the screenshot task, a preset acquisition module is called to acquire an image of the target window of the edge node to obtain screenshot data and a baseline interface screenshot.

[0052] In this embodiment, screenshot data refers to pixel data from edge node screens that has undergone preliminary compression and encryption. Edge nodes can be smartphones or other industrial control terminals. Screenshot data is real-time screenshot data, while the baseline interface screenshot is a comparison baseline image.

[0053] S30: Input the screenshot data and the baseline interface screenshot into a pre-trained cloud AI analysis module to calculate and obtain analysis and processing data.

[0054] The cloud AI analysis module uses an AI model based on a twin neural network architecture and is deployed on a cloud analysis server (cloud server).

[0055] S40: calling a preset anomaly detection module, performing time series modeling and prediction on the analyzed and processed data, and calculating an anomaly prediction value.

[0056] S50: Determine whether the difference between the actual observation value and the abnormal prediction value is greater than the error threshold. If so, trigger an alarm.

[0057] In this embodiment, the actual observation value can be taken from the baseline interface screenshot and is generally used as the baseline parameter.

[0058] Through the above steps S10-S50, the method of this embodiment fully utilizes the data processing core of the twin network architecture, improves the overall data processing efficiency, and solves the problem of low efficiency of screenshot processing in the existing technology. On this basis, the method of this embodiment also uses the anomaly detection module to model and predict the time series of the analyzed and processed data, and calculates the anomaly prediction value. This method can obtain more accurate analysis and prediction results.

[0059] Instead, the edge-cloud collaborative computing framework built by the method of this embodiment can deploy screenshot collection and preliminary processing on edge nodes, and deploy complex AI reasoning and large-scale data analysis on the cloud. The edge nodes collect and pre-process data in real time, and the cloud is responsible for deep model calculation and data storage, and the two work together. This architecture not only ensures low-latency response, but also utilizes the powerful computing power of the cloud, achieving efficient resource utilization and system scalability.

[0060] For example, if the SCADA (Supervisory Control And Data Acquisition) system interface of a factory site is covered by an unauthorized pop-up window, it may lead to erroneous operation as the application background. The "normal operation full screen" is used as the baseline interface screenshot, and the screenshot data is extracted from the current interface. The above-mentioned cloud AI analysis module combined with the anomaly detection module can successfully intercept 47 abnormal interface operations, and the average alarm response time is less than 3s, which greatly reduces the cost of manual inspection and significantly improves the level of automation.

[0061] Preferably, before step S10, the method of this embodiment further includes:

[0062] S100: The user configures monitoring targets and scheduling strategies.

[0063] S101: Set a screenshot module according to the monitoring target and scheduling strategy.

[0064] In this embodiment, the user can pre-configure the monitoring target and scheduling strategy to generate a program script based on the Cron expression. For example, the above program script can be:

[0065]

[0066]

[0067] It's important to note that the above script (screenshot module) automatically triggers a screenshot of the monitored target at midnight each day, retaining the seven most recent screenshots. The script also supports enhanced Cron expression syntax, such as @midnight+30min (execute 30 minutes after midnight), to automatically avoid system peak hours. Two or more screenshot modules can be configured, one for real-time screenshots and another for baseline interface screenshots.

[0068] Preferably, the method of this embodiment can also configure an "anti-jitter" strategy in the screenshot module. When the current task fails due to network jitter, an exponential backoff retry is automatically performed within an interval of 5-15 minutes to ensure that the screenshot task is completed within the available window.

[0069] Compared with the prior art, the method of this embodiment relies on program scripts to automatically capture images of the monitored target, thus solving the problem of low automation level in the prior art.

[0070] Furthermore, the acquisition module includes a virtualization interface, and the above step S20 includes:

[0071] S21: calling the virtualization interface to obtain data from the target window of the edge node to obtain pixel data.

[0072] S22: compressing and encrypting the pixel data to obtain screenshot data.

[0073] In this embodiment, the virtualization interface can be any one of DirectX, OpenGL, RDP or screen capture API. The specific virtualization interface is determined by the type of edge node.

[0074] In other embodiments, the virtualization interface may also use the operating system provided by the edge node.

[0075] By compressing the pixel data through the above steps S21-S22, the data storage and transmission efficiency can be optimized, and encrypting the compressed pixels can greatly improve the security of the pixel data during transmission.

[0076] Furthermore, before step S30, the method of this embodiment further includes:

[0077] In the edge node, image preprocessing is performed on the baseline interface screenshot.

[0078] In this embodiment, image preprocessing includes resolution normalization, color space conversion, and sensitive area desensitization. Sensitive area desensitization includes keyword recognition and ID card recognition desensitization. Image preprocessing of baseline interface screenshots improves the performance of the cloud-based AI analysis module, protects privacy, and unifies data formats.

[0079] Furthermore, in the process of step S30 above, in order to further improve the security of data transmission, the method of this embodiment further includes:

[0080] The screenshot data and baseline interface screenshots at the edge node are transmitted to the cloud AI analysis module through a preset secure channel.

[0081] Preferably, the above secure channel uses TLS (Transport Layer Security). TLS is a secure communication protocol based on encryption technology, used to establish an end-to-end encrypted channel on the Internet to protect the confidentiality, integrity and identity authentication of data during transmission.

[0082] Specifically, the above step S30 includes:

[0083] S31: Extract visual features of the baseline interface screenshots.

[0084] In this embodiment, the visual features include grayscale histogram and texture features.

[0085] S32: Input the visual features and screenshot data into the pre-trained cloud-based AI analysis module to calculate and obtain analysis and processing data.

[0086] It should be explained that the cloud-based AI analysis module belongs to the Siamese network, which uses a twin neural network structure to simultaneously process two inputs (visual features and screenshot data). The twin network consists of two convolutional neural network branches with the same structure and shared parameters. Each branch contains several convolutional layers, pooling layers, and fully connected layers, which are used to encode the input image into a feature vector of fixed dimension. For example, each branch can use a 4-layer convolutional network to output a 128-dimensional feature embedding. The feature vectors output by the two branches are used to calculate the similarity score through the L2 norm or cosine similarity function. The higher the score, the more similar the interface, and the low score indicates a significant change. The cloud-based AI analysis module outputs the similarity value between the current interface (screenshot data) and the baseline interface (visual features) in real time, as well as information such as the heat map of the changed area for subsequent judgment and visualization.

[0087] For example, the network architecture of the cloud AI analysis module is as follows:

[0088] Convolutional Networks:

[0089] Conv1: 32 3×3 convolution kernels + BatchNorm + ReLU + 2×2 maximum pooling;

[0090] Conv2: 64 3×3 convolution kernels + BatchNorm + ReLU + 2×2 maximum pooling;

[0091] Conv3: 128 3×3 convolution kernels + BatchNorm + ReLU + 2×2 max pooling;

[0092] Conv4: 256 3×3 convolution kernels + BatchNorm + ReLU + 2×2 max pooling;

[0093] Embedding layer: Flatten the pooled features and output a 128-dimensional vector through two fully connected layers (FC1: 512→256, FC2: 256→128).

[0094] As for the loss function of the cloud-based AI analysis module, the method of this embodiment adopts contrastive loss (ContrastiveLoss) and combines it with the cosine similarity regularization term to enhance the ability to distinguish between small interface changes (such as text color and shadow) and structural changes (such as button position).

[0095] During the pre-training process, "difficult negative samples" (high similarity but actual changes) and "difficult positive samples" (low similarity but the interface has not actually changed) can be automatically sampled based on the similarity score of the previous model output, thereby accelerating the convergence of the model.

[0096] More specifically, the analyzed and processed data includes a heat map of highlighted difference areas and multi-scale fusion information. The method in this embodiment uses Grad-CAM (Gradient-weighted Class Activation Mapping) technology to compare target channels in two branches to generate a heat map of highlighted difference areas. Regarding multi-scale fusion information, the method in this embodiment extracts feature vectors from the Conv2 and Conv4 layers respectively, and obtains them through adaptive weighted fusion. This approach preserves global layout information while capturing local detail differences.

[0097] For example, a bank routinely releases a new version of its APP every week and needs to automatically verify the layout consistency of the "Transfer" and "Bill" pages. This is used as an application example: the "stable version image after the last release" is used as the baseline interface screenshot, and the screenshot data is obtained from the "current release candidate version". The above-mentioned cloud-based AI analysis module is used to verify on 5,000 pairs of interface samples. The accuracy rate can reach 98.7%, and the recall rate can reach 97.5%. Compared with the existing technology, the pixel difference is improved by about 12%.

[0098] In other embodiments, the AI ​​analysis module may also use other convolutional network structures as feature encoders, such as ResNet or MobileNet, to adapt to different hardware platforms. Similarity calculations may use different metrics such as cosine similarity or point multiplication weighting.

[0099] Furthermore, the anomaly detection module includes an LSTM model and an ARIMA model, and the above step S40 includes:

[0100] S41: Calling the LSTM model to predict the similarity value of the analyzed data at the next moment, and generating a nonlinear prediction result based on multiple similarity values.

[0101] S42: Calling the ARIMA model to perform statistical prediction on the similarity sequence of the analyzed and processed data to generate a linear prediction result.

[0102] S43: Perform weighted fusion on the nonlinear prediction result and the linear prediction result to calculate an abnormal prediction value.

[0103] It should be explained that the LSTM model is a special recurrent neural network (RNN) designed specifically to address the vanishing and exploding gradient problems encountered by traditional RNNs when processing long sequences of data. It can more effectively capture long-term dependencies in the data. The ARIMA model is a classic time series forecasting method suitable for analyzing non-stationary time series data. This embodiment's method achieves a "complementary" advantage by combining the two models through a weighted fusion.

[0104] Through the above-mentioned steps S41 to S43 and the mechanism for judging abnormalities, the calculated abnormality prediction value is made more accurate and the labor cost is lower.

[0105] In other embodiments, the anomaly detection module can replace or integrate other time series models (such as Transformer or GKSLSTM), or add an attention mechanism after LSTM to enhance the ability to capture sequence features. ARIMA can also be replaced with a forecasting model such as Prophet or TBATS to accommodate multi-period or non-stationary sequences.

[0106] Furthermore, if the number of abnormal prediction values ​​is greater than or equal to a preset threshold, the method of this embodiment further includes:

[0107] The anomaly prediction value is added to the sample training set, and the model weight of the anomaly detection module is updated based on the incremental learning mechanism or the transfer learning mechanism.

[0108] In this embodiment, in order to cope with the long-term evolution of interface content, the method of this embodiment introduces an online incremental learning strategy, which updates the screenshot samples and model weights through regular system discrimination or event-driven methods. When a conclusively marked abnormal sample or new scene sample is detected, it is added to the training set, and the LSTM weights are updated using incremental training or transfer learning. Specifically, by maintaining an incremental set of LSTM autoencoders, each autoencoder is fine-tuned on the new data, and weighted combination is performed according to performance, ultimately achieving the above effect. This incremental learning mechanism enables the model to continuously adapt to the distribution of new data, reduces the cost of full retraining, and can ensure the long-term stability and efficiency of the model.

[0109] Preferably, the specific steps of implementing the incremental learning mechanism include:

[0110] S400: Predefine a sliding window sample buffer.

[0111] For example, a "candidate sample pool" of size W (e.g., W = 1,000) is maintained in the cloud, which receives the latest screenshots and their similarity labels reported by edge nodes in real time. Only the most recent W normal / abnormal labeled samples are saved. When the candidate sample pool is full, it is updated according to the "eliminate the oldest sample" or "replace the low-information sample" strategy.

[0112] S401: Execute the weak learner (LSTM autoencoder) set.

[0113] Specifically, initialize the Kth autoencoder model. Each autoencoder model consists of a two-layer LSTM encoder and a decoder to reconstruct the similarity time series. When the i-th model is online, all autoencoder models are fully pre-trained on offline historical data and their initial verification errors are recorded.

[0114] S402: When the number of abnormal prediction values ​​in the candidate sample pool reaches or exceeds a threshold, a fine-tuning cycle is started.

[0115] Specifically, for each autoencoder model, T-step incremental training is performed on the most recent candidate samples, and only the weights of the outermost fully connected layer are updated. The calculation expression for the model weights of the above-mentioned updated anomaly detection module is:

[0116]

[0117] Where w i represents the weight coefficient of the i-th model, represents the average reconstruction error of the i-th model in the new sample training set, δ represents a constant greater than 0, It indicates that the summation is performed from j=1 to j=K, where K represents the total number of models.

[0118] Furthermore, the abnormality discriminant formula of the above step S50 is:

[0119]

[0120] Where s t Indicates the actual observation value at time t. If it is taken from the baseline interface screenshot, the value can be 1. It represents the time series similarity prediction value after weighted fusion, that is, the abnormal prediction value, with a value range of 0-1, θ alert Indicates the error threshold.

[0121] The calculation formula for the abnormal prediction value is:

[0122]

[0123] Where K represents the total number of autoencoder models, Represents the abnormal prediction value of the i-th model.

[0124] Furthermore, if the weights w of a certain autoencoder model are fine-tuned over multiple cycles i If the performance remains at the lowest level, the model will be eliminated and the newly initialized autoencoder will be fully retrained with the latest samples before being replaced to prevent the integrated model from degrading.

[0125] Through the above technical solution, the method of this embodiment has at least the following three advantages:

[0126] 1. This embodiment introduces a twin neural network to compare the interface similarities between screenshots taken at different times, achieving a quantitative description of interface changes. This deep feature-based comparison method is more robust than traditional pixel difference or static template matching, and can automatically ignore dynamic content differences in the interface, improving recognition accuracy.

[0127] 2. This example method effectively combines an LSTM network with an ARIMA statistical model, fusing their prediction results through residual weighting to improve the accuracy and reliability of interface trend prediction. This method leverages the strengths of both deep learning and classical algorithms, improving the detection performance of complex time series anomalies.

[0128] 3. In a frequently changing application environment, the method in this embodiment employs an incremental learning strategy to continuously optimize the model as new data is added. Through mechanisms such as incremental integration or transfer learning, the model continuously captures the dynamic characteristics of the system. This mechanism reduces retraining overhead, enabling the system to operate online for extended periods while maintaining high detection accuracy.

[0129] Example 2

[0130] like Figure 2As shown, based on the first embodiment, this embodiment discloses a computer system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the periodic screenshot analysis method based on the AI ​​model recorded in the first embodiment is implemented.

Claims

1. A regular screenshot analysis method based on AI model, characterized in that: include: In response to the triggering of the preset screenshot module, a screenshot task is generated; According to the screenshot task, a preset acquisition module is called to acquire an image of the target window of the edge node to obtain screenshot data and a screenshot of the reference interface; Inputting the screenshot data and the baseline interface screenshot into a pre-trained cloud-based AI analysis module to calculate and obtain analysis and processing data; wherein the cloud-based AI analysis module adopts an AI model based on a twin neural network architecture; Calling a preset anomaly detection module to perform time series modeling and prediction on the analyzed and processed data, and calculating an anomaly prediction value; Determine whether the difference between the actual observation value and the abnormal prediction value is greater than the error threshold. If so, trigger an alarm.

2. The AI ​​model-based regular screenshot analysis method according to claim 1, characterized in that: The anomaly detection module includes an LSTM model and an ARIMA model; the analysis and processing data are modeled and predicted in time series, and an anomaly prediction value is calculated, specifically: Calling the LSTM model to predict the similarity value of the analyzed and processed data at the next moment, and generating a nonlinear prediction result based on multiple similarity values; Calling the ARIMA model to perform statistical prediction on the similarity sequence of the analyzed and processed data to generate a linear prediction result; The nonlinear prediction result and the linear prediction result are weightedly fused to calculate the abnormal prediction value.

3. The AI ​​model-based regular screenshot analysis method according to claim 1, characterized in that: If the number of abnormal prediction values ​​is greater than or equal to a preset threshold, the method further includes: The anomaly prediction value is added to the sample training set, and the model weight of the anomaly detection module is updated based on an incremental learning mechanism or a transfer learning mechanism.

4. The AI ​​model-based regular screenshot analysis method according to claim 3, characterized in that: The calculation expression for updating the model weight of the anomaly detection module is: Where w i represents the weight coefficient of the i-th model, represents the average reconstruction error of the i-th model in the new sample training set, δ represents a constant greater than 0, It indicates that the summation is performed from j=1 to j=K, where K represents the total number of models.

5. The AI ​​model-based regular screenshot analysis method according to claim 1, characterized in that: The acquisition module includes a virtualization interface, calling a preset acquisition module to acquire an image of a target window of an edge node to obtain screenshot data, including: Calling the virtualization interface to acquire data from the target window of the edge node to obtain pixel data; The pixel data is compressed and encrypted to obtain the screenshot data.

6. The AI ​​model-based periodic screenshot analysis method according to claim 1, characterized in that: Input the screenshot data and the baseline interface screenshot into a pre-trained cloud AI analysis module to calculate and obtain analysis and processing data, including: Extracting visual features of the baseline interface screenshot; The visual features and the screenshot data are input into a pre-trained cloud-based AI analysis module to calculate and obtain analysis and processing data.

7. The AI ​​model-based periodic screenshot analysis method according to claim 1, characterized in that: After inputting the screenshot data and the baseline interface screenshot into a pre-trained cloud-based AI analysis module, the method further includes: The screenshot data and the baseline interface screenshot located at the edge node are transmitted to the cloud AI analysis module through a preset secure channel.

8. The AI ​​model-based periodic screenshot analysis method according to claim 1, characterized in that: Before inputting the screenshot data and the baseline interface screenshot into a pre-trained cloud-based AI analysis module, the method further includes: In the edge node, image preprocessing is performed on the screenshot of the reference interface.

9. The AI ​​model-based periodic screenshot analysis method according to claim 1, characterized in that: The screenshot module uses a program script based on Cron expression.

10. A computer system, characterized in that: It includes a processor and a memory, the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the periodic screenshot analysis method based on the AI ​​model described in any one of claims 1 to 9 is implemented.

Images