Intelligent anomaly detection system based on multi-model iterative learning and automation method thereof

Abstract

The invention provides an intelligent anomaly detection system based on multi-model iterative learning and an automation method thereof, and the system comprises a self-adaptive data processing module which automatically loads and preprocesses input data; multiple preset deep learning models selected through configuration files are integrated in the multi-model self-learning detection engine, and all the preset deep learning models are driven by adopting uniform interface specifications and configuration files; the dynamic threshold self-adaptive decision-making module is used for reconstructing error statistics on the model based on training data and automatically calculating and generating a decision-making threshold for abnormity judgment; and the automatic platform and service module is used for packaging the training and detection functions of the multi-model self-learning detection engine into services called through a RESTful API (Application Program Interface). According to the invention, various preset deep learning models can be supported and switched, automatic model training and evaluation and model version management based on timestamps and version numbers can be realized, a self-adaptive decision threshold is generated, and an interface calling service is provided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, machine learning and automation system, in particular to an intelligent anomaly detection system based on multi-model iterative learning, and further relates to an automation method applied to the intelligent anomaly detection system based on multi-model iterative learning. BACKGROUND

[0002] With the rapid development of Industry 4.0 and Internet of Things technology, the amount of data generated by various industries is growing exponentially, and the demand for anomaly detection in artificial intelligence, machine learning and automation systems is increasingly urgent. Traditional anomaly detection methods mainly include: statistical methods based on data distribution assumptions, such as the 3σ principle, which are not suitable for complex and nonlinear data distribution; machine learning methods, such as Isolation Forest, which require manual feature engineering and have limited generalization ability; single deep learning methods, such as simple autoencoders, which can automatically learn features but are prone to overfitting to specific patterns and cannot handle diverse anomaly patterns.

[0003] That is, the prior art has the following deficiencies: first, lack of systematic model switching and management, the prior art usually uses only one model, when the model effect is not good or the data pattern changes, manual intervention is needed to replace / switch the model and retrain, the implementation process is tedious and not continuous; second, low degree of automation, requiring a large amount of manual configuration of parameters, selection of models and adjustment of thresholds, unable to achieve truly "configure and run" automated detection; third, weak iteration and evolution ability, traditional systems lack automated model iterative training, version control and performance tracking mechanisms, making it difficult to achieve continuous learning and performance improvement of the model; fourth, low system integration, detection algorithms are disconnected from application systems, lack of a unified and API interface managed automation platform, unable to achieve end-to-end automation services. SUMMARY

[0004] The technical problem to be solved by the present application is to provide an intelligent anomaly detection system based on multi-model iterative learning, which can support and switch multiple preset deep learning models through a configuration method, can realize automated model training, evaluation and model version management based on timestamps and version numbers, and can automatically calculate and generate decision thresholds for anomaly judgment based on training data, providing services that can be called through RESTful API interfaces. On this basis, an automation method applied to the intelligent anomaly detection system based on multi-model iterative learning is further provided.

[0005] To this end, the present application provides an intelligent anomaly detection system based on multi-model iterative learning, comprising: an adaptive data processing module for automatically loading and preprocessing input data according to a configuration file; The multi-model self-learning detection engine integrates multiple preset deep learning models selected by a configuration file inside the multi-model self-learning detection engine, and all the preset deep learning models adopt a unified interface specification and a configuration file driver; the multi-model self-learning detection engine has the capabilities of automatic model training, evaluation, and model version management based on a timestamp and a version number; The dynamic threshold self-adaptive decision module is used for reconstructing error statistics on the model based on training data after the model training is completed, and automatically calculating and generating a decision threshold for anomaly judgment; The automatic platform and service module is used for encapsulating the training and detection functions of the multi-model self-learning detection engine as services called through a RESTful API interface.

[0006] The further improvement of the application is that the implementation process of the multi-model self-learning detection engine includes the following steps: Step A1, designing a unified interface specification and an internal model library, defining a unified interface specification, pre-agreeing standard initialization, forward propagation and parameter loading methods through the interface specification, and setting multiple preset deep learning models according to different detection scenarios; Step A2, realizing configuration-based model switching and instantiation, first designing a unified detection system class, which is used to read the deep learning model type model_type field in the configuration file during initialization; then realizing a model factory inside the multi-model self-learning detection engine, which dynamically instantiates the corresponding model class according to the deep learning model type model_type field in the configuration file, and passes the hyperparameters related to the deep learning model in the configuration file as parameters into the model instantiation process; the configuration file contains the deep learning model type model_type field and the hyperparameters of each preset deep learning model; Step A3, developing an iterative learning and robustness guarantee mechanism, periodically executing the model training process through an automatic script, using a validation dataset to monitor the model performance in real time during the training process, and triggering an early stopping mechanism when it is detected that the loss of the deep learning model on the validation dataset no longer decreases; after the training is completed, the model type, timestamp and version number of the deep learning model are recorded and saved as a unique folder name; when executing a detection task, the latest version file of the corresponding model type is automatically found and loaded, and when the performance of the latest version file of the deep learning model is found to be degraded, it is automatically rolled back to the previous stable version.

[0007] The further improvement of the application is that in step A1, the implementation process of defining a unified interface specification includes the following sub-steps: Step A101, determine the interface method including initialization method, forward propagation method and parameter loading method, the initialization method is used for initializing the hyperparameter of deep learning model;The forward propagation method is used for receiving input data and returning the output of deep learning model;The parameter loading method is used for receiving path string, for loading pre-trained model parameters; Step A102, create an abstract base class, define the interface method in the abstract base class, and mark the interface method as an abstract method; Step A103, inherit the abstract base class in each specific model class, and implement the interface method.

[0008] Further improvement of the application is that the step A2 includes the following substeps: Step A201, define a detection system class responsible for initialization and running management; Step A202, in the initialization of the detection system class, read the configuration file, parse the deep learning model type model_type field and its hyperparameter model_params field, and store the deep learning model type model_type field and its hyperparameter model_params field as the attribute of the detection system class; Step A203, call the model factory according to the deep learning model type model_type field to dynamically instantiate the corresponding deep learning model, and pass the hyperparameter model_params field to the model factory, and pass it to the constructor of the deep learning model through the model factory, to complete the instantiation of the model class.

[0009] Further improvement of the application is that in the dynamic threshold adaptive decision module, first calculate the mean μ and standard deviation σ of the training data reconstruction error E, then combine the adjustment factor factor representing the business risk tolerance, and calculate and generate the decision threshold Threshold through the formula Threshold=μ+(factor×σ).

[0010] Further improvement of the application is that for each data point vector X in the training set, a reconstruction vector X' is generated through the deep learning model, and the reconstruction error E of the training data is calculated through the formula E=(1 / n)×Σ(X_i-X'_i)², wherein n is the dimension of the data point vector, X_i represents the i th data point vector, X'_i represents the i th reconstruction vector, and i is a natural number.

[0011] Further improvement of the application is that the implementation process of the adaptive data processing module includes the following steps: Step B1, an intelligent data loader is implemented to parse a configuration file provided by a user, extract metadata of a data source in the configuration file, including a file path, a separator, an index of a time column and a date format, and when a user does not completely specify a feature column of the metadata, automatically scan a data set through the intelligent data loader to select all numerical columns except the time column as input features; Step B2, automatic data preprocessing is performed, after data loading, a data cleaning process is automatically performed, for each feature column, first calculate the first quartile Q1 and the third quartile Q3, and obtain the interquartile range IQR through the formula IQR=Q3-Q1; then, identify all data points below Q1-1.5*IQR or above Q3+1.5*IQR as outliers, and smooth the outliers by using a linear interpolation method; finally, the data is normalized by a standardization method, and input into subsequent model training.

[0012] Further improvement of the application is that the implementation process of the service encapsulated in the automation platform and the service module and called through the RESTful API interface includes the following steps: Step C1, define an interface layer, build an interface layer using a Web framework, define HTTP endpoints, use a training task POST / train to receive a training request, use a detection task POST / detect to receive a detection request, process HTTP requests and implement data verification logic; Step C2, implement a service layer, after receiving an API interface request, create and start a background training task through the service layer, call an API interface of the multi-model self-learning detection engine, pass request parameters to the multi-model self-learning detection engine, execute a machine learning task, and process the start, monitoring and result return of the learning task; Step C3, instantiate and call the API interface of the multi-model self-learning detection engine through the service layer; Step C4, implement platform capabilities, the interface layer provides services for user interfaces or external systems through HTTP endpoints, including API interfaces for training requests and detection requests; and through a WebSocket full-duplex communication service, real-time push alarm information when an anomaly is detected.

[0013] The application also provides an automated method for intelligent anomaly detection based on multi-model iterative learning, applied to the intelligent anomaly detection system based on multi-model iterative learning as described above, and including the following steps: Step S1, for configuration data processing, automatically load and preprocess input data according to a configuration file; Step S2, for automatically training the model and versioning the model, according to the model type specified in the configuration file, one of a set of preset deep learning models is selected for training, and after the training is completed, the generated model file is versioned and saved; Step S3, for generating an adaptive decision threshold, using the training data set to calculate the reconstruction error through the trained model, and automatically generating a decision threshold based on the statistical distribution of the reconstruction error; Step S4, for implementing an online detection service, when a detection request is received, automatically loading the latest version of the specified model type, and using the decision threshold generated in step S3 to determine the abnormality of the new data.

[0014] Further improvement of the application is that it further includes step S5 for configuring a timing task to periodically or according to preset conditions trigger the execution of the training process to form an iterative learning closed loop to automatically retrain all preset deep learning models and generate updated versions of the deep learning models to realize the continuous evolution of the intelligent anomaly detection system; the process of the iterative learning closed loop includes: starting the training task, reading the configuration file, automatically loading and preprocessing the input data through the adaptive data processing module, selecting the preset deep learning model through the configuration file, splitting the data set, automatically training the model and versioning the model, dynamically calculating the adaptive decision threshold, and completing and deploying the current deep learning model.

[0015] Compared with the prior art, the application has the beneficial effects that: first, the input data is automatically loaded and preprocessed according to the configuration file; then, a plurality of preset deep learning models selected through the configuration file are integrated in the multi-model self-learning detection engine, and all preset deep learning models adopt a unified interface specification and configuration file driving, so that a plurality of preset deep learning models can be supported and switched through a configuration method, and automatic model training, iteration, evaluation and model version management based on timestamp and version number can be realized; and after the model training is completed, the reconstruction error statistics of the training data on the model are calculated, and the decision threshold for anomaly determination is automatically calculated and generated to optimize the calculation process of the adaptive decision threshold; finally, the training and detection functions of the multi-model self-learning detection engine are packaged as a service called through a RESTful API interface to realize end-to-end automated services. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is an interaction processing module block diagram of an embodiment of the application; Figure 2 is a principle diagram of the interaction of an embodiment of the application with an external data source; Figure 3is a process schematic diagram of an iterative learning closed loop realized by an embodiment of the present application; Figure 4 is a principle schematic diagram of a model version management and loading mechanism of an embodiment of the present application; Figure 5 is a generation principle schematic diagram of an adaptive decision threshold of an embodiment of the present application. DETAILED DESCRIPTION

[0017] In the description of the present application, if "several" is involved, it means more than one; if "multiple" is involved, it means more than two; if "greater than", "less than", "more than" is involved, it should be understood as not including the number itself; if "and above", "and below", "within" are involved, they should be understood as including the number itself. If "first", "second" and the like are involved, it should be understood that they are only used to distinguish the same or similar technical features, and cannot be understood as implying / indicating the relative importance of the technical features, cannot be understood as implying / indicating the number of technical features, and cannot be understood as implying / indicating the sequence of technical features.

[0018] The preferred embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0019] As shown in the figure, the present embodiment provides an intelligent anomaly detection system based on multi-model iterative learning, which comprises: Figures 1 to 5 An adaptive data processing module 101 is configured to automatically load and preprocess input data according to a configuration file. A multi-model self-learning detection engine 102 is internally integrated with multiple preset deep learning models selected through a configuration file, and all preset deep learning models adopt a unified interface specification and configuration file driving. The multi-model self-learning detection engine 102 has the capabilities of automatic model training, evaluation, and model version management based on time stamp and version number. A dynamic threshold adaptive decision module 103 is configured to reconstruct error statistics on the model based on training data after the model training is completed, and automatically calculate and generate decision thresholds for anomaly judgment. An automatic platform and service module 104 is configured to encapsulate the training and detection functions of the multi-model self-learning detection engine 102 as services called through a RESTful API interface.

[0020] Figure 1 ​In the middle, focusing on the internal logic of the core processing layer of the embodiment, the core data flow and control flow relationship between the adaptive data processing module 101, the multi-model self-learning detection engine 102, the dynamic threshold self-adaptive decision module 103 and the automatic platform and service module 104 are shown. This figure highlights how the system works together to complete a self-learning and detection task.

[0021] Figure 2 In the middle, the intelligent anomaly detection system of the embodiment is shown as a whole, how it interacts with the external environment. Figure 2 In the middle, it is clearly shown how the intelligent anomaly detection system loads and preprocesses data from external data sources; how it is driven through configuration files, such as reading configuration files to obtain configuration information required for running, such as model type, hyperparameter, data source information, etc.; and how it provides services for user interfaces or external systems through its API interface, and finally outputs real-time alerts.

[0022] That is, the embodiment first automatically loads and preprocesses input data according to the configuration file; then integrates multiple preset deep learning models selected through the configuration file in the multi-model self-learning detection engine 102, and all preset deep learning models use unified interface specifications and configuration file driving, so it can support and switch multiple preset deep learning models through configuration, and can realize automatic model training, iteration, evaluation and model version management based on timestamp and version number; and after the model training is completed, the error statistics on the model are reconstructed based on the training data, and the decision threshold for anomaly judgment is automatically calculated and generated to optimize the calculation process of its adaptive decision threshold; finally, the training and detection functions of the multi-model self-learning detection engine 102 are also encapsulated as services that can be called through RESTful API interface, realizing end-to-end automated services.

[0023] Regarding the integration process of preset deep learning models, a unified interface specification (referred to as unified interface) and factory pattern (configuration factory instantiation) are used.

[0024] The embodiment integrates multiple preset deep learning models selected through the configuration file in the multi-model self-learning detection engine 102, which is the core of multi-model integration, not using complex model fusion algorithms (such as Ensembling or Stacking), but innovating at the system architecture level: the embodiment uses a software design pattern that combines "unified interface constraints" and "configuration factory instantiation" to seamlessly integrate and manage multiple heterogeneous deep learning models. See Step A2 and its associated implementation process of the multi-model self-learning detection engine 102, including but not limited to implementation process, function call and parameter design.

[0025] More specific implementation process, see the following description and explanation.

[0026] The core of the adaptive data processing module 101 described in this embodiment is an intelligent data loader (DataLoader), which realizes the automatic loading and preprocessing of data access by analyzing the JSON configuration file provided by the user; the configuration file is preferably a JSON configuration file.

[0027] The implementation process of the adaptive data processing module 101 described in this embodiment preferably includes steps B1 and B2.

[0028] Step B1 is used to implement an intelligent data loader (DataLoader). The intelligent data loader (DataLoader) extracts the metadata of the data source in the configuration file by analyzing the configuration file provided by the user, including the file path, separator, index of the time column, and date format; when the user does not completely specify the feature column of the metadata, the intelligent data loader automatically scans the data set and selects all numerical columns except the time column as input features. By default, the user defines the metadata of the data source in the configuration file in a declarative manner, and the intelligent data loader automatically scans the data set, which can realize the "configuration-driven automation" process, so that the intelligent anomaly detection system can adapt to different table data sources without modifying the code.

[0029] Step B2 is used to perform automatic data preprocessing. After data loading, the data cleaning process is automatically executed, which preferably includes an outlier processing mechanism based on the IQR (interquartile range) method. The implementation process is as follows: for each feature column, first calculate the first quartile Q1 and the third quartile Q3, and then obtain the interquartile range IQR through the formula IQR=Q3-Q1; then, identify all data points below Q1-1.5×IQR or above Q3+1.5×IQR as outliers, and use linear interpolation to smooth the outliers to eliminate the interference of extreme values on model training; finally, the data is normalized by standardization method and input into subsequent model training. The advantage of this design is that it can seamlessly integrate the automatic loading and preprocessing of data access into the automatic anomaly detection process as an automated step to ensure model training quality and provide high-quality input for subsequent model training.

[0030] In the multi-model self-learning detection engine 102, the entire integration process of the plurality of preset deep learning models selected through the configuration file can be divided into the following key steps: defining a unified interface specification, establishing an internal and complementary model library (src / model.py), implementing on-demand instantiation through a "model factory", and platform-based performance evaluation.

[0031] Specifically, the implementation process of the multi-model self-learning detection engine 102 (the entire integration process of the plurality of preset deep learning models) includes steps A1 to A3 by default.

[0032] Step A1 is used to design a unified interface specification and an internal model library. A unified interface specification is defined, which pre-agrees standard initialization, forward propagation and parameter loading methods through the interface specification, and a plurality of preset deep learning models are set according to different detection scenarios. Therefore, the multi-model self-learning detection engine 102 internally stores a set of designed preset deep learning model libraries that can cover different detection scenarios. All preset deep learning models follow a unified interface specification, ensuring that all deep learning models have standard initialization, forward propagation and parameter loading methods.

[0033] As shown in Figure 4 , the storage structure and dynamic loading mechanism of the model library are shown. Figure 4 A model repository (also referred to as an internal model library or model library) is shown, in which the directories of different model types (such as Model A and Model B) are clearly distinguished by grouping boxes. Each directory stores model files generated after multiple iterations of training, with different timestamps and version numbers. Figure 4 It is clearly shown that when a detection request arrives, the intelligent anomaly detection system automatically locates the directory of the specified model type and automatically selects the latest model version for loading and execution.

[0034] First, the embodiment in step A1 is used to define a unified interface specification, corresponding to BaseModel in src / model.py: a unified and abstract BaseModel base class is first defined, which inherits from the standard torch.nn.Module. This base class agrees on the core behaviors that all models integrated by the platform must have, namely the standard __init__() constructor and forward() forward propagation method. This allows the upper engine to interact with any model in the same way.

[0035] Then, the embodiment in step A1 also establishes an internal and complementary model library (src / model.py).

[0036] This embodiment forms a complementary advantage through a set of pre-installed deep learning models at the system architecture level, each with different architectures and capabilities.

[0037] Among them, SimpleAutoencoder (autoencoder model): as a benchmark model, it has simple structure and high computational efficiency, and is good at capturing global and nonlinear reconstruction patterns of data. It is suitable for scenes with relatively fixed data patterns.

[0038] VAE (Variational Autoencoder): Compared with SimpleAutoencoder, it learns the latent probability distribution of data; this makes it more robust to slight fluctuations and noise of normal data, and can effectively reduce false positives caused by normal jitter.

[0039] TransformerModel (deep learning architecture model based on attention mechanism): Its core advantage is the self-attention mechanism, which can capture long-distance dependencies and context information in time series data; it is suitable for detecting complex and context-dependent anomalies that require long-term historical data to determine.

[0040] VariationalTransformer, a model combining Variational Autoencoder (VAE) and Transformer architecture: It combines the advantages of VAE and Transformer, which can capture long-term temporal dependencies and are not sensitive to normal fluctuations; it is suitable for processing systems with complex long-term patterns and dynamic changes, and is the most advanced option in the model library.

[0041] Regarding the strategy for users to select models, the default recommendation is to select the most suitable model based on the following points: based on data characteristics (prior knowledge), if the user's time series data has obvious seasonality, periodicity or long-term dependence, then it is better to choose Transformer or VariationalTransformer model as a better starting point; based on task requirements, if the business scenario is very sensitive to false positives, then choosing VAE or VariationalTransformer, which is more robust to noise, may be better than SimpleAutoencoder.

[0042] Step A2, for realizing configuration-based model switching and instantiation. The multi-model switching of this embodiment is a configuration-based switching mechanism specified by the user on demand; the core of the multi-model self-learning detection engine 102 of this embodiment is a unified detection system class. Therefore, the step A2 first designs a unified detection system class, which is used to read the "model_type" field specified by the user in the configuration file during initialization, and the "model_type" field is used as the deep learning model type model_type field; then a model factory (ModelFactory) is implemented inside the multi-model self-learning detection engine 102, which dynamically and on demand instantiates the corresponding model class according to the deep learning model type model_type field in the configuration file, such as the "simple", "vae" and the like, and passes the hyperparameters (such as hidden_dim) related to the deep learning model in the configuration file into the model instantiation process. The configuration file contains the deep learning model type model_type field and the hyperparameters of each preset deep learning model.

[0043] Therefore, unlike the prior art, the "integration and built-in" of the deep learning model of this embodiment is realized by the combination of such "unified interface constraint" and "configuration factory instantiation", and the user only needs to modify one field in the configuration file to seamlessly switch the core detection algorithm used by the entire intelligent anomaly detection system, that is, seamlessly and automatically switch the preset deep learning model.

[0044] The step A2 of this embodiment realizes on-demand instantiation through the "model factory", which is used for AnomalyDetectionSystem in the corresponding code src / detector.py: this is the core mechanism for realizing multi-model integration. The AnomalyDetectionSystem class is the controller of the entire detection engine.

[0045] In the implementation process, preferably, the parameters are designed as follows: in the config / *.json configuration file, the embodiment designs a key parameter "model_type"; the user declares the model to be used by modifying this string (such as "simple", "vae", "transformer"). The function call and process are as follows: when the train() method of the AnomalyDetectionSystem is called, an internal logic similar to the factory pattern is executed; first, the "model_type" value in the configuration file is read, and then through an if-elif-else selection structure, a specific model class is dynamically and on-demand instantiated, and the hyperparameters related to the model in the configuration file are passed in through the constructor, including but not limited to "hidden_dim", "latent_dim", etc. The asterisk indicates a file name wildcard.

[0046] It is worth noting that the system needs to integrate deep learning models of multiple architectures that are completely different, and their initialization parameters and data processing methods (such as whether serialization is required) are different. If independent calling logic is written for each model, it will lead to code redundancy, difficulty in maintenance and expansion. Therefore, the embodiment will encounter the following technical difficulties in the actual implementation process: the complexity of heterogeneous model management and calling.

[0047] To this end, the software design patterns of "unified interface constraint" and "configuration factory instantiation" can well solve this technical difficulty. All models are abstracted as "plugins" with unified behavior, and the engine calls them through the configuration file without needing to care about their internal specific implementation.

[0048] The embodiment realizes the complete decoupling of the core engine and the specific algorithm through this architecture; makes the system have platform-level scalability, and it becomes simple and efficient to integrate any new algorithm in the future, which is one of the core advantages of the embodiment that distinguishes it from single algorithm implementation or tightly coupled systems.

[0049] Step A3, for the development of iterative learning and robustness guarantee mechanism. Through the periodic execution of the model training process by the automated script, iterative training is achieved. During the training process, the model performance is monitored in real time using the validation dataset, and when it is detected that the loss of the deep learning model on the validation dataset is no longer decreasing, an early stopping mechanism is triggered, thereby effectively preventing model overfitting. After the training is completed, the model type, timestamp and version number of the deep learning model are recorded and saved as a unique folder name, and the folder corresponding to the unique folder name includes the model type, timestamp and version number of the deep learning model. When performing a detection task, the latest version file of the corresponding model type is automatically found and loaded, and when the performance of the latest version file of the deep learning model is found to be degraded, the system automatically rolls back to the previous stable version, thereby effectively guaranteeing the stability and reliability of the intelligent anomaly detection system during the continuous learning process.

[0050] Also worth mentioning is that in the actual development process, it is found that the deep learning model (especially the Transformer with a large number of layers) is prone to overfitting when trained for a long time on some datasets. That is, the model performs excellently on the training set, but the performance on new data (validation set) that has not been seen begins to decline, resulting in poor generalization ability, which is unacceptable in actual deployment. This is one of the technical difficulties encountered in the actual research and development process of the present embodiment.

[0051] In order to overcome this technical difficulty, the present embodiment designs and integrates an automated robustness guarantee mechanism.

[0052] First, the validation set mechanism is introduced: in the automated training process (corresponding to the train method of the code src / detector.py), the present embodiment requires that the dataset be divided into a training set and a validation set according to the "validation_split" parameter (such as 0.2) in the configuration file. This is the first line of defense against overfitting.

[0053] Then, the Early Stopping logic is integrated: after each training cycle (epoch) is completed, the system will automatically evaluate the model performance on the validation set (calculate validation_loss). The present embodiment sets a "patience" counter, and if the validation set loss does not decrease for a number of consecutive cycles, the training will automatically terminate early, and the model state with the best performance will be saved.

[0054] Finally, design model version rollback: the model version management mechanism (corresponding to the code_get_latest_model_path and versioned saving logic) of the embodiment not only saves the latest model, but also provides the possibility of performance rollback. When the new version of the model generated by iterative learning performs worse than the old version in independent benchmark tests, the system administrator can easily delete the new version file, and the system will automatically roll back to the previous more stable version when it is loaded next time.

[0055] The advantages and reasons of the above design are that this set of combined mechanisms cooperatively solidify the validation set, early stopping, and version rollback into the automated engineering process, ensuring that the self-learning process of the system can be fully trained and will not cause performance degradation due to overfitting, greatly improving the reliability of automated training and the generalization ability of the final model.

[0056] The implementation process of the multi-model self-learning detection engine 102 described in the embodiment also preferably includes step A4 for system integration and testing. In step S4, the outputs of steps S1 to S3 described above are integrated into the multi-model self-learning detection engine 102, and functional testing and performance evaluation are performed to further ensure the stability and reliability of the model.

[0057] Based on the performance evaluation results (posterior knowledge) of the platform, the most important strategy of the embodiment is to enable users to use the iterative learning capability provided by the embodiment to perform a round of automated comparative testing on all models in the model library on their own real data set. The intelligent anomaly detection system will automatically generate a report containing "validation set loss (val_loss)" for each model and each iteration training, such as logs / iterative_learning_report_*.json, which includes the loss (val_loss) on the validation set. Users only need to simply compare these reports and select the model with the lowest and most stable loss (val_loss). This model is the optimal model based on data-driven; at this time, fill in the "model_type" of this optimal model in the configuration file of the production environment.

[0058] As for the strategy of user selection model, the default suggestion is to select the most suitable model based on the following points: based on data characteristics (prior knowledge), if the user's time series data has obvious seasonality, periodicity or long-term dependence, it is better to choose Transformer or VariationalTransformer model as a better starting point; if the data is multi-dimensional, but there is no obvious time correlation between dimensions, and the normal mode is relatively stable, it is reasonable to start testing from SimpleAutoencoder or VAE with lower computational cost. Based on task requirements, if the business scenario is very sensitive to false positives, for example, monitoring of non-critical business to avoid alarm fatigue, then choosing VAE or VariationalTransformer which is more robust to noise may be better than SimpleAutoencoder. Based on the limitation of computing resources and response time: if the computing resources of the deployment environment are limited, or the detection delay is required to be extremely low, then the SimpleAutoencoder with simpler structure will be more advantageous than the complex Transformer model.

[0059] In more detail, in step A1 of the embodiment, the implementation process of defining a unified interface specification includes the following sub-steps: Step A101, determine the interface methods including initialization method, forward propagation method and parameter loading method, the initialization method is used to initialize the hyperparameters of the deep learning model, such as learning rate, number of layers, number of neurons, etc.; the forward propagation method is used to receive input data and return the output of the deep learning model; the parameter loading method is used to receive a path string for loading pre-trained model parameters, so as to allow the model to load pre-trained model parameters from a specified path, so that the model can quickly adapt to new tasks or continue previous training; Step A102, create an abstract base class, define the interface methods in the abstract base class, and mark the interface methods as abstract methods, so that any subclass that inherits the abstract base class must implement these methods; therefore, the abstract base class can be used as a template to ensure that all specific model classes implement the necessary interface methods; Step A103, each specific model class inherits the abstract base class and implements the interface method, so that the interface method defined in the abstract base class can be implemented in each specific model class to provide specific functional logic.

[0060] Step A2 of the embodiment preferably includes the following sub-steps: Step A201, define a detection system class responsible for initialization and running management, such as a detection system class named DetectionSystem, which is used to handle tasks such as model loading, training, deployment, etc. Step A202, in the initialization of the detection system class, read the configuration file, parse the deep learning model type model_type field and its hyperparameters model_params field, and store the deep learning model type model_type field and its hyperparameters model_params field as attributes of the detection system class for subsequent steps; Step A203, according to the deep learning model type model_type field, call the model factory (ModelFactory) to dynamically instantiate the corresponding deep learning model, and pass the hyperparameters model_params field to the model factory, and pass it to the constructor of the deep learning model through the model factory, to complete the instantiation of the model class.

[0061] Therefore, the embodiment is different from the prior art in that the multi-model integration in the prior art usually refers to integration at the algorithm level (such as ensemble learning). Unlike the prior art, the "integration" of the embodiment is integration at the system architecture level, allowing users to switch and deploy any one of the models through a simple configuration method.

[0062] Based on the above differences, the advantages of the embodiment include: high flexibility and scalability, the architecture of the embodiment is "pluggable"; it reduces the threshold for use and experiment, and comparing the effects of different models on a specific data set becomes an automated and scriptable process; systematic life cycle management, the embodiment integrates the entire life cycle of model training, versioning, deployment, and threshold generation into unified automated management.

[0063] In addition, the model selection strategy of the embodiment is complementary to the advantages.

[0064] In the embodiment, the final decision-making right of model selection is given to the user, which is a "human-computer collaboration" strategy. The system is responsible for minimizing the cost of model selection, and the user makes the final decision according to his own business insight.

[0065] In the dynamic threshold self-adaptive decision module 103 described in the embodiment, the mean μ and standard deviation σ of the training data reconstruction error E are calculated first, then the adjustment factor factor representing the business risk tolerance is combined, and the decision threshold Threshold is calculated and generated through the formula Threshold = μ + (factor × σ).

[0066] It should be noted that, unlike the prior art, the mean μ and the standard deviation σ described in the embodiment are not statistical values of the original data, but statistical values of the model reconstruction error E. This enables the decision threshold Threshold to be dynamically bound to the specific model, the specific training data, and the specific learning effect, and measures the degree to which the data points deviate from the "normal pattern understood by the model", so it can be more accurate than the prior art. At the same time, the parameter factor is defined as an adjustment factor for business risk tolerance, which is a configurable factor that can be set and adjusted according to actual conditions, and its preferred value range is by default between [2.0, 5.0], a lower adjustment factor factor is suitable for a high recall rate scenario, and a higher adjustment factor factor is suitable for a high precision rate scenario, providing an effective correspondence between technology and business risk.

[0067] After the model training is completed, the system inputs the entire training data set into the solidified model again. For each data point vector X in the training set, a reconstruction vector X' is generated through the deep learning model, and the reconstruction error E of the training data is calculated by the formula E=(1 / n)×Σ(X_i-X'_i)², where n is the dimension of the data point vector, X_i represents the i-th data point vector, X'_i represents the i-th reconstruction vector, and i is a natural number. The embodiment traverses all training samples to obtain a complete reconstruction error distribution that reflects the degree to which the model understands normal data.

[0068] As Figure 5 shown is the core algorithm of the dynamic threshold adaptive decision module 103 (also referred to as the dynamic threshold decision module). The process starts from the left side with the input of the trained model and the training data set, goes through two core processing steps of calculating the reconstruction error E and the statistical error distribution, and finally generates the final adaptive decision threshold in combination with the business risk tolerance adjustment factor (i.e., the adjustment factor factor) input from the top as an independent parameter. Figure 5 The layout shown clearly separates the data flow and the parameter input, highlighting the algorithm flow.

[0069] By default, the adjustment factor factor can be a static configuration, but the embodiment allows a feedback-based dynamic adaptive adjustment mechanism as its preferred implementation to constitute a special processing step of the application, which is more conducive to the decision threshold Threshold to be dynamically bound to the specific model, the specific training data, and the specific learning effect.

[0070] The reason for this design is that the static adjustment factor factor provides business interpretability, but the optimal value still needs to be set manually based on experience. To further improve the intelligent and automated level of the system, the embodiment designs a mechanism to allow the adjustment factor factor to be self-optimized according to the feedback of actual alarms, and the implementation process preferably includes steps D1 to D5.

[0071] Step D1, introduce a feedback mechanism, in the automation platform and service module 104, in addition to pushing the alarm, an API interface (for example, POST / alerts / {alert_id} / feedback) for alarm feedback is provided. After handling an alarm, the operation and maintenance personnel or downstream automation system can call this API interface and input the feedback label, such as { "is_true_positive": true} or { "is_true_positive": false}.

[0072] Step D2, design a dynamic adjustment algorithm, periodically count the recent alarm feedback through a timing task.

[0073] Step D3, calculate the precision, calculate the precision of the recent alarm by the formula precision of recent alarm (Precision) = (number of true positive alarms) / (total number of alarms).

[0074] Step D4, set the target precision, set a target precision (TargetPrecision) in the configuration file in advance, for example, the target precision is set to 0.8 by default, that is, it is hoped that 80% of the alarms are accurate.

[0075] Step D5, dynamically adjust the adjustment factor factor, the system dynamically adjusts the factor value according to the difference between the current precision and the target precision. By default, if the false positive is too high, that is, the precision of the recent alarm is less than the target precision, then the adjustment factor factor is multiplied by 1.05, and the result is used as the latest adjustment factor, so that the system is more accurate / conservative; if there are too few alarms, there may be a false negative problem, then the adjustment factor factor is multiplied by 0.95, and the result is used as the latest adjustment factor, so that the system is more sensitive to avoid false negatives.

[0076] The preferred adjustment algorithm pseudo code is: / / Pseudo code IF (current_precision < target_precision): / / Too many false positives, need to increase the threshold to make the system more conservative, factor = factor x 1.05, ELSE IF (current_precision > target_precision + tolerance): / / Alarm too few, may be missed, need to reduce the threshold, make the system more sensitive, factor = factor x 0.95.

[0077] The value of the above-mentioned adjusted adjustment factor factor will be persisted and used for the next threshold calculation. This feedback-based closed-loop adaptive adjustment mechanism is one of the core innovations of the embodiment at the decision-making level. It changes the threshold adjustment from a static, open problem to a dynamic, convergent, and closed-loop problem with a clear optimization goal. This makes the system not only learn the normal mode of the data, that is, it can be trained through the model; It can also learn the judgment standard of the operation and maintenance personnel, that is, it is realized through feedback, so that the suppression of alarm storm and the discovery process of key exceptions become more intelligent and automated.

[0078] In the automatic platform and service module 104 described in the embodiment, the implementation process of the service encapsulated by the RESTful API interface call includes the following steps: Step C1, define the interface layer, use the Web framework to build the interface layer, define the HTTP endpoint, use the training task POST / train to receive the training request, use the detection task POST / detect to receive the detection request, process the HTTP request and implement the data verification logic; Step C2, implement the service layer, after receiving the API interface request, create and start a background training task through the service layer, call the API interface of the multi-model self-learning detection engine 102, pass the request parameters to the multi-model self-learning detection engine 102, execute the machine learning task, process the start, monitoring and result return of the learning task; The service layer serves as a bridge between the interface layer and the multi-model self-learning detection engine 102, responsible for business logic orchestration; Step C3, the core engine layer, that is, the multi-model self-learning detection engine 102; instantiate and call the API interface of the multi-model self-learning detection engine 102 through the service layer to execute the actual machine learning task; Step C4 implements platform capabilities. This layered and decoupled encapsulation approach allows core AI capabilities to be developed and iterated independently of web services. The interface layer provides services to the user interface or external systems via HTTP endpoints, including API interfaces for providing training and detection requests. The platform provides services to the user interface or external systems via APIs and pushes alarm information in real time when anomalies are detected through WebSocket full-duplex communication service.

[0079] This embodiment also provides an automated method for intelligent anomaly detection based on multi-model iterative learning, applied to the intelligent anomaly detection system based on multi-model iterative learning as described above, and includes the following steps: Step S1 is used for configurable data processing, which automatically loads and preprocesses input data according to the configuration file; Step S2 is used to automatically train the model and save the model in a versioned manner. According to the model type specified in the configuration file, one of the preset deep learning models is selected for training, and the generated model file is saved in a versioned manner after training is completed. Step S3 is used to generate an adaptive decision threshold. The reconstruction error is calculated using the training dataset through the trained model, and the decision threshold is automatically generated based on the statistical distribution of the reconstruction error. Step S4 is used to implement the online detection service. When a detection request is received, the latest version of the specified model type is automatically loaded, and the decision threshold generated in step S3 is used to determine the anomalies of the new data.

[0080] like Figure 3 As shown, this embodiment also includes step S5, which is used to configure a timed task to periodically or according to preset conditions trigger the execution of the training process, forming an iterative learning closed loop, so as to automatically retrain all preset deep learning models, generate updated versions of deep learning models, and realize the continuous evolution of the intelligent anomaly detection system. The process of the iterative learning closed loop includes: starting the training task, reading the configuration file, automatically loading and preprocessing the input data through the adaptive data processing module 101, selecting the preset deep learning model through the configuration file, segmenting the dataset, automatically training the model and saving the model in a versioned manner, dynamically calculating the adaptive decision threshold, and completing and deploying the current deep learning model. Figure 3 This demonstrates the end-to-end automated main workflow of this embodiment. A separate block diagram clearly illustrates the "iterative learning loop," showing that the main workflow can be triggered periodically or on demand to achieve continuous model evolution.

[0081] Preferably, the embodiment also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the intelligent anomaly detection system as described above or the automated method of intelligent anomaly detection as described above.

[0082] In summary, the intelligent anomaly detection system based on multi-model iterative learning and the automated method thereof implemented by the embodiment have the following remarkable technical effects in actual application and testing: First, high automation and ease of use. Users only need to provide data and a configuration file, and the system can automatically complete the whole process of training, decision threshold setting, model version management, anomaly detection and iterative learning, greatly reducing the use threshold. In detail, the embodiment has the following advantages.

[0083] High flexibility and scalability: the architecture of the embodiment is "pluggable". Even if more advanced detection algorithms appear in the future, they only need to be developed according to the unified interface specification and added to the "model factory" branch, which can be easily integrated into the platform without changing the core process of the whole system.

[0084] Reduced use and experiment threshold: in the prior art, the cost of replacing and testing different models is very high. In the embodiment, comparing the effects of different models on a specific data set becomes an automated and scriptable process (such as the iterative learning loop in main.py), which greatly facilitates users to find the optimal model for their business scenarios.

[0085] Systematic life cycle management: the embodiment not only integrates the model itself, but more importantly, it integrates the whole life cycle of the model, including training, versioning, deployment, threshold generation, etc. into unified automated management, which is not available in the prior art of simply integrating model algorithms.

[0086] Second, systematic multi-model support. A variety of complementary preset deep learning models are seamlessly integrated in one platform, and users can switch between them through simple configuration to cope with different data challenges.

[0087] That is, as described above, the embodiment implements the core of multi-model integration, not using complex model fusion algorithms such as Ensembling or Stacking, but innovating at the system architecture level: through the combination of "unified interface constraints" and "configurable factory instantiation" software design patterns, seamless integration and management of multiple heterogeneous deep learning models are achieved.

[0088] Third, reliable and continuous learning capability, providing optimized iterative learning and model version management mechanisms, combined with early stopping, model rollback, and other robustness designs, to ensure safe and effective performance improvement through continuous training.

[0089] Fourth, precise adaptive decision-making, with dynamic threshold generation algorithms enabling the system to intelligently set alarm thresholds based on actual data and model conditions, effectively reducing false positives and false negatives.

[0090] Fifth, complete platformized services, providing API interfaces and complete backend services to easily integrate anomaly detection capabilities into any existing business system or monitoring platform.

[0091] Below, an application example of industrial equipment time series data anomaly detection based on the implementation of the present embodiment will be provided to demonstrate the end-to-end automated operation of the intelligent anomaly detection system in the intelligent manufacturing scenario; its application scenario is: real-time anomaly monitoring of the selected 5-dimensional sensor time series data of industrial equipment.

[0092] Step 1: System configuration.

[0093] The user creates a telemetry_config.json configuration file, which defines data sources, model parameters, and training parameters. In this application example, the user chooses to use a transformer model. The pseudo-code representation is as follows: JSON { "data": { "file_path": " / path / to / data.csv",...}, "model": { "model_type": "transformer",...}, "training": { "epochs": 100,...}, "detection": { "threshold_factor": 3.0} } Step 2: Trigger automated training through API interface or automation script.

[0094] The user initiates the self-learning process by calling the corresponding interface of the training task POST / train, or directly executing a script. This process strictly follows the main workflow shown in Figure 3 .

[0095] At this point, the intelligent anomaly detection system performs: The input data is loaded and preprocessed by the adaptive data processing module 101.

[0096] The multi-model self-learning detection engine 102 instantiates the TransformerModel model according to the configuration and performs training.

[0097] After training, the system automatically saves the model version, and the management mechanism is as shown in Figure 4

[0098] At the same time, the dynamic threshold adaptive decision module 103 automatically calculates the dynamic decision threshold, and the generation process is as shown in Figure 5

[0099] Step 3: Perform anomaly detection.

[0100] The external system submits new data to be detected through the detection task POST / detect corresponding interface.

[0101] At this time, the intelligent anomaly detection system performs: The automation platform and service module 104 receives the request and calls the multi-model self-learning detection engine 102.

[0102] The engine automatically loads the latest version of the transformer model directory according to the mechanism shown in Figure 4

[0103] Use the loaded model and the calculated decision threshold to make an abnormality determination on the new data.

[0104] Return the detection result and push the alarm in real time through the WebSocket full-duplex communication service.

[0105] Step 4: Iterative learning and model evolution.

[0106] The administrator configures a timing task to periodically execute the training process, forming an iterative learning closed loop in Figure 3 The intelligent anomaly detection system will automatically retrain all pre-set deep learning models to generate updated versions of the models, thereby realizing the continuous evolution of the system.

[0107] The above is a further detailed description of the present application in combination with specific preferred embodiments, and cannot be considered as limiting the specific implementation of the present application to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be considered as falling within the protection scope of the present application.​​​

Claims

1. An intelligent anomaly detection system based on multi-model iterative learning, characterized in that, include: The adaptive data processing module is used to automatically load and preprocess input data according to the configuration file; The multi-model self-learning detection engine integrates multiple preset deep learning models selected through configuration files, and all preset deep learning models adopt a unified interface specification and configuration file-driven approach. The multi-model self-learning detection engine has the ability to automatically train and evaluate models, as well as manage model versions based on timestamps and version numbers. The dynamic threshold adaptive decision module is used to reconstruct error statistics on the model based on training data after the model training is completed, and automatically calculate and generate decision thresholds for anomaly detection. The automation platform and service module is used to encapsulate the training and detection functions of the multi-model self-learning detection engine into services that can be called through a RESTful API interface.

2. The intelligent anomaly detection system based on multi-model iterative learning according to claim 1, characterized in that, The implementation process of the multi-model self-learning detection engine includes the following steps: Step A1: Design a unified interface specification and built-in model library. Define a unified interface specification, and pre-determine the standard initialization, forward propagation and parameter loading methods through the interface specification. Set up a variety of preset deep learning models according to different detection scenarios. Step A2 implements configuration-based model switching and instantiation. First, a unified detection system class is designed to read the deep learning model type `model_type` field from the configuration file during initialization. Then, a model factory is implemented within the multi-model self-learning detection engine. This model factory dynamically instantiates the corresponding model class based on the deep learning model type `model_type` field in the configuration file, and passes the hyperparameters related to the deep learning model in the configuration file as parameters during model instantiation. The configuration file contains the deep learning model type `model_type` field and the hyperparameters of each preset deep learning model. Step A3 involves developing an iterative learning and robustness assurance mechanism. This mechanism uses automated scripts to periodically execute the model training process. During training, the model performance is monitored in real time using a validation dataset. When the loss of the deep learning model on the validation dataset no longer decreases, an early stopping mechanism is triggered. After training is complete, the model type, timestamp, and version number of the deep learning model are recorded and saved as a unique folder name. When performing detection tasks, the latest version file of the corresponding model type is automatically located and loaded. If the performance of the deep learning model in the latest version file degrades, the system automatically rolls back to the previous stable version.

3. The intelligent anomaly detection system based on multi-model iterative learning according to claim 2, characterized in that, In step A1, the process of defining a unified interface specification includes the following sub-steps: Step A101: Determine the interface methods including an initialization method, a forward propagation method, and a parameter loading method. The initialization method is used to initialize the hyperparameters of the deep learning model; the forward propagation method is used to receive input data and return the output of the deep learning model; the parameter loading method is used to receive a path string and load the pre-trained model parameters. Step A102: Create an abstract base class, define the interface methods in the abstract base class, and mark the interface methods as abstract methods; Step A103: In each specific model class, inherit the abstract base class and implement the interface methods.

4. The intelligent anomaly detection system based on multi-model iterative learning according to claim 2, characterized in that, Step A2 includes the following sub-steps: Step A201: Define the detection system class responsible for initialization and operation management; Step A202: In the initialization of the detection system class, read the configuration file, parse out the deep learning model type field model_type and its hyperparameter field model_params, and store the deep learning model type field model_type and its hyperparameter field model_params as attributes of the detection system class; Step A203: Based on the deep learning model type field model_type, call the model factory to dynamically instantiate the corresponding deep learning model, and pass the hyperparameter field model_params to the model factory, which then passes it to the constructor of the deep learning model to complete the instantiation of the model class.

5. The intelligent anomaly detection system based on multi-model iterative learning according to any one of claims 1 to 4, characterized in that, In the dynamic threshold adaptive decision module, the mean μ and standard deviation σ of the training data reconstruction error E are first calculated. Then, combined with the adjustment factor factor representing the business risk tolerance, the decision threshold Threshold is calculated and generated using the formula Threshold=μ+(factor×σ).

6. The intelligent anomaly detection system based on multi-model iterative learning according to claim 5, characterized in that, For each data point vector X in the training set, a reconstruction vector X' is generated by a deep learning model. The reconstruction error E of the training data is calculated by the formula E=(1 / n)×Σ(X_i-X'_i)², where n is the dimension of the data point vector, X_i represents the i-th data point vector, X'_i represents the i-th reconstruction vector, and i is a natural number.

7. The intelligent anomaly detection system based on multi-model iterative learning according to any one of claims 1 to 4, characterized in that, The implementation process of the adaptive data processing module includes the following steps: Step B1: Implement the intelligent data loader to parse the user-provided configuration file and extract metadata of the data source in the configuration file, including file path, delimiter, index of time column and date format; when the user does not fully specify the feature columns of the metadata, the intelligent data loader automatically scans the dataset and selects all numeric columns except the time column as input features. Step B2 involves automated data preprocessing. After data loading, a data cleaning process is automatically executed. For each feature column, the first quartile Q1 and the third quartile Q3 are calculated, and the interquartile range IQR is obtained using the formula IQR = Q3 - Q1. Then, all data points below Q1 - 1.5 × IQR or above Q3 + 1.5 × IQR are identified as outliers, and linear interpolation is used to smooth these outliers. Finally, the data is normalized using a standardization method and then input into the subsequent model training.

8. The intelligent anomaly detection system based on multi-model iterative learning according to any one of claims 1 to 4, characterized in that, The implementation process of the service encapsulated in the automation platform and service module as a service called through a RESTful API interface includes the following steps: Step C1: Define the interface layer. Use a web framework to build the interface layer, define HTTP endpoints, use the training task POST / train to receive training requests, use the detection task POST / detect to receive detection requests, process HTTP requests and implement data verification logic. Step C2: Implement the service layer. After receiving the API interface request, create and start a background training task through the service layer, call the API interface of the multi-model self-learning detection engine, pass the request parameters to the multi-model self-learning detection engine, execute the machine learning task, and handle the start-up, monitoring and result return of the learning task. Step C3: Instantiate and call the API interface of the multi-model self-learning detection engine through the service layer; Step C4: Implement platform capabilities. The interface layer provides services to the user interface or external systems through HTTP endpoints, including providing API interfaces for training requests and detection requests; and pushes alarm information in real time when anomalies are detected through WebSocket full-duplex communication service.

9. An automated method for intelligent anomaly detection based on multi-model iterative learning, characterized in that, An intelligent anomaly detection system based on multi-model iterative learning as described in any one of claims 1 to 8 is applied, and includes the following steps: Step S1 is used for configurable data processing, which automatically loads and preprocesses input data according to the configuration file; Step S2 is used to automatically train the model and save the model in a versioned manner. According to the model type specified in the configuration file, one of the preset deep learning models is selected for training, and the generated model file is saved in a versioned manner after training is completed. Step S3 is used to generate an adaptive decision threshold. The reconstruction error is calculated using the training dataset through the trained model, and the decision threshold is automatically generated based on the statistical distribution of the reconstruction error. Step S4 is used to implement the online detection service. When a detection request is received, the latest version of the specified model type is automatically loaded, and the decision threshold generated in step S3 is used to determine the anomalies of the new data.

10. The automated method for intelligent anomaly detection based on multi-model iterative learning according to claim 9, characterized in that, It also includes step S5, which is used to configure a timed task to periodically or according to preset conditions to trigger the execution of the training process, forming an iterative learning closed loop, so as to automatically retrain all preset deep learning models, generate updated versions of deep learning models, and realize the continuous evolution of the intelligent anomaly detection system. The iterative learning closed-loop process includes: starting the training task, reading the configuration file, automatically loading and preprocessing the input data through the adaptive data processing module, selecting a preset deep learning model through the configuration file, segmenting the dataset, automatically training the model and saving the model in versions, dynamically calculating the adaptive decision threshold, and completing and deploying the current deep learning model.

Images