Abnormal information processing method based on knowledge graph and large model training

Abstract

The invention relates to the technical field of artificial intelligence, and discloses an abnormal information processing method based on a knowledge graph and large model training, and the method specifically comprises the following steps: 1, obtaining to-be-detected time series data through the knowledge graph; 2, inputting the to-be-detected time sequence data into a pre-trained sequence anomaly detection model, outputting an anomaly detection result, constructing a sample data acquisition script template by using a first preset parameter, and obtaining an anomaly detection result through the sample data acquisition script template, a sample data preprocessing script template and a model training script template. The pre-trained sequence anomaly detection model for anomaly detection is obtained, so that the human input is saved, the training time of the model is saved, the service updating efficiency of the sequence anomaly detection model is improved, and the quality of produced or processed products is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an abnormal information processing method based on knowledge graph and large model training. Background Art

[0002] Artificial intelligence is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. Industrial data covers many different types of data, some of the main types include: Production data: Production data covers various indicators in the production process, such as output, production rate, equipment utilization, downtime, etc. These data can help companies optimize production processes and improve efficiency and quality. Equipment data: Equipment data includes real-time monitoring data of equipment operating status, temperature, pressure, vibration and other parameters. By analyzing equipment data, predictive maintenance can be performed to prevent equipment failure and downtime. Quality data: Quality data records various indicators of product quality, such as defect rate, qualified rate, return rate, etc. These data can help companies monitor product quality and take timely measures to improve the production process. Energy data: Energy data includes energy consumption, such as the usage and cost of resources such as electricity, natural gas and water. By analyzing energy data, companies can identify opportunities for energy conservation and emission reduction and reduce production costs. Supply chain data: Supply chain data covers data on raw material procurement, production planning, inventory management and other links. By analyzing supply chain data, companies can optimize supply chain management and improve logistics efficiency. Environmental data: Environmental data records the environmental impacts of the production process, such as wastewater discharge, exhaust gas emissions, noise, etc. Companies need to monitor and report this data to ensure compliance with environmental regulations. In real life, industrial data contains a large amount of time series data. Time series data refers to a series of data arranged in the order of the time of occurrence of an action. Therefore, time series data records the fluctuation information of an action in the time dimension. For example, in the field of network security, when conducting operation and maintenance management, time series data can be used as an observation indicator to detect anomalies in time series data, which can lay the foundation for subsequent fault location and other processing.

[0003] For the time series data collected in real time by the industrial data acquisition and monitoring system, abnormal information processing is performed and potential or hidden problems are mined from the abnormal information, which will affect the quality of the products produced or processed. In the time series anomaly detection task, the time series anomaly detection model needs to be trained by manually written scripts. Each time a detection is required, it may be necessary to log in to the relevant server or other client to rewrite the script or modify some parameters before executing the script, which not only makes the process cumbersome, but also affects the update efficiency of the model detection service.

[0004] To this end, we propose an abnormal information processing method based on knowledge graph and large model training. Summary of the invention

[0005] The present invention mainly solves the technical problems existing in the above-mentioned prior art and provides an abnormal information processing method based on knowledge graph and large model training.

[0006] In order to achieve the above objectives, the present invention adopts the following technical solution, which is an abnormal information processing method based on knowledge graph and large model training, and specifically includes the following steps:

[0007] Step 1: Obtain the time series data to be tested through the knowledge graph;

[0008] Step 2: Input the time series data to be detected into a pre-trained sequence anomaly detection model, output the anomaly detection result, use the first preset parameter to construct a sample data acquisition script template, wherein the first preset parameter represents the associated parameter of the initial training sample data; use the second preset parameter to construct a sample data preprocessing script template, wherein the second preset parameter is determined based on the first preset parameter; and use the third preset parameter to construct a model training script template, wherein the third preset parameter is determined based on the second preset parameter;

[0009] Step 3: when it is determined that the sample data acquisition script template, the sample data preprocessing script template and the model training script template have all passed the tests, determine the training identification information of the sequence anomaly detection model, call the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the training identification information, and execute the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the preset training cycle and the preset training time parameter value, and generate the model parameter information;

[0010] Step 4: construct a sample data acquisition script template, test the sample data acquisition script template, construct a sample data preprocessing script template, test the sample data preprocessing script template, construct a model training script template, test the model training script template, determine the training identification information of the sequence anomaly detection model, call the sample data acquisition script template, the sample data preprocessing script template and the model training script template, and after executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the preset training cycle and the preset training time parameter value, generate model parameter information, and obtain the pre-trained sequence anomaly detection model according to the model parameter information;

[0011] Step 5: Use the preset training time parameter value to execute the sample data acquisition script template to generate initial training sample data information; use the preset training time parameter value and the initial training sample data information to execute the sample data preprocessing script template to generate target training sample data information; use the preset training time parameter value and the target training sample data information to execute the model training script template to generate model parameter information.

[0012] Preferably, the pre-trained sequence anomaly detection model is determined by the model parameter information generated after executing the sample data acquisition script template to generate initial training sample data information, executing the sample data preprocessing script template to process the initial training sample data information to generate target training sample data information, and executing the model training script template to process the target training sample data information according to the preset training cycle and preset training time parameter values.

[0013] Preferably, the associated parameters include storage address parameters of the initial training sample data and time parameters of the initial training sample data.

[0014] Preferably, the step of obtaining the time series data to be detected in the first step includes: obtaining the time series data from the knowledge graph data source, performing data cleaning and formatting on the time series data, and obtaining the time series data to be detected.

[0015] Preferably, the step of inputting the time series data to be detected into a pre-trained sequence anomaly detection model in the second step includes: inputting the time series data to be detected into the model according to a preset data format, performing anomaly detection on the input data using the model, and outputting anomaly detection results.

[0016] Preferably, the step of generating model parameter information in the third step includes: training the input training sample data according to a preset training algorithm and training parameters, continuously adjusting the model parameters until the model converges or reaches a preset number of training rounds, and generating final model parameter information.

[0017] Preferably, the steps of constructing a sample data acquisition script template, a sample data preprocessing script template and a model training script template in the fourth step include: writing corresponding script templates according to the sample data acquisition method, preprocessing method and training method, and testing and optimizing the script templates to ensure the correctness and efficiency of the script templates.

[0018] Preferably, the step of executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template in the fifth step includes: according to the preset training cycle and training time parameter value, the sample data acquisition script template, the sample data preprocessing script template and the model training script template are periodically executed, and the training sample data and model parameter information are continuously updated to improve the accuracy and adaptability of the model.

[0019] Beneficial Effects

[0020] The present invention provides an abnormal information processing method based on knowledge graph and large model training. The abnormal information processing method based on knowledge graph and large model training provides an automated initial training sample data information acquisition, initial training sample data information preprocessing and sequence abnormality detection model training method for the sequence abnormality detection model through sample data acquisition script template, sample data preprocessing script template and model training script template, and obtains a pre-trained sequence abnormality detection model for abnormality detection, which not only saves manpower input, but also saves model training time, improves the service update efficiency of the sequence abnormality detection model, and improves the quality of the produced or processed products. At the same time, through the preset training cycle and training time parameter values, the training sample data and model parameter information are updated regularly, the real-time and accuracy of the model are guaranteed, and the efficiency of industrial data collection is improved. In addition, the present invention also uses knowledge graph and large model training technology to improve the accuracy and efficiency of abnormality detection, and provides strong support for operation and maintenance management in fields such as network security. DETAILED DESCRIPTION

[0021] Embodiment 1: An abnormal information processing method based on knowledge graph and large model training specifically includes the following steps: Step 1: Acquire time series data to be detected through knowledge graph; Step 2: Input the time series data to be detected into a pre-trained sequence anomaly detection model, output the anomaly detection result, and use the first preset parameter to construct a sample data acquisition script template, wherein the first preset parameter represents the associated parameter of the initial training sample data; Use the second preset parameter to construct a sample data preprocessing script template, wherein the second preset parameter is determined based on the first preset parameter; and Use the third preset parameter to construct a model training script template, wherein the third preset parameter is determined based on the second preset parameter; Step 3: When it is determined that the sample data acquisition script template, the sample data preprocessing script template and the model training script template have all passed the test, determine the training identification information of the sequence anomaly detection model, and call the sample data acquisition script template according to the training identification information. Take the script template, the sample data preprocessing script template and the model training script template, and execute the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the preset training cycle and the preset training time parameter value, and generate model parameter information; Step 4: construct the sample data acquisition script template, test the sample data acquisition script template, construct the sample data preprocessing script template, test the sample data preprocessing script template, construct the model training script template, test the model training script template, determine the training identification information of the sequence anomaly detection model, call the sample data acquisition script template, the sample data preprocessing script template and the model training script template, and execute the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the preset training cycle and the preset training time parameter value, and generate model parameter information, and obtain the pre-trained sequence anomaly detection model according to the model parameter information;Step 5: Using the preset training time parameter value, executing the sample data acquisition script template to generate initial training sample data information, using the preset training time parameter value and the initial training sample data information, executing the sample data preprocessing script template to generate the target training sample data information, using the preset training time parameter value and the target training sample data information, executing the model training script template to generate the model parameter information, through the sample data acquisition script template, the sample data preprocessing script template and the model training script template, an automated initial training sample data information acquisition, initial training sample data information preprocessing and sequence anomaly detection model training method is provided for the sequence anomaly detection model, and a pre-trained sequence anomaly detection model for anomaly detection is obtained, which not only saves manpower input, but also saves model training time, improves the service update efficiency of the sequence anomaly detection model, and improves the quality of the produced or processed products. At the same time, through the preset training cycle and training time parameter value, the training sample data and model parameter information are updated regularly, the real-time and accuracy of the model are guaranteed, and the efficiency of industrial data collection is improved. In addition, the present invention also uses knowledge graphs and large model training technology to improve the accuracy and efficiency of anomaly detection, and provides strong support for operation and maintenance management in the fields of network security. ;

[0022] Embodiment 2: Based on Embodiment 1, the pre-trained sequence anomaly detection model is determined by the model parameter information generated after executing the sample data acquisition script template to generate initial training sample data information, executing the sample data preprocessing script template to process the initial training sample data information to generate target training sample data information, and executing the model training script template to process the target training sample data information according to the preset training cycle and preset training time parameter values.

[0023] Embodiment 3: Based on Embodiment 1 and Embodiment 2, the associated parameters include storage address parameters of the initial training sample data and time parameters of the initial training sample data. Preferably, the step of obtaining the time series data to be detected in the first step includes: obtaining the time series data from the knowledge graph data source, performing data cleaning and formatting on the time series data, and obtaining the time series data to be detected. The step of inputting the time series data to be detected into a pre-trained sequence anomaly detection model in the second step includes: inputting the time series data to be detected into the model according to a preset data format, performing anomaly detection on the input data using the model, and outputting anomaly detection results. The step of generating model parameter information in the third step includes: training the input training sample data according to a preset training algorithm and training parameters, and continuously adjusting the model parameters until the model converges or reaches a preset number of training rounds, and generating the final model parameter information. The step of constructing a sample data acquisition script template, a sample data preprocessing script template, and a model training script template in the fourth step includes: According to the method of obtaining, preprocessing and training the sample data, corresponding script templates are written respectively, and the script templates are tested and optimized to ensure the correctness and efficiency of the script templates. The steps of executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template in the fifth step include: according to the preset training cycle and training time parameter value, the sample data acquisition script template, the sample data preprocessing script template and the model training script template are regularly executed to continuously update the training sample data and model parameter information to improve the accuracy and adaptability of the model. Through the sample data acquisition script template, the sample data preprocessing script template and the model training script template, an automated initial training sample data information acquisition, initial training sample data information preprocessing and sequence anomaly detection model training method is provided for the sequence anomaly detection model, and a pre-trained sequence anomaly detection model for anomaly detection is obtained, which not only saves manpower input, but also saves model training time and improves the service update efficiency of the sequence anomaly detection model.

[0024] Working principle: Obtain the time series data to be detected: Obtain the time series data from the knowledge graph data source, and clean and format the data to obtain the time series data to be detected that meets the model input requirements. Build and test script templates: According to the sample data acquisition method, preprocessing method and training method, respectively build sample data acquisition script templates, sample data preprocessing script templates and model training script templates, and test and optimize these script templates to ensure their correctness and efficiency. Determine the training identification information and execute the script template: After determining that the script template test has passed, determine the training identification information of the sequence anomaly detection model, and then, according to the training identification information, call the corresponding script template, and execute these script templates according to the preset training cycle and training time parameter values ​​to generate model parameter information. Train the sequence anomaly detection model: Use the generated model parameter information to obtain a pre-trained sequence anomaly detection model. This model can output anomaly detection results based on the input time series data to be detected. Update the model: By periodically executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template, the training sample data and model parameter information are continuously updated to maintain the real-time and accuracy of the model.

[0025] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An abnormal information processing method based on knowledge graph and large model training, characterized in that: The specific steps include: Step 1: Obtain the time series data to be tested through the knowledge graph; Step 2: Input the time series data to be detected into a pre-trained sequence anomaly detection model, output the anomaly detection result, use the first preset parameter to construct a sample data acquisition script template, wherein the first preset parameter represents the associated parameter of the initial training sample data; use the second preset parameter to construct a sample data preprocessing script template, wherein the second preset parameter is determined based on the first preset parameter; and use the third preset parameter to construct a model training script template, wherein the third preset parameter is determined based on the second preset parameter; Step 3: when it is determined that the sample data acquisition script template, the sample data preprocessing script template and the model training script template have all passed the test, determine the training identification information of the sequence anomaly detection model, call the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the training identification information, and execute the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the preset training cycle and the preset training time parameter value, and generate the model parameter information; Step 4: construct a sample data acquisition script template, test the sample data acquisition script template, construct a sample data preprocessing script template, test the sample data preprocessing script template, construct a model training script template, test the model training script template, determine the training identification information of the sequence anomaly detection model, call the sample data acquisition script template, the sample data preprocessing script template and the model training script template, and after executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template according to the preset training cycle and the preset training time parameter value, generate model parameter information, and obtain the pre-trained sequence anomaly detection model according to the model parameter information; Step 5: Use the preset training time parameter value to execute the sample data acquisition script template to generate initial training sample data information; use the preset training time parameter value and the initial training sample data information to execute the sample data preprocessing script template to generate target training sample data information; use the preset training time parameter value and the target training sample data information to execute the model training script template to generate model parameter information.

2. The abnormal information processing method based on knowledge graph and large model training according to claim 1 is characterized in that: The pre-trained sequence anomaly detection model is determined by the model parameter information generated after executing the sample data acquisition script template to generate initial training sample data information, executing the sample data preprocessing script template to process the initial training sample data information to generate target training sample data information, and executing the model training script template to process the target training sample data information according to the preset training cycle and the preset training time parameter value.

3. The abnormal information processing method based on knowledge graph and large model training according to claim 1 is characterized in that: The associated parameters include storage address parameters of the initial training sample data and time parameters of the initial training sample data.

4. The abnormal information processing method based on knowledge graph and large model training according to claim 1 is characterized in that: The step of obtaining the time series data to be detected in the first step includes: obtaining the time series data from the knowledge graph data source, performing data cleaning and formatting on the time series data, and obtaining the time series data to be detected.

5. The abnormal information processing method based on knowledge graph and large model training according to claim 1 is characterized in that: The step of inputting the time series data to be detected into a pre-trained sequence anomaly detection model in the second step includes: inputting the time series data to be detected into the model according to a preset data format, using the model to perform anomaly detection on the input data, and outputting anomaly detection results.

6. The abnormal information processing method based on knowledge graph and large model training according to claim 1 is characterized in that: The step of generating model parameter information in the third step includes: training the input training sample data according to the preset training algorithm and training parameters, continuously adjusting the model parameters until the model converges or reaches the preset number of training rounds, and generating the final model parameter information.

7. The abnormal information processing method based on knowledge graph and large model training according to claim 1 is characterized in that: The steps of constructing a sample data acquisition script template, a sample data preprocessing script template and a model training script template in the fourth step include: writing corresponding script templates according to the sample data acquisition method, preprocessing method and training method, and testing and optimizing the script templates to ensure the correctness and efficiency of the script templates.

8. The abnormal information processing method based on knowledge graph and large model training according to claim 1 is characterized in that: The steps of executing the sample data acquisition script template, the sample data preprocessing script template and the model training script template in the fifth step include: according to the preset training cycle and training time parameter values, the sample data acquisition script template, the sample data preprocessing script template and the model training script template are executed regularly, and the training sample data and model parameter information are continuously updated to improve the accuracy and adaptability of the model.