A large model-based log anomaly analysis system and method

Abstract

The application discloses a log abnormality analysis system and method based on a large model, relates to the technical field of log abnormality analysis, and comprises the following steps: collecting log data of a transaction system and a security system in a chain store operation system, and pre-processing the collected data; inputting the collected historical log data into a large model to train the model; analyzing whether the log data collected in real time is abnormal, and analyzing abnormal log data; prompting abnormal conditions detected according to generated instructions, and optimizing the chain store operation system; the application analyzes logs by introducing a large model, introduces a system abnormality risk value and a correlation degree between a transaction system and a security system, solves the problem that the probability of a chain reaction between security abnormality and transaction interruption caused by a high correlation degree between two systems is high, and avoids the problem that a security blind area is caused by excessive isolation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of log anomaly analysis, and particularly relates to a log anomaly analysis system and method based on a large model. BACKGROUND

[0002] In the process of chain store operation and network management, various devices and application programs will generate a large amount of log data, which records various state information of system operation; it is very important to detect and identify abnormal conditions from log data in time and accurately, and to optimize and adjust the system based on the detected abnormal data, for guaranteeing stable operation of the system and improving system security; the traditional log anomaly analysis method usually relies on pre-set rules or simple monitoring and analysis models, and cannot adapt to complex and variable log data and abnormal conditions; with the expansion of chain store business, network management requirements become increasingly complex, resulting in continuous increase in the scale and types of log data, and the traditional method has problems such as low accuracy of abnormality detection, inability to process complex semantic information, etc. when processing large-scale and unstructured log data, and it is difficult to meet the efficiency and reliability requirements of log processing in actual application; with the development of large model technology, the powerful natural language processing and data analysis capabilities of large model provide a new way for log anomaly analysis; however, the current technology lacks evaluation technology for the overall system abnormal risk value, cannot achieve comprehensive system detection according to the overall state of the system, and in the environment of chain stores with high intelligence, the correlation between the two systems may be higher, and special attention should be paid to prevent the chain reaction of "security anomaly-transaction interruption", and also to avoid security blind spots caused by excessive isolation, so the current technology still has problems to be solved in terms of detection of mutual influence of abnormal conditions generated between the two systems and control of the correlation between the two systems. SUMMARY

[0003] The present application aims to provide a log anomaly analysis system and method based on a large model to solve the problems in the prior art.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a log anomaly analysis method based on a large model, the method comprising the following steps:

[0005] S1, collecting log data of a transaction system and a security system in a chain store operation system, and pre-processing the collected data;

[0006] S2, inputting the collected historical log data into a large model to train the model;

[0007] S3, using the large model trained and fine-tuned to analyze the log data collected in real time, analyzing whether the system has an abnormality according to the log data collected in real time, and analyzing the abnormal log data;

[0008] S4, prompting the detected abnormal situation according to the generated instruction, and optimizing the chain store operation system.

[0009] Further, in step S1: a training data collection mechanism is set, log data is collected every time interval Δt1, n pieces of log data are collected for model training, denoted as model training data; a real-time detection data collection mechanism is set, log data is collected every time interval Δt2, denoted as real-time log data, which is used for real-time analysis of log data anomalies; the collected model training data and real-time log data are preprocessed, the collected log data is cleaned and denoised through regular expressions; the repeated data in the collected log data is identified and removed through the time window deduplication method.

[0010] Further, in step S2: the collected and preprocessed model training data is input into the large model based on the Transformer architecture to pretrain the large model, and the large model learns the patterns and rules of normal log data through supervised learning using the collected training data; the understanding and processing capability of the large model for the log in the transaction system and the security system in the chain store is enhanced.

[0011] Further, in step S3: the large model after training is used to extract and analyze the causes of abnormal logs, the set of transaction system abnormalities in the transaction system that cause abnormal logs to appear is {a1, a2, …, ap}; the set of security system abnormalities in the security system that cause abnormal logs to appear is {b1, b2, …, bq}; in the abnormal log analysis process, the abnormal logs are divided into single system abnormal logs and associated system abnormal logs according to the causes of the abnormal logs; if the cause of the abnormal log in the transaction system is the transaction system abnormality a p , where i=1, 2, …p, this abnormal log is judged as a single system abnormal log D1 in the transaction system; if the cause of the abnormal log in the security system is the security system abnormality b q , where j=1, 2, …q, this abnormal log is judged as a single system abnormal log D2 in the security system; if the cause of the abnormal log in the transaction system is the security system abnormality b i , or the security system abnormality b j and the transaction system abnormality a j , this abnormal log is judged as an associated system abnormal log D3 in the transaction system; if the cause of the abnormal log in the security system is the transaction system abnormality a j , or the transaction system abnormality a i and the security system abnormality b i , this abnormal log is judged as an associated system abnormal log D4 in the security system. i j ​If the abnormal log is the single system abnormal log D1 in the transaction system, the abnormal log is determined as the transaction system abnormal log D1;

[0012] After detecting the abnormal log in real time through the large model, instructions are generated for different abnormal logs to optimize the running system, and the detection and optimization results are as follows:

[0013] If the single system abnormal log D1 of the transaction system is detected, the transaction system abnormal correction instruction x1 is generated;

[0014] If the single system abnormal log D2 of the security system is detected, the security system abnormal correction instruction x2 is generated;

[0015] If the correlation system abnormal log D3 in the transaction system is detected, the joint abnormal correction instruction x3 for D3 is generated;

[0016] If the correlation system abnormal log D4 in the security system is detected, the joint abnormal correction instruction x4 for D4 is generated;

[0017] At the same time, the abnormal logs of the transaction system and the security system in the chain store are identified and detected through the large model after training, and the number of detected abnormal logs is recorded every time interval Δt2. The number of detected abnormal logs in the transaction system is denoted as n1, and the number of detected abnormal logs in the security system is denoted as n2. After recording the number of abnormal logs, whether the operation system has an abnormal risk is analyzed through the abnormal log condition, the operation system abnormal risk threshold W0 is set, and the operation system abnormal risk value W is analyzed according to the number of detected abnormal logs:

[0018]

[0019] Wherein, W represents the operation system abnormal risk value, represents the influence weight of the number of abnormal logs in the transaction system on the system risk value; represents the influence weight of the number of abnormal logs in the security system on the system risk value; n0 represents the set abnormal log number standard value; represents the influence weight of the correlation degree between the security system and the transaction system on the system risk value; R represents the correlation degree between the security system and the transaction system;

[0020] After analyzing the operation system abnormal risk value, the obtained operation system abnormal risk value is compared with the set operation system abnormal risk threshold, and the comparison analysis result is as follows:

[0021] If W≤W0, it is judged that the operation system abnormal risk value is normal, and no optimization prompt instruction needs to be generated;

[0022] ​If W > W0, it is judged that the abnormal risk value of the operation system is too high, and the operation system of the chain store is controlled to enter a high abnormal risk mode; through the calculation and comparative analysis of the operation system risk value, the high risk state of the operation system can be found in time;

[0023] After the operation system of the chain store enters the high abnormal risk mode, the detected abnormal logs are further analyzed, the number of D3 is recorded as n3, and the number of D4 is recorded as n4; then the correlation degree between the security system and the transaction system is calculated according to the following formula:

[0024] ;

[0025] Among them, represents the influence weight of the number of correlation system abnormal logs in the transaction system on the system correlation degree; represents the influence weight of the number of correlation system abnormal logs in the security system on the system correlation degree;

[0026] The system correlation degree threshold range [R min , R max ] is set, the system correlation degree R calculated by detection is compared with the set system correlation degree threshold R min and R max , and the analysis result of the correlation degree between the security system and the transaction system is used to judge the reason for the high abnormal risk value of the operation system of the chain store; the analysis result of the correlation degree between the security system and the transaction system is as follows:

[0027] If R min ≤ R ≤ R max , it means that the correlation degree between the transaction system and the security system is normal, and the reason for the high abnormal risk value of the operation system of the chain store is that the number of single system abnormal logs is high;

[0028] If R > R max , it means that the correlation degree between the transaction system and the security system is abnormally high, and the reason for the high abnormal risk value of the operation system of the chain store is that the number of correlation system abnormal logs is high;

[0029] If R < R min , it means that the correlation degree between the transaction system and the security system is abnormally low, and the reason for the high abnormal risk value of the operation system of the chain store is that the number of correlation system abnormal logs is high.

[0030] Further, in step S3: in the system anomaly correction process, a single system anomaly log cause correction channel M1 and an associated system anomaly log cause correction channel M2 are set respectively, when the chain store operation system does not enter a high abnormal risk mode, the correction power of channel M1 is P1, the correction power of channel M2 is P2, and P1=P2=1 / 2P0, P0 represents the total system anomaly correction power; after the chain store operation system enters the high abnormal risk mode, the correction power of channel M1 and channel M2 is adjusted according to the system correlation, so as to improve the system anomaly correction efficiency; the correction power of channel M1 after adjustment is represented as , and the correction power of channel M2 is represented as , and the adjustment is divided into the following two cases:

[0031] If R min ≤R≤R max , the correction power of channel M1 is adjusted to be higher, and the adjustment result is: ; and the correction power of channel M2 is adjusted to be lower: ;

[0032] If R>R max or R<R min , the correction power of channel M2 is adjusted to be higher, and the adjustment result is: ; and the correction power of channel M1 is adjusted to be lower: ;

[0033] After the correction power adjustment, after the abnormal correction for a time length Δt3, the operation system abnormal risk value is detected and analyzed again: if the analysis result is W≤W0, the high abnormal risk mode of the chain store operation system is removed; if the analysis result is W>W0, it is judged that the automatic correction of the chain store operation system is abnormal, and an instruction y1 is generated to prompt the staff to perform manual detection on the chain store operation system.

[0034] According to the analysis of the correlation between the transaction system and the security system, the risk state that the correlation between the two systems is too high or too low can be monitored in time and warned and controlled in time, the correlation between the transaction system and the security system is controlled within the set threshold range, the problem that the probability of the chain reaction between security anomalies and transaction interruptions is high due to high coupling degree of the two systems in the store with high intelligent degree is solved, and the problem of security blind area caused by excessive isolation is also avoided.

[0035] Further, in step S4: the anomaly in the transaction system and the security system in the chain store operation system is corrected according to the generated instruction;

[0036] If a transaction system anomaly correction instruction x1 is generated, the transaction system anomaly ai is corrected;

[0037] If the security system anomaly correction instruction x2 is generated, the security system anomaly b corresponding to D2 in the security system is corrected j or corrected;

[0038] If the joint anomaly correction instruction x3 for D3 is generated, the security system anomaly b corresponding to D3 in the security system and the transaction system is corrected j or b j and a i is corrected;

[0039] If the joint anomaly correction instruction x4 for D4 is generated, the transaction system anomaly a corresponding to D4 in the transaction system and the security system is corrected i or a i and b j is corrected;

[0040] If the instruction y1 is generated, the visual device is prompted for the abnormality of the automatic correction of the chain store operation system: the staff is prompted to adjust the operation system architecture to comprehensively detect the chain store operation system.

[0041] A log anomaly analysis system based on a large model, the system comprising: a log collection and preprocessing module, a model training module, an abnormal log detection and analysis module, and a warning and optimization adjustment module;

[0042] The log collection and preprocessing module is used to collect log data of the transaction system and the security system in the chain store operation system, and to preprocess the collected data;

[0043] The model training module is used to input the collected historical log data into the large model to train the model;

[0044] The abnormal log detection and analysis module uses the large model trained and fine-tuned to analyze the real-time collected log data, analyzes whether the system has an abnormality according to the real-time collected log data, and analyzes the abnormal log data;

[0045] The warning and optimization adjustment module corrects the causes of different abnormal logs according to the generated instructions, and prompts the abnormality of the automatic correction of the chain store operation system in the visual device according to the generated instructions.

[0046] Further, the log collection and preprocessing module comprises a training data collection unit, a real-time detection data collection unit and a preprocessing unit; the training data collection unit is used to collect log data for large model training and fine-tuning; the real-time detection data collection unit is used to collect log data for real-time detection; the preprocessing unit cleans and denoises the collected log data, and removes repeated data in the log data.

[0047] Further, the model training module inputs the collected training data into the large model, and uses the collected training data to let the large model learn the mode and rule of normal log data in a supervised learning manner.

[0048] Further, the abnormal log detection and analysis module comprises an abnormal log cause analysis unit, a system abnormal risk analysis unit, a system correlation degree analysis unit and a correction power adjustment unit; the abnormal log cause analysis unit analyzes the cause and type of the detected abnormal log according to the large model, and generates different optimization instructions for different types of abnormal logs; the system abnormal risk analysis unit analyzes the system abnormal risk according to the monitored abnormal log data, sets a threshold for comparative analysis, and thus evaluates the abnormal risk degree of the system; the system correlation degree analysis unit analyzes the system correlation degree between the transaction system and the security system through the related system abnormal log data in the abnormal log, sets a threshold for comparative analysis, and thus evaluates the correlation degree between the two systems; the correction power adjustment unit adjusts the system abnormal correction power according to the analysis result of the correlation degree between the transaction system and the security system.

[0049] Compared with the prior art, the present application has the following beneficial effects:

[0050] The present application analyzes the logs generated by the transaction system and the security system of the operation system in the chain store by introducing a large model, and improves the understanding and processing capacity of the large model for the logs in the transaction system and the security system in the chain store after training the large model with the collected training data. The present application evaluates the abnormal degree of the operation system in the chain store by introducing a system abnormal risk value and detecting abnormal log data, and can discover the high-risk state of the system in time according to the comparison analysis result of the system abnormal risk value and the set threshold value, and control the chain store operation system to trigger a high abnormal risk mode. In the high risk mode state, the type of abnormal log and the correlation degree between the transaction system and the security system are further analyzed, the correlation degree between the transaction system and the security system is analyzed through the detected abnormal log type and quantity, the abnormal log type triggering the high abnormal risk mode is judged through the analysis of the correlation degree between the transaction system and the security system, and then the system abnormal correction power is adjusted according to the obtained correlation degree between the transaction system and the security system, so that the abnormal log causing the system to enter the high risk mode state is corrected with high power, thereby improving the system abnormal correction and optimization efficiency. After efficient system abnormal correction, the correlation degree between the transaction system and the security system can be controlled within the set threshold value range, solving the problem of high probability of chain reaction between security abnormalities and transaction interruptions caused by high correlation between the two systems in the store with high intelligent degree, and avoiding the problem of security blind area caused by excessive isolation. BRIEF DESCRIPTION OF DRAWINGS

[0051] Fig. 1 A method flow diagram of a log abnormality analysis method based on a large model according to the present application;

[0052] Fig. 2 A structure diagram of a log abnormality analysis system based on a large model according to the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0054] As shown in Figs. 1-2 , the present application provides a technical solution, a log abnormality analysis method based on a large model, as shown in Fig. 1 , the method comprises the following steps:

[0055] S1, collect log data of a transaction system and a security system in a chain store operation system, and pre-process the collected data;

[0056] S2, input the collected historical log data into a large model, and train the model;

[0057] S3, analyze real-time collected log data by using the large model after training and fine-tuning, analyze whether the system has an abnormality according to the real-time collected log data, and analyze abnormal log data;

[0058] S4, prompt the detected abnormal situation according to the generated instructions, and optimize the chain store operation system.

[0059] In step S1: set a training data collection mechanism, collect log data every time length Δt1, collect n log data for model training, denoted as model training data; set a real-time detection data collection mechanism, collect log data every time length Δt2, denoted as real-time log data, for real-time analysis of log data abnormalities; pre-process the collected model training data and real-time log data, clean and denoise the collected log data through regular expressions; identify and remove duplicate data in the collected log data through a time window deduplication method.

[0060] In step S2: input the collected and pre-processed model training data into a large model based on the Transformer architecture to pre-train the large model, and use the collected training data to let the large model learn the pattern and rule of normal log data through supervised learning.

[0061] In step S3: use the trained large model to extract and analyze the causes of abnormal logs, the set of transaction system abnormalities that cause abnormal logs in the transaction system is {a1, a2, …, ap}; the set of security system abnormalities that cause abnormal logs in the security system is {b1, b2, …, bq}; in the abnormal log analysis process, the abnormal logs are divided into single system abnormal logs and associated system abnormal logs according to the causes of abnormal logs; if the cause of the abnormal log in the transaction system is the transaction system abnormality ai, where i=1, 2, …p, this abnormal log is judged as a single system abnormal log D1 in the transaction system; if the cause of the abnormal log in the security system is the security system abnormality bj, where j=1, 2, …q, this abnormal log is judged as a single system abnormal log D2 in the security system; if the cause of the abnormal log in the transaction system is the security system abnormality bj, or the cause of the abnormal log in the security system is the transaction system abnormality ai, the abnormal log is judged as an associated system abnormal log D3. p q i j j j ​​​​​and transaction system abnormality a i If the abnormality log is caused by the transaction system abnormality a i or the transaction system abnormality a i and security system abnormality b j , the abnormality log is determined as the associated system abnormality log D4 in the security system.

[0062] After detecting the abnormality log in real time through the large model, instructions are generated to optimize the running system according to different abnormality logs, and the detection and optimization results are as follows:

[0063] If the single system abnormality log D1 of the transaction system is detected, the transaction system abnormality correction instruction x1 is generated.

[0064] If the single system abnormality log D2 of the security system is detected, the security system abnormality correction instruction x2 is generated.

[0065] If the associated system abnormality log D3 in the transaction system is detected, the joint abnormality correction instruction x3 for D3 is generated.

[0066] If the associated system abnormality log D4 in the security system is detected, the joint abnormality correction instruction x4 for D4 is generated.

[0067] At the same time, the abnormality logs of the transaction system and the security system in the chain store are identified and detected through the large model after training, and the number of detected abnormality logs is recorded every time interval Δt2. The number of detected abnormality logs in the transaction system is denoted as n1, and the number of detected abnormality logs in the security system is denoted as n2. After recording the number of abnormality logs, whether the operation system has an abnormal risk is analyzed through the abnormality log condition, the operation system abnormal risk threshold value is set as W0, and the operation system abnormal risk value W is analyzed according to the number of detected abnormality logs.

[0068] ;

[0069] Wherein, represents the influence weight of the number of abnormality logs in the transaction system on the system risk value; represents the influence weight of the number of abnormality logs in the security system on the system risk value; n0 represents the set abnormality log number standard value; represents the influence weight of the correlation degree between the security system and the transaction system on the system risk value; R represents the correlation degree between the security system and the transaction system.

[0070] After analyzing the operation system abnormal risk value, the obtained operation system abnormal risk value is compared with the set operation system abnormal risk threshold value, and the comparison analysis result is as follows:

[0071] If W≤W0, it is judged that the system abnormal risk value is normal, and no optimization prompt instruction needs to be generated;

[0072] If W>W0, it is judged that the system abnormal risk value is too high, and the chain store operation system is controlled to enter a high abnormal risk mode;

[0073] After the chain store operation system enters the high abnormal risk mode, the detected abnormal log is further analyzed, the number of D3 is recorded as n3, and the number of D4 is recorded as n4; then the correlation degree between the security system and the transaction system is calculated according to the following formula:

[0074]

[0075] Among them, represents the influence weight of the number of correlation system abnormal logs in the transaction system on the system correlation degree; represents the influence weight of the number of correlation system abnormal logs in the security system on the system correlation degree;

[0076] The system correlation degree threshold range [R min , R max ] is set, the detected system correlation degree R is compared with the set system correlation degree threshold R min and R max , and the correlation degree between the security system and the transaction system is analyzed according to the analysis result, the reason for causing the high abnormal risk value of the chain store operation system is judged; the correlation degree analysis result between the security system and the transaction system is as follows:

[0077] If R min ≤R≤R max , it indicates that the correlation degree between the transaction system and the security system is normal, and the reason for causing the high abnormal risk value of the chain store operation system is that the number of single system abnormal logs is high;

[0078] If R>R max , it indicates that the correlation degree between the transaction system and the security system is abnormally high, and the reason for causing the high abnormal risk value of the chain store operation system is that the number of correlation system abnormal logs is high;

[0079] If R<R min , it indicates that the correlation degree between the transaction system and the security system is abnormally low, and the reason for causing the high abnormal risk value of the chain store operation system is that the number of correlation system abnormal logs is high;

[0080] ​In step S3: in the system anomaly correction process, a single system anomaly log cause correction channel M1 and a related system anomaly log cause correction channel M2 are set respectively, when the chain store operation system does not enter the high abnormal risk mode, the correction power of channel M1 is P1, the correction power of channel M2 is P2, and P1=P2=1 / 2P0, P0 represents the total power of system anomaly correction; after the chain store operation system enters the high abnormal risk mode, the correction power of channel M1 and channel M2 is adjusted according to the system correlation, the correction power of channel M1 after adjustment is represented as , and the correction power of channel M2 after adjustment is represented as , which is adjusted as follows:

[0081] If R min ≤R≤R max , the correction power of channel M1 is adjusted to be higher, and the adjustment result is: ; and the correction power of channel M2 is adjusted to be lower: ;

[0082] If R>R max or R<R min , the correction power of channel M2 is adjusted to be higher, and the adjustment result is: ; and the correction power of channel M1 is adjusted to be lower: ;

[0083] After the correction power is adjusted, the operation system abnormal risk value is detected and analyzed again after the abnormal correction for a time length Δt3: if the analysis result is W≤W0, the high abnormal risk mode of the chain store operation system is removed; if the analysis result is W>W0, it is judged that the automatic correction of the chain store operation system is abnormal, and an instruction y1 is generated to prompt the staff to manually detect the chain store operation system.

[0084] In step S4: the anomaly in the transaction system and the security system in the chain store operation system is corrected according to the generated instruction;

[0085] If a transaction system anomaly correction instruction x1 is generated, the transaction system anomaly a i corresponding to D1 in the transaction system is corrected;

[0086] If a security system anomaly correction instruction x2 is generated, the system security anomaly b j corresponding to D2 in the security system is corrected or corrected;

[0087] If a joint anomaly correction instruction x3 for D3 is generated, the security system anomaly b j corresponding to D3 in the security system and the transaction system is corrected or b j and ai correction is made to a

[0088] If the joint abnormality correction instruction x4 for D4 is generated, the transaction system abnormality a corresponding to D4 in the transaction system and the security system is corrected i or a i and b j correction is made to a

[0089] If the instruction y1 is generated, the visual device is prompted that the automatic correction of the chain store operation system is abnormal, prompting the staff to adjust the operation system architecture to comprehensively detect the chain store operation system.

[0090] A log abnormality analysis system based on a large model, as shown in Fig. 2 The system comprises a log collection and preprocessing module, a model training module, an abnormal log detection and analysis module, and a warning and optimization adjustment module.

[0091] The log collection and preprocessing module is used to collect log data of the transaction system and the security system in the chain store operation system, and to preprocess the collected data.

[0092] The model training module is used to input the collected historical log data into the large model to train the model.

[0093] The abnormal log detection and analysis module analyzes the real-time collected log data by using the large model after training and fine-tuning, analyzes whether the system has an abnormality according to the real-time collected log data, and analyzes the abnormal log data.

[0094] The warning and optimization adjustment module corrects the causes of different abnormal logs according to the generated instructions, and prompts the visual device that the automatic correction of the chain store operation system is abnormal according to the generated instructions.

[0095] The log collection and preprocessing module comprises a training data collection unit, a real-time detection data collection unit, and a preprocessing unit; the training data collection unit is used to collect log data for training and fine-tuning of the large model; the real-time detection data collection unit is used to collect log data for real-time detection; and the preprocessing unit cleans and denoises the collected log data, and removes repeated data in the log data.

[0096] The model training module inputs the collected training data into the large model, and learns the mode and rule of normal log data by using the collected training data in a supervised learning manner.

[0097] The abnormal log detection and analysis module comprises an abnormal log cause analysis unit, a system abnormal risk analysis unit, a system correlation degree analysis unit and a correction power adjustment unit; the abnormal log cause analysis unit analyzes the cause and type of the detected abnormal log according to the large model, and generates different optimization instructions for different types of abnormal logs; the system abnormal risk analysis unit analyzes the system abnormal risk according to the monitored abnormal log data, and sets a threshold for comparative analysis, so as to evaluate the abnormal risk degree of the system; the system correlation degree analysis unit analyzes the system correlation degree between the transaction system and the security system through the related system abnormal log data in the abnormal log, and sets a threshold for comparative analysis, so as to evaluate the correlation degree between the two systems; the correction power adjustment unit adjusts the system abnormal correction power according to the analysis result of the correlation degree between the transaction system and the security system.

[0098] In step S1, a training data collection mechanism is set, log data is collected every time interval Δt1=1h, n=500 pieces of log data are collected for model training, denoted as model training data; a real-time detection data collection mechanism is set, log data is collected every time interval Δt2=2h, denoted as real-time log data, which is used for real-time analysis of log data abnormalities; the collected model training data and real-time log data are preprocessed, the collected log data is cleaned and denoised through regular expressions; the repeated data in the collected log data is identified and removed through the time window deduplication method.

[0099] In step S2, the model training data collected and preprocessed are input into the large model based on the Transformer architecture to pretrain the large model, and the large model learns the mode and rule of normal log data through supervised learning using the collected training data.

[0100] In step S3, the large model after training is used to extract and analyze the cause of the abnormal log, the set of transaction system abnormalities in the transaction system that cause the abnormal log to appear is {a1, a2, …, ap}; the set of security system abnormalities in the security system that cause the abnormal log to appear is {b1, b2, …, bq}; in the abnormal log analysis process, the abnormal log is divided into single system abnormal log and related system abnormal log according to the cause of the abnormal log; if the cause of the abnormal log in the transaction system is detected as the transaction system abnormality ai, i=1, 2, …p, the abnormal log is judged as the single system abnormal log D1 in the transaction system; if the cause of the abnormal log in the security system is detected as the security system abnormality bi, i=1, 2, …q, the abnormal log is judged as the single system abnormal log D2 in the security system. p q i j ​​​, j = 1, 2, … q, then this abnormal log is judged as a single system abnormal log D2 in the security system; if the cause of the abnormal log in the transaction system is detected as security system abnormality b j or security system abnormality b j and transaction system abnormality a i , then this abnormal log is judged as a related system abnormal log D3 in the transaction system; if the cause of the abnormal log in the security system is detected as transaction system abnormality a i or transaction system abnormality a i and security system abnormality b j , then this abnormal log is judged as a related system abnormal log D4 in the security system;

[0101] After detecting the abnormal log in real time through the large model, instructions are generated for different abnormal logs to optimize the running system, and the detection and optimization results are as follows:

[0102] If a single system abnormal log D1 of the transaction system is detected, a transaction system abnormal correction instruction x1 is generated;

[0103] If a single system abnormal log D2 of the security system is detected, a security system abnormal correction instruction x2 is generated;

[0104] If a related system abnormal log D3 in the transaction system is detected, a joint abnormal correction instruction x3 for D3 is generated;

[0105] If a related system abnormal log D4 in the security system is detected, a joint abnormal correction instruction x4 for D4 is generated;

[0106] At the same time, through the large model trained and fine-tuned, the abnormal logs of the transaction system and the security system in the chain store are identified and detected, and the number of detected abnormal logs is recorded every time interval Δt2 = 2h, the number of detected abnormal logs in the transaction system is recorded as n1 = 5, and the number of detected abnormal logs in the security system is recorded as n2 = 8; After recording the number of abnormal logs, whether the operation system has an abnormal risk is analyzed through the abnormal log situation, the operation system abnormal risk threshold is set as W0 = 0.6, and the operation system abnormal risk value W is analyzed according to the number of detected abnormal logs:

[0107] ;

[0108] wherein W represents the operation system abnormal risk value, = 0.3 represents the influence weight of the number of abnormal logs in the transaction system on the system risk value; = 0.3 represents the influence weight of the number of abnormal logs in the security system on the system risk value; n0 = 10 represents the set abnormal log number standard value; = 0.4 represents the influence weight of the correlation degree between the security system and the transaction system on the system risk value; R represents the correlation degree between the security system and the transaction system; W = 0.478 is calculated;

[0109] After analyzing the operation system abnormal risk value, the operation system abnormal risk value obtained is compared with the set operation system abnormal risk threshold value, and the comparison analysis result is as follows:

[0110] According to the calculation result, W≤W0, it is judged that the system abnormal risk value is normal, and no early warning instruction needs to be generated;

[0111] After the cause of the abnormal log is extracted and analyzed by the large model, the number of D3 is recorded as n3=1, and the number of D4 is recorded as n4=2. The correlation degree between the security system and the transaction system is calculated according to the following formula:

[0112]

[0113] Among them, = 0.6 represents the influence weight of the number of correlation system abnormal logs in the transaction system on the system correlation degree; = 0.4 represents the influence weight of the number of correlation system abnormal logs in the security system on the system correlation degree; R = 0.22 is calculated;

[0114] The system correlation degree threshold value range [R min , R max ] is set, that is, [0.2, 0.6]. The detected and calculated system correlation degree R is compared with the set system correlation degree threshold value R min and R max , and the analysis result is as follows:

[0115] According to the calculation result, R min ≤R≤R max , it is judged that the correlation degree between the transaction system and the security system is normal.

[0116] In step S4: according to the generated instruction, the abnormality in the transaction system and the security system in the chain store operation system is corrected;

[0117] If the transaction system abnormal correction instruction x1 is generated, the transaction system abnormality a i corresponding to D1 in the transaction system is corrected;

[0118] If the security system abnormal correction instruction x2 is generated, the system security abnormality b j corresponding to D2 in the security system is corrected;

[0119] ​If a joint abnormality correction instruction x3 for D3 is generated, the security system abnormality b corresponding to D3 in the security system and the transaction system is corrected j or the b j and the a i are corrected.

[0120] If a joint abnormality correction instruction x4 for D4 is generated, the transaction system abnormality a corresponding to D4 in the transaction system and the security system is corrected i or the a i and the b j are corrected.

[0121] It will be obvious to those skilled in the art that the application is not limited to the details of the foregoing exemplary embodiments and can be practiced with modification and alteration within the scope of the claims. The application described is not limited to the exact details shown and described, for changes can be made to the details within the scope of the application. Accordingly, no matter how many points of view are taken, the embodiments are to be considered illustrative and not restrictive, the scope of the application being defined by the appended claims rather than by the foregoing description, and all changes coming within the meaning and range of equivalency of the claims are to be embraced within the application. No reference signs in the claims should be considered to be a limitation on the scope of the claims.

Claims

1. A large model-based log anomaly analysis method, characterized in that: The method comprises the following steps: S1, collecting log data of a transaction system and a security system in a chain store operation system, and preprocessing the collected data; S2, inputting the collected historical log data into a large model to train the model; S3, analyzing the log data collected in real time by using the large model after training and fine-tuning, analyzing whether the system is abnormal according to the log data collected in real time, and analyzing the abnormal log data; S4, prompting the detected abnormal conditions according to the generated instructions, and optimizing the chain store operation system; In step S3: after detecting abnormal logs in real time by the large model, instructions are generated for different abnormal logs to optimize the running system, and the detection and optimization results are as follows: If a single system abnormal log D1 of the transaction system is detected, a transaction system abnormal correction instruction x1 is generated; If a single system abnormal log D2 of the security system is detected, a security system abnormal correction instruction x2 is generated; If a related system abnormal log D3 in the transaction system is detected, a joint abnormal correction instruction x3 for D3 is generated; If a related system abnormal log D4 in the security system is detected, a joint abnormal correction instruction x4 for D4 is generated; At the same time, the large model after training is used to identify and detect the abnormal logs of the transaction system and the security system in the chain store, and the number of detected abnormal logs is recorded once every time interval Δt2, the number of detected abnormal logs in the transaction system is recorded as n1, and the number of detected abnormal logs in the security system is recorded as n2; after recording the number of abnormal logs, whether the operation system has an abnormal risk is analyzed through the abnormal log condition, the operation system abnormal risk threshold W0 is set, and the operation system abnormal risk value W is analyzed according to the number of detected abnormal logs: ； wherein, represents the influence weight of the number of abnormal logs in the transaction system on the system risk value; represents the influence weight of the number of abnormal logs in the security system on the system risk value; n0represents the set standard value of the number of abnormal logs; represents the influence weight of the correlation degree between the security system and the transaction system on the system risk value; R represents the correlation degree between the security system and the transaction system. After the chain store operation system enters a high abnormal risk mode, the detected abnormal logs are further analyzed, the number of D3 is recorded as n3, and the number of D4 is recorded as n4; D3 represents a related system abnormal log in the transaction system; D4 represents a related system abnormal log in the security system; then the correlation degree between the security system and the transaction system is calculated according to the following formula: ； wherein, represents the influence weight of the number of abnormal logs of the associated system on the system correlation degree in the transaction system; represents the influence weight of the number of abnormal logs of the associated system on the system correlation degree in the security system.

2. The log anomaly analysis method based on a large model according to claim 1, characterized in that: In step S1: set a training data collection mechanism, collect log data once every time interval Δt1, collect n pieces of log data for model training, and record them as model training data; set a real-time detection data collection mechanism, collect log data once every time interval Δt2, and record them as real-time log data for real-time analysis of log data abnormalities; The collected model training data and real-time log data are preprocessed, and the collected log data is cleaned and denoised by using a regular expression; Repeated data in the collected log data is identified and removed by using a time window deduplication method.

3. The log anomaly analysis method based on a large model according to claim 1, characterized in that: In step S2: input the collected and preprocessed model training data into a large model based on a Transformer architecture to pretrain the large model, and use the collected training data to let the large model learn the patterns and rules of normal log data through supervised learning.

4. The log anomaly analysis method based on a large model according to claim 1, characterized in that: In step S3: the causes of the abnormal logs are extracted and analyzed by using the trained large model, the set of transaction system abnormalities in the transaction system that cause the abnormal logs to appear is {a1, a2, …, ap}; the set of security system abnormalities in the security system that cause the abnormal logs to appear is {b1, b2, …, bq}; in the abnormal log analysis process, the abnormal logs are divided into single system abnormal logs and associated system abnormal logs according to the causes of the abnormal logs; if the cause of the abnormal log in the transaction system is the transaction system abnormality ai, where i=1, 2, …p, the abnormal log is judged as the single system abnormal log D1 in the transaction system; if the cause of the abnormal log in the security system is the security system abnormality bj, where j=1, 2, …q, the abnormal log is judged as the single system abnormal log D2 in the security system; if the cause of the abnormal log in the transaction system is the security system abnormality bj, or the cause of the abnormal log in the security system is the transaction system abnormality ai, the abnormal log is judged as the associated system abnormal log D3 in the transaction system; if the cause of the abnormal log in the security system is the transaction system abnormality ai, or the cause of the abnormal log in the transaction system is the security system abnormality bj, the abnormal log is judged as the associated system abnormal log D4 in the security system. p q i j j j i i i j ​​​​​​​​​​ After analyzing the operation system abnormal risk value, the operation system abnormal risk value obtained is compared with the set operation system abnormal risk threshold value, and the comparison analysis result is as follows: If W≤W0, it is judged that the system abnormal risk value is normal, and no optimization prompt instruction needs to be generated; If W>W0, it is judged that the system abnormal risk value is too high, and the control chain store operation system enters a high abnormal risk mode; A system correlation threshold range [R min , R max ] is set, the calculated system correlation R is compared with the set system correlation thresholds R min and R max , and the analysis result of the correlation between the security system and the transaction system is used to determine the reason for the high risk value of the abnormal operation system of the chain store; the analysis result of the correlation between the security system and the transaction system is as follows: If R min ≤ R ≤ R max , it indicates that the correlation degree between the transaction system and the security system is normal, and it is judged that the reason for the high risk value of the chain store operation system abnormality is the high number of single system abnormal logs. If R > R max or R < R min , it indicates that the correlation degree between the transaction system and the security system is abnormally high or abnormally low, and it is judged that the reason for the high risk value of the operation system of the chain store is the high number of correlation system abnormal logs.

5. The log anomaly analysis method based on a large model according to claim 4, characterized in that: In step S3: During the system abnormal correction process, a single system abnormal log cause correction channel M1 and a related system abnormal log cause correction channel M2 are set respectively, when the chain store operation system does not enter the high abnormal risk mode, the correction power of channel M1 is P1, the correction power of channel M2 is P2, and P1=P2=1 / 2P0, P0 represents the total system abnormal correction power; After the chain store operation system enters a highly abnormal risk mode, the correction power of the channel M1 and the channel M2 is adjusted according to the system correlation degree, and the correction power of the channel M1 after adjustment is represented as , and the correction power of the channel M2 after adjustment is represented as , and the adjustment is divided into the following two cases: If R min ≤ R ≤ R max , the modified power of channel M1 is adjusted to be: ; and the modified power of channel M2 is adjusted to be: ; If R > R max or R < R min , the modified power of channel M2 is adjusted to be higher, and the result is: ; and the modified power of channel M1 is adjusted to be lower: ; After the correction power is adjusted, after abnormal correction for a time length Δt3, the operation system abnormal risk value is detected and analyzed again: if the analysis result is W≤W0, the high abnormal risk mode of the chain store operation system is removed; If the analysis result is W>W0, it is judged that the automatic correction of the operation system of the chain store operation system is abnormal, and an instruction y1 is generated to prompt the staff to perform manual detection on the chain store operation system.

6. The log anomaly analysis method based on a large model according to claim 4, characterized in that: In step S4: the generated instruction is used to correct the abnormality in the transaction system and the security system in the chain store operation system; If the transaction system abnormality correction instruction x1 is generated, the transaction system abnormality a corresponding to D1 in the transaction system is corrected i is corrected; If the security system abnormal correction instruction x2 is generated, the system security abnormality b corresponding to the D2 in the security system is corrected j or corrected; If the joint abnormality correction instruction x3 for D3 is generated, the security system abnormality b corresponding to D3 in the security system and the transaction system is corrected j or b j and a i is corrected; If a joint abnormal correction instruction x4 for D4 is generated, the transaction system abnormality a corresponding to D4 in the transaction system and the security system is corrected i or a i and b j are corrected; If the instruction y1 is generated, the automatic correction of the chain store operation system is prompted on the visual device: the staff is prompted to adjust the operation system architecture to comprehensively detect the chain store operation system.

7. A large model-based log anomaly analysis system applied to the large model-based log anomaly analysis method of claim 1, characterized in that: The system comprises a log collection and preprocessing module, a model training module, an abnormal log detection and analysis module, and a warning and optimization adjustment module; The log collection and preprocessing module is used to collect log data of the transaction system and the security system in the chain store operation system, and to preprocess the collected data; The model training module is used to input the collected historical log data into a large model to train the model; The abnormal log detection and analysis module uses the trained and fine-tuned large model to analyze the real-time collected log data, analyzes whether the system has an abnormality according to the real-time collected log data, and analyzes the abnormal log data; The warning and optimization adjustment module corrects the causes of different abnormal logs according to the generated instruction, and prompts the automatic correction of the chain store operation system on the visual device according to the generated instruction.

8. The large model-based log anomaly analysis system of claim 7, wherein: The log collection and preprocessing module comprises a training data collection unit, a real-time detection data collection unit, and a preprocessing unit; The training data collection unit is used to collect log data for training and fine-tuning of the large model; The real-time detection data collection unit is used to collect log data for real-time detection; The preprocessing unit cleans and denoises the collected log data, and removes repeated data in the log data.

9. The large model-based log anomaly analysis system of claim 7, wherein: The model training module inputs the collected training data into the large model, and uses the collected training data to let the large model learn the mode and rules of normal log data in a supervised learning manner.

10. The large model-based log anomaly analysis system of claim 7, wherein: The abnormal log detection analysis module comprises an abnormal log cause analysis unit, a system abnormal risk analysis unit, a system correlation degree analysis unit and a correction power adjustment unit; the abnormal log cause analysis unit analyzes the cause and type of the detected abnormal log according to the large model, and generates different optimization instructions for different types of abnormal logs; The system abnormal risk analysis unit analyzes the system abnormal risk indicator according to the monitored abnormal log data, sets a threshold for comparative analysis, thereby evaluating the abnormal risk degree of the system; the system correlation degree analysis unit analyzes the system correlation degree between the transaction system and the security system through the related system abnormal log data in the abnormal log, sets a threshold for comparative analysis, thereby evaluating the correlation degree between the two systems; the correction power adjustment unit adjusts the system abnormal correction power according to the analysis result of the correlation degree between the transaction system and the security system.

Images