Method for Improving the Efficiency of AI Intelligent Analysis in Abnormal Detection of Tobacco Warehouse Business Data

Abstract

The present invention provides a method for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis, comprising the following steps: pre-configuring alarm parameters and rules; collecting system data; pushing the system data into the AI intelligent operation and maintenance system for machine learning, and obtaining alarm data through unsupervised learning; and pre-processing the system data before pushing the system data into the AI intelligent operation and maintenance system for machine learning. The pre-processing method comprises: dividing the stored system data indicators into two categories, the first category including count value and average response time, and the second category including success rate and number of errors; after the two categories of data are processed for quality using the first-order difference algorithm and the binomial distribution algorithm respectively, the process proceeds to step 2 for machine learning. This method can effectively reduce the impact of data loss caused by a large number of business interruptions on the accuracy of AI intelligent analysis results, and improve the accuracy of AI intelligent analysis in the efficiency of data anomaly detection in tobacco warehousing business.

Description

Technical Field

[0001] The present invention relates to the technical field of methods for detecting anomaly in tobacco warehousing business data, and in particular to a method for improving the efficiency of AI intelligent analysis in detecting anomaly in tobacco warehousing business data. Background Art

[0002] Responding to the national call for digital transformation of enterprises, tobacco companies are accelerating their development in new technologies such as cloud computing, big data, and artificial intelligence to promote digital industrialization and industrial digital transformation. As companies increasingly embrace internet technology, they are increasingly introducing distributed systems, new technologies, and new components. This has led to the continuous expansion of IT architectures, exponential growth in data, and increasing complexity in operations and maintenance. The use of artificial intelligence (AI) to analyze massive amounts of data and implement intelligent anomaly detection and fault location for monitoring indicators is currently a common technique, with numerous successful experiences in financial and internet companies. However, its implementation in tobacco warehousing has been less effective. This is because the majority of tobacco warehousing data indicators are small datasets, making machine learning ineffective. Improving the efficiency of AI-powered intelligent analysis for data anomaly detection in this area without changing the data characteristics of tobacco warehousing remains a challenge for tobacco operations and maintenance personnel.

[0003] At the same time, when applying AI intelligence to anomaly detection scenarios in tobacco warehousing business, we found that warehousing business data is scarce. During business processes such as raw material management and finished product management, a large number of pauses will occur, and data will also be lost during business pauses. Insufficient learning data will lead to a decrease in the accuracy of the anomaly detection model.

[0004] Among them, the intelligent operation and maintenance system of China Tobacco's new digital warehousing system introduces AI technology based on the traditional IT operation and maintenance monitoring system. It uses historical data through machine learning to train an algorithm model that can be used for anomaly detection and judgment. This AI technology is relatively mature and uses existing algorithms or productized algorithm combinations. The steps are: 1. Use monitoring and collection tools to collect system data; 2. Push the data into the intelligent operation and maintenance system for machine learning, and obtain alarm data through unsupervised learning; 3. Configure alarm rules in the platform settings. The configuration content includes: fixed thresholds, dynamic threshold tolerance, and the time and number of anomalies found in the indicator before the system issues an alarm. However, the alarms obtained by machine learning currently have the problem of false alarms, missed alarms, or false alarms caused by over-sensitivity of the system. However, AI technology has greatly improved compared with the fixed threshold technology of traditional alarms. Summary of the Invention

[0005] In order to solve the above technical problems, the first purpose of the present invention is to provide a method for improving the efficiency of AI intelligent analysis in detecting anomaly in tobacco warehousing business data. This method can effectively reduce the impact of data loss caused by a large number of business interruptions on the accuracy of AI intelligent analysis results, compress the number of false alarms generated by machine learning, and improve the accuracy of AI intelligent analysis in detecting anomaly in tobacco warehousing business data.

[0006] The second object of the present invention is to provide a system for implementing the above method to improve the efficiency of AI intelligent analysis in detecting anomaly in tobacco warehousing business data.

[0007] Based on the above objectives, one aspect of the present invention provides a method for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis, comprising the following steps:

[0008] Pre-configure alarm parameters and rules. The alarm parameters include fixed thresholds and dynamic threshold tolerances. The rules include the time required for each indicator to detect at least a certain number of abnormal points before the system issues an alarm.

[0009] Collect system data;

[0010] Push system data into the AI intelligent operation and maintenance system for machine learning, and obtain alarm data through unsupervised learning;

[0011] It also includes: before pushing the system data into the AI intelligent operation and maintenance system for machine learning, preprocessing the system data, the preprocessing method includes: dividing the stored system data indicators into two categories, the first category includes count value and average response time, and the second category includes success rate and number of errors; after the two categories of data are processed using the first-order difference algorithm and the binomial distribution algorithm respectively, enter step two for machine learning.

[0012] As a preference, after the time series processed by the binomial distribution algorithm is stabilized, a new time series data set is generated and then imported into an AI intelligent analysis system that is sensitive to feature period extraction to perform multi-indicator anomaly location detection.

[0013] Preferably, the pre-configuring of alarm parameters and rules on the platform includes summarizing and publishing two setting suggestions for alarm rules based on long-term observation and experimentation of business data.

[0014] As a preference, set the following alarm parameters and rules:

[0015] a. Separate monitoring alarm thresholds for count-type service indicator data based on peak and off-season data. At the same time, widen the baseband, increase tolerance, and reduce alarms. The upper baseband limit is only used to prevent external attacks. Alarm thresholds for traffic-type indicators can be combined with fixed thresholds for online baseband based on dynamic thresholds based on machine learning.

[0016] b. For indicator data of the average response time type, cancel the lower baseband alarm and make a comprehensive judgment based on the upper baseband alarm combined with the database SQL statement response indicator.

[0017] Preferably, the equation for the first-order difference processing is: Among them, the value of a variable at time t is recorded as y t , the values at time t and time t-1 can be described by a first-order linear difference equation.

[0018] Preferably, the binomial distribution formula is: Here, b represents the probability of the binomial distribution, n represents the number of trials, and x represents the number of times a certain outcome occurs.

[0019] Another aspect of the present invention provides a system for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis, the system being used to implement the above-mentioned method, and comprising:

[0020] A parameter storage unit is used to pre-configure alarm parameters and rules. The alarm parameters include fixed thresholds and dynamic threshold tolerances. The rules include the time and number of abnormal points found by each indicator for the system to issue an alarm.

[0021] Data acquisition unit, used for collecting system data;

[0022] A preprocessing unit preprocesses the system data before pushing it into the AI intelligent operation and maintenance system for machine learning. The preprocessing method includes: dividing the stored system data indicators into two categories, the first category including count value and average response time, and the second category including success rate and number of errors; after the two categories of data are processed using the first-order difference algorithm and the binomial distribution algorithm respectively, the machine learning is then carried out in step 2;

[0023] The AI intelligent operation and maintenance unit pushes the pre-processed system data into the AI intelligent operation and maintenance system for machine learning, and obtains alarm data through unsupervised learning.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention can effectively reduce the impact of data loss caused by a large number of business interruptions in tobacco warehousing operations (such as raw material management, finished product management, etc.) on the accuracy of AI intelligent analysis results, effectively reduce the number of false alarms generated by machine learning, and improve the accuracy of AI intelligent analysis in the efficiency of anomaly detection in tobacco warehousing business data. It enables AI intelligent analysis tools to be effectively implemented in the tobacco warehousing field, reduces the workload of operation and maintenance personnel, and effectively ensures the stable operation of tobacco warehousing business systems. In the era of deep business digitalization, it has reference and promotion significance for anomaly detection of business systems that generate massive data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings in the specification, which constitute a part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute a limitation on this application.

[0027] Figure 1 It is a schematic flow diagram of the present invention;

[0028] Figure 2 This is a warning diagram before the implementation of the present invention;

[0029] Figure 3 This is a diagram showing the alarm convergence effect after the implementation of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this embodiment have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs.

[0032] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0033] like Figure 1 As shown, this embodiment provides a method for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis, including the following steps:

[0034] Pre-configure alarm parameters and rules. The alarm parameters include fixed thresholds and dynamic threshold tolerances. The rules include the time required for each indicator to detect at least a certain number of abnormal points before the system issues an alarm.

[0035] Collect system data;

[0036] Push system data into the AI intelligent operation and maintenance system for machine learning, and obtain alarm data through unsupervised learning;

[0037] It also includes: before pushing the system data into the AI intelligent operation and maintenance system for machine learning, pre-processing the system data, the pre-processing method includes: dividing the stored system data indicators into two categories, the first category includes count value and average response time, and the second category includes success rate and number of errors; after the two types of data are processed using the first-order difference algorithm and the binomial distribution algorithm respectively, the machine learning is carried out in step 2. Specifically, during the processing, in the anomaly detection scenario of AI intelligent tobacco warehousing business, the count value and average response time type data of the key data indicators before the model detection processing are processed are processed using the first-order difference; after the time series of the success rate and error number type data appearing in the anomaly detection scenario is processed using the binomial distribution, it is stabilized, and a new time series data set is generated and then imported into the AI intelligent analysis system that is sensitive to feature period extraction for multi-indicator anomaly location detection; at the same time, based on long-term observation and experimentation of business data, two suggestions for setting alarm rules are summarized and released in advance to effectively improve the alarm accuracy of the new digital storage system.

[0038] In the actual production process, the above alarm parameters and rules can be shown in Table 1:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057] As a better implementation method, after processing with the binomial distribution algorithm, the time series is stabilized to generate a new time series data set, which is then imported into an AI intelligent analysis system that is sensitive to feature period extraction to perform multi-indicator anomaly location detection.

[0058] As a preferred implementation method, the pre-configuration of alarm parameters and rules on the platform includes summarizing and publishing two setting suggestions for alarm rules based on long-term observation and experimentation of business data.

[0059] As a preferred implementation method, in addition to data processing, the following specifications are made for alarm settings based on the characteristics of the business, which can effectively reduce alarm noise without increasing the system's missed alarm rate. The alarm settings include:

[0060] a. Tobacco production is centrally planned by the State Tobacco Administration, with distinct peak and off-peak seasons. This can be divided into peak and off-peak seasons, and monitoring alarm thresholds for count-type business indicators such as business volume can be set separately. At the same time, the baseband width can be widened, the tolerance can be increased, and alarms can be reduced. The upper baseband limit is recommended only for preventing external attacks. Therefore, the alarm threshold for business volume indicators can be based on the dynamic threshold of machine learning, combined with the fixed threshold limit of the online baseband.

[0061] b. Average response time indicator data. Since storage services involve many handheld devices and are affected by the on-site network environment, the upper and lower baseband ranges of the average response time are relatively wide, making false alarms more likely to occur. The lower baseband alarm can be canceled, and the upper baseband alarm can be combined with the database SQL statement response indicator to make a comprehensive judgment.

[0062] As a preferred embodiment, the equation for the first-order difference processing is: Among them, the value of a variable at time t is recorded as y t , the values at time t and time t-1 can be described by a first-order linear difference equation.

[0063] As a preferred implementation, the binomial distribution formula is Here, b represents the probability of the binomial distribution, n represents the number of trials, and x represents the number of times a certain outcome occurs.

[0064] Based on the above steps, the efficiency of AI intelligent analysis in detecting anomaly in tobacco warehousing business data can be effectively improved, the accuracy of alarms can be improved, and false alarms can be reduced. For comparison, please refer to Figure 2 and Figure 3 .

[0065] This embodiment further provides a system for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis. The system is used to implement the above method, and the system includes:

[0066] A parameter storage unit is used to pre-configure alarm parameters and rules. The alarm parameters include fixed thresholds and dynamic threshold tolerances. The rules include the time and number of abnormal points found by each indicator for the system to issue an alarm.

[0067] Data acquisition unit, used for collecting system data;

[0068] A preprocessing unit preprocesses the system data before pushing it into the AI intelligent operation and maintenance system for machine learning. The preprocessing method includes: classifying the stored system data indicators into two categories: the first category includes count value and average response time, and the second category includes success rate and number of errors; the two categories of data are processed using the first-order difference algorithm and the binomial distribution algorithm respectively, and then enter step 2 for machine learning;

[0069] The AI intelligent operation and maintenance unit pushes the pre-processed system data into the AI intelligent operation and maintenance system for machine learning, and obtains alarm data through unsupervised learning, which can effectively reduce the impact of data missing on the training effect of the algorithm model.

[0070] In summary, in the above implementation steps, in the process of AI intelligent implementation of tobacco warehousing business anomaly detection scenarios, it was found that warehousing business data is scarce. In business processes such as raw material management and finished product management, a large number of pauses will occur. During the business pause, data will also be missing. Insufficient learning data will lead to a decrease in the accuracy of the anomaly detection model. This method mainly uses the first-order difference processing of key data indicators such as count value and average response time before model detection processing, and the binomial distribution processing of data such as success rate and number of errors. After the time series is stabilized, a new time series data set is generated and then imported into the AI intelligent analysis system sensitive to feature period extraction for multi-indicator anomaly location detection, solving the problem of low AI intelligent analysis accuracy caused by data scarcity.

[0071] Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are illustrative and cannot be understood as limiting the present invention. Those skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and purpose of the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for improving the efficiency of AI intelligent analysis in detecting data anomalies in tobacco warehousing operations, comprising the following steps: Pre-configure alarm parameters and rules. The alarm parameters include fixed thresholds and dynamic threshold tolerances. The rules include the time required for each indicator to detect at least a certain number of abnormal points before the system issues an alarm. Set the alarm parameters and rules as follows: a. Separate monitoring alarm thresholds for count-type service indicator data based on peak and off-season data. At the same time, widen the baseband, increase tolerance, and reduce alarms. The upper baseband limit is only used to prevent external attacks. Alarm thresholds for traffic-type indicators can be combined with a fixed upper baseband limit threshold based on machine learning-based dynamic thresholds. b. For average response time indicator data, cancel the lower baseband alarm and combine the upper baseband alarm with the database SQL statement response indicator to make a comprehensive judgment; Collect system data; Push system data into the AI intelligent operation and maintenance system for machine learning, and obtain alarm data through unsupervised learning; Before pushing system data into the AI intelligent operation and maintenance system for machine learning, the system data is preprocessed. The preprocessing method includes: classifying the stored system data indicators into two categories: the first category includes count values and average response time, and the second category includes success rate and number of errors; After the two types of data are processed using the first-order difference algorithm and the binomial distribution algorithm respectively, they enter step 2 for machine learning.

2. A method for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis according to claim 1, characterized in that: After the time series processed by the binomial distribution algorithm is stabilized, a new time series data set is generated and then imported into the AI intelligent analysis system that is sensitive to feature period extraction to perform multi-indicator anomaly location detection.

3. The method for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis according to claim 1 is characterized in that: The pre-configured alarm parameters and rules include summarizing and publishing two setting suggestions for alarm rules based on long-term observation and experiments on business data.

4. The method for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis according to claim 1, characterized in that: The equation for the first-order difference process is ΔY t =Y t+1 -Y t , where the value of a variable at time t is recorded as Y t , the values at time t and time t+1 are described by first-order linear difference equations.

5. The method for improving the efficiency of data anomaly detection in tobacco warehousing business using AI intelligent analysis according to claim 1, characterized in that: The binomial distribution formula is Here, b represents the probability of the binomial distribution, n represents the number of trials, and x represents the number of times a certain outcome occurs.

6. A system for improving the efficiency of AI intelligent analysis in detecting data anomalies in tobacco warehousing business, characterized by: For implementing the method according to any one of claims 1 to 5, the system comprises: A parameter storage unit is used to pre-configure alarm parameters and rules. The alarm parameters include fixed thresholds and dynamic threshold tolerances. The rules include the time and number of abnormal points found by each indicator for the system to issue an alarm. Set the alarm parameters and rules as follows: a. Separate monitoring alarm thresholds for count-type service indicator data based on peak and off-season data. At the same time, widen the baseband, increase tolerance, and reduce alarms. The upper baseband limit is only used to prevent external attacks. Alarm thresholds for traffic-type indicators can be combined with a fixed upper baseband limit threshold based on machine learning-based dynamic thresholds. b. For average response time indicator data, cancel the lower baseband alarm and combine the upper baseband alarm with the database SQL statement response indicator to make a comprehensive judgment; Data acquisition unit, used for collecting system data; A preprocessing unit preprocesses the system data before pushing it into the AI intelligent operation and maintenance system for machine learning. The preprocessing method includes: classifying the stored system data indicators into two categories: the first category includes count value and average response time, and the second category includes success rate and number of errors; the two categories of data are processed using the first-order difference algorithm and the binomial distribution algorithm respectively, and then enter step 2 for machine learning; The AI intelligent operation and maintenance unit pushes the pre-processed system data into the AI intelligent operation and maintenance system for machine learning, and obtains alarm data through unsupervised learning.

Images