AI network security operation and maintenance data processing system based on machine learning
Through the AI network security operation and maintenance data processing system based on machine learning, combined with STL decomposition and data feature analysis, the abnormal detection accuracy of AI network security operation and maintenance data is improved, and the problem of insufficient detection accuracy in traditional methods is solved.
Patent Information
- Application Number
- CN202411197307.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The traditional method of abnormal detection of AI network security operation and maintenance data is relatively simple, and the STL algorithm is difficult to fully capture seasonal terms and trend terms, resulting in some data in the residual terms being unable to be accurately identified, reducing the accuracy of abnormal detection.
Using an AI network security operation and maintenance data processing system based on machine learning, STL decomposition is performed through the data processing module, the data analysis module obtains the degree of deviation of data points, the abnormal judgment module calculates the local similarity degree and characteristic persistence indicators, and the data detection module filters out Data points with higher degree of non-exception are detected as abnormal data points.
The accuracy of abnormal detection of AI network security operation and maintenance data is improved by filtering out data points with high deviation and irregular characteristics, the phenomenon of mistakenly identifying normal data as abnormal data is enhanced, and the accuracy of detection is enhanced.
Smart Images

Figure CN118984243A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to an AI network security operation and maintenance data processing system based on machine learning. Background Art
[0002] The AI network security operation and maintenance data processing system can continuously monitor the network and system status, detect abnormal activities in real time and issue alarms, help identify potential vulnerabilities and security risks in the system, conduct risk assessments and take remedial measures, analyze system performance data, optimize resource allocation, and improve system stability and efficiency. Since the data types involved in AI network security operation and maintenance are very wide, including a large amount of time series data, such as: timestamps of log events, time series of network traffic, occurrence time of security events, etc., they are often seasonal and periodic, and the STL algorithm, as a time series analysis method, can effectively capture these periodic changes. By performing anomaly detection on the residual term, data points that are significantly different from the expected pattern can be identified. These data points may represent potential security threats or system failures. Therefore, in the prior art, the STL algorithm is used to perform time series decomposition on the time series data of AI network security operation and maintenance data to obtain residual terms, and then the residual terms are used as the basis for abnormal data point detection to achieve anomaly detection of AI network security operation and maintenance data, thereby helping to identify abnormal behaviors, predict trends, or conduct real-time monitoring and response.
[0003] However, some data in the residual term cannot be fully captured by the seasonal term or trend term in the STL algorithm, and these data points are likely to be normal data. Therefore, the traditional method is relatively simple in detecting anomalies in AI network security operation and maintenance data with seasonal terms and trend terms, and the anomaly detection effect of all data points in the residual term is poor.
[0004] Therefore, how to improve the accuracy of anomaly detection in AI network security operation and maintenance data using the STL algorithm has become an urgent problem to be solved. Summary of the invention
[0005] In view of this, an embodiment of the present invention provides an AI network security operation and maintenance data processing system based on machine learning to solve the problem of how to improve the accuracy of anomaly detection of network security operation and maintenance data using the STL algorithm.
[0006] An embodiment of the present invention provides an AI network security operation and maintenance data processing system based on machine learning, the system comprising: A data processing module, used to obtain a data indicator involved in AI network security operation and maintenance within a preset period of time, obtain a data sequence of the data indicator, perform STL decomposition on the data sequence, and obtain a corresponding residual term; A data analysis module, configured to obtain the degree of deviation of each data point in the residual term according to the distribution characteristics of all data points in the residual term, and obtain at least one suspected abnormal data point according to the degree of deviation of each data point in the residual term; An abnormality judgment module is used to obtain, for any suspected abnormal data point, a local similarity degree of the suspected abnormal data point according to changes in data points within a local range of the suspected abnormal data point, obtain a characteristic persistence index of the suspected abnormal data point within a preset time range, and calculate the non-abnormality degree of the suspected abnormal data point according to the local similarity degree and the characteristic persistence index; The data detection module is used to obtain the non-abnormal degree of all suspected abnormal data points in the residual term, obtain the abnormal data points in the residual term according to the non-abnormal degree of all suspected abnormal data points in the residual term, and perform anomaly detection on the AI network security operation and maintenance data according to the abnormal data points.
[0007] Furthermore, the data analysis module obtains the degree of deviation of each data point in the residual term according to the distribution characteristics of all data points in the residual term, including: Obtaining the data mean of all data points in the residual term, and constructing a horizontal straight line in the residual term as a residual line according to the data mean; For any data point in the residual term, the vertical distance from the data point to the residual line is calculated, the vertical distance from each data point in the residual term to the residual line is calculated to obtain the average distance, and the ratio of the vertical distance to the average distance is used as the independent variable of the hyperbolic tangent function to obtain the degree of deviation of the data point.
[0008] Furthermore, the data analysis module obtains at least one suspected abnormal data point according to the degree of deviation of each data point in the residual term, including: For any data point, if the deviation degree of the data point is within a preset deviation degree range, the data point is regarded as a suspected abnormal data point.
[0009] Furthermore, the abnormality judgment module obtains the local similarity of the suspected abnormal data point according to the change of the data point in the local range of the suspected abnormal data point, including: The data point adjacent to the suspected abnormal data point on the left is taken as the left neighboring point, the data point adjacent to the suspected abnormal data point on the right is taken as the right neighboring point, and the suspected abnormal data point, the left neighboring point and the right neighboring point are connected to obtain a characteristic triangle; Respectively acquiring a triangle adjacent to the left and a triangle adjacent to the right of the characteristic triangle as adjacent characteristic triangles of the characteristic triangle; According to the difference in structural features between the adjacent characteristic triangles and the characteristic triangle, the local similarity of the suspected abnormal data points is obtained.
[0010] Furthermore, the abnormality judgment module obtains the local similarity of the suspected abnormal data points according to the structural feature difference between the adjacent characteristic triangles and the characteristic triangle, including: For any adjacent characteristic triangle, respectively calculate the absolute value of the difference between the two internal angles of the characteristic triangle and the adjacent characteristic triangle in each same direction, and obtain the average value of all the absolute values of the difference; The average value is used as the independent variable of the cosine function to obtain the similarity value between the characteristic triangle and the adjacent characteristic triangles. According to the similarity value between the characteristic triangle and each of the adjacent characteristic triangles, the average value of the similarity value is obtained as the local similarity degree of the suspected abnormal data point.
[0011] Furthermore, the abnormality judgment module obtains the characteristic persistence index of the suspected abnormal data point within a preset time range, including: In the residual term, a first total number of all data points of the suspected abnormal data points within a preset time range is counted, a second total number of all suspected abnormal data points of the suspected abnormal data points within the preset time range is counted, and the ratio between the first total number and the second total number is calculated to obtain a characteristic persistence index of the suspected abnormal data point.
[0012] Furthermore, the abnormality judgment module calculates the non-abnormality degree of the suspected abnormal data point according to the local similarity degree and the characteristic persistence index, including: The local similarity degree and the characteristic persistence index of the suspected abnormal data point are weightedly summed to obtain the non-abnormal degree of the suspected abnormal data point.
[0013] Furthermore, the data detection module obtains abnormal data points in the residual term according to the non-abnormal degree of all suspected abnormal data points in the residual term, including: For any non-abnormal degree, if the non-abnormal degree is within a preset non-abnormal degree range, the suspected abnormal data point corresponding to the non-abnormal degree is marked as an abnormal data point.
[0014] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The present invention provides an AI network security operation and maintenance data processing system based on machine learning, comprising a data processing module, which is used to obtain a data indicator involved in AI network security operation and maintenance within a preset time period, and correspondingly obtain a data sequence of the data indicator, and perform STL decomposition on the data sequence to obtain a corresponding residual term; a data analysis module, which is used to obtain the deviation degree of each data point in the residual term according to the distribution characteristics of all data points in the residual term, and obtain at least one suspected abnormal data point according to the deviation degree of each data point in the residual term; an abnormality judgment module, which is used to obtain the local similarity degree of the suspected abnormal data point according to the change of data points in the local range of the suspected abnormal data point for any suspected abnormal data point, obtain the characteristic persistence index of the suspected abnormal data point within a preset time range, and calculate the non-abnormal degree of the suspected abnormal data point according to the local similarity degree and the characteristic persistence index; a data detection module, which is used to obtain the non-abnormal degree of all suspected abnormal data points in the residual term, obtain the abnormal data points in the residual term according to the non-abnormal degree of all suspected abnormal data points in the residual term, and perform abnormality detection on AI network security operation and maintenance data according to the abnormal data points. Among them, by obtaining the degree of deviation of all data points in the residual term, data points with higher deviation are screened out as suspected abnormal data points. Because abnormal data points often show drastic changes or fluctuations, and have irregular characteristics in the change trend, according to these characteristics, for the suspected abnormal data points screened out above, the local similarity of the suspected abnormal data points is calculated by analyzing the distribution characteristics of the data in the local range of the suspected abnormal data points, and the persistence of the suspected abnormal data points in a certain period of time (that is, the characteristic persistence index) is analyzed. The non-abnormal degree of the suspected abnormal data points is obtained through the local similarity and persistence of the suspected abnormal data points, and the suspected abnormal data points that meet the non-abnormal degree requirements are found and defined as abnormal data points. Then, anomaly detection is performed on the AI network security operation and maintenance data based on the abnormal data points, so as to improve the accuracy of anomaly detection on the AI network security operation and maintenance data using the STL algorithm. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0016] Figure 1 This is a structural block diagram of an AI network security operation and maintenance data processing system based on machine learning provided in Embodiment 1 of the present invention; Figure 2It is a schematic diagram of constructing a feature triangle and adjacent feature triangles according to suspected abnormal data points in a residual term provided by the first embodiment of the present invention. DETAILED DESCRIPTION
[0017] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, but should not be understood as limiting the present disclosure.
[0018] It should be noted that the terms "first", "second", etc. in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.
[0019] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.
[0020] See also Figure 1 , is a structural block diagram of an AI network security operation and maintenance data processing system based on machine learning provided in Embodiment 1 of the present invention, such as Figure 1 As shown, the system may include: The data processing module 11 is used to obtain a data indicator involved in AI network security operation and maintenance within a preset time period, obtain a data sequence of the data indicator, perform STL decomposition on the data sequence, and obtain a corresponding residual term.
[0021] First, determine a data indicator involved in AI network security operation and maintenance that needs to be obtained, such as log data, network traffic data, user behavior data, etc. Set the preset period to 24 hours, there is no restriction here, implementers can set it according to the implementation scenario, set the sampling frequency to 1 minute, there is no restriction here, implementers can set it according to the implementation scenario, collect the data of the data indicator once every 1 minute, and obtain the data sequence corresponding to the data indicator within 24 hours. In order to ensure the accuracy and consistency of each data in the data sequence, after obtaining the data sequence, each data in the data sequence is preprocessed to achieve the purpose of data cleaning for the data sequence, where data preprocessing includes but is not limited to: data cleaning, filling missing values, deleting samples or features where missing values are located, detecting and processing outliers, and deleting duplicate samples or features; data normalization, ensuring that the data meets specific standards or specifications, such as unit consistency, unified data format, etc.; data conversion, such as normalizing or standardizing the data to ensure that different indicators are comparable. Among them, data preprocessing technology belongs to the prior art and will not be repeated here.
[0022] Then, the STL algorithm is used to decompose all the data in the data sequence, and data distribution diagrams of the three parts, namely, the trend term, the seasonal term and the residual term, are obtained respectively. Among them, the trend term is the trend component of long-term changes in the data sequence, reflecting the basic trend of the data over time; the seasonal term is the periodic repetitive pattern in the data sequence, reflecting the regular fluctuation of the data in the data sequence; the residual term is the randomly fluctuating data in the data sequence after removing the trend term and the seasonal term. Among them, the STL algorithm is a prior art and will not be described here.
[0023] The data analysis module 12 is used to obtain the deviation degree of each data point in the residual term according to the distribution characteristics of all data points in the residual term, and obtain at least one suspected abnormal data point according to the deviation degree of each data point in the residual term.
[0024] When using the STL algorithm to perform anomaly detection on AI network security operation and maintenance data in a traditional way, all data in the residual term is used as abnormal data points for anomaly detection. However, since some data cannot be fully captured by the seasonal term and trend term in the STL algorithm, these data points show periodic fluctuations or continuous characteristics in the residual term. Data points with such performance should be marked as normal data points rather than abnormal data points. Therefore, it is necessary to analyze the distribution characteristics of each data point in the residual term, obtain abnormal data points, and use the abnormal data points as the basis for anomaly detection on AI network security operation and maintenance data.
[0025] In the residual term, abnormal data points usually appear as data points that deviate from the normal pattern. Therefore, according to the distribution characteristics of all data points in the residual term, the degree of deviation of all data points in the residual term is obtained. The higher the degree of deviation, the more the data point deviates from the normal data range and the more likely it is an abnormal data point. Therefore, the data points corresponding to the higher degree of deviation are screened out as suspected abnormal data points to determine whether the data point is an abnormal data point.
[0026] Among them, obtaining the degree of deviation of all data points in the residual term includes: Obtaining the data mean of all data points in the residual term, and constructing a horizontal straight line in the residual term as a residual line according to the data mean; For any data point in the residual term, the vertical distance from the data point to the residual line is calculated, the vertical distance from each data point in the residual term to the residual line is calculated to obtain the average distance, and the ratio of the vertical distance to the average distance is used as the independent variable of the hyperbolic tangent function to obtain the degree of deviation of the data point.
[0027] In one implementation, the data mean of all data points in the residual term is first calculated, and a horizontal straight line is constructed in the residual term according to the data mean as the residual line of the residual term.
[0028] For the ith data point in the residual term, a straight line perpendicular to the residual line is drawn through the ith data point to obtain the intersection of the straight line and the residual line, and the Euclidean distance between the intersection and the ith data point is calculated, and the Euclidean distance is used as the distance from the ith data point to the residual line, where the Euclidean distance is a prior art and will not be described here. Similarly, the distances from all data points in the residual term to the residual line are calculated to obtain the average distance, and the degree of deviation of the ith data point is obtained based on the distance from the ith data point to the residual line and the average distance. The calculation formula for the degree of deviation is: in, Indicates the degree of deviation of the i-th data point, represents the distance from the i-th data point to the residual line, n represents the number of data points in the residual term, It represents the average distance from all data points in the residual term to the residual line, and tanh() represents the hyperbolic tangent function.
[0029] It should be noted that the residual line is used to characterize the distribution of normal data points in the residual term. Therefore, the distance from the i-th data point to the residual line is The larger the value of is, the farther the vertical distance between the ith data point and the residual line is. The data point deviates from the normal data point distribution, and the greater the deviation of the ith data point is. Tanh is used to The value is fixed between [0, 1]. The closer the value is to 1, the more the i-th data point deviates from the normal data range and the more likely it is an abnormal data point.
[0030] Similarly, the degree of deviation of all data points in the residual term is obtained, and then the data points corresponding to the higher degree of deviation are screened out as suspected abnormal data points.
[0031] Among them, the data points corresponding to the higher degree of deviation are screened out as suspected abnormal data points, including: For any data point, if the deviation degree of the data point is within a preset deviation degree range, the data point is regarded as a suspected abnormal data point.
[0032] In one embodiment, the preset deviation range is set to (0.7, 1), which is not limited here. The implementer can set it according to the implementation scenario. For any data point, if the deviation degree of the data point is within (0.7, 1), the data point is regarded as a suspected abnormal data point. Similarly, all suspected abnormal data points in the residual term are obtained.
[0033] The abnormality judgment module 13 is used to obtain, for any suspected abnormal data point, the local similarity of the suspected abnormal data point according to the change of data points within the local range of the suspected abnormal data point, obtain the characteristic persistence index of the suspected abnormal data point within a preset time range, and calculate the non-abnormality degree of the suspected abnormal data point according to the local similarity and the characteristic persistence index.
[0034] Since some data in the residual term cannot be fully captured by the seasonal term or trend term in the STL algorithm, these data show periodic fluctuations or continuous characteristics in the residual term. Therefore, the non-abnormal degree of each suspected abnormal data point is obtained based on this characteristic to determine whether each suspected abnormal data point is an abnormal data point.
[0035] Among them, any suspected abnormal data point in the residual term Take this as an example to obtain suspected abnormal data points The non-anomalous degree of the process includes: (1) Obtaining the local similarity of the suspected abnormal data point according to the change of the data point in the local range of the suspected abnormal data point, including: The data point adjacent to the suspected abnormal data point on the left is taken as the left neighboring point, the data point adjacent to the suspected abnormal data point on the right is taken as the right neighboring point, and the suspected abnormal data point, the left neighboring point and the right neighboring point are connected to obtain a characteristic triangle; Respectively acquiring a triangle adjacent to the left and a triangle adjacent to the right of the characteristic triangle as adjacent characteristic triangles of the characteristic triangle; According to the difference in structural features between the adjacent characteristic triangles and the characteristic triangle, the local similarity of the suspected abnormal data points is obtained.
[0036] In one embodiment, the residual term and the suspected abnormal data point The left adjacent data point is regarded as a suspected abnormal data point The left neighbor of The right neighbor point, the left neighbor point, the right neighbor point and the suspected abnormal data point Connect the lines to get suspected abnormal data points characteristic triangle.
[0037] Further, obtain each data point located to the left of the left neighbor in the residual term, calculate the horizontal distance between these data points and the left neighbor, sort all the horizontal distances from small to large, take the data points corresponding to the first three horizontal distances after sorting, connect these three data points, and obtain the feature triangle adjacent to the left of the feature triangle. Similarly, obtain the feature triangle adjacent to the right of the feature triangle, and use the feature triangle adjacent to the right of the feature triangle and the feature triangle adjacent to the left of the feature triangle as the adjacent feature triangles of the feature triangle. Figure 2 , which is a schematic diagram of constructing feature triangles and adjacent feature triangles based on suspected abnormal data points in the residual term. Figure 2 In the figure, the gray data points are suspected abnormal data points , and suspected abnormal data points The left adjacent data point is the left adjacent point 1, which is the suspected abnormal data point The right adjacent data point is right adjacent point 2, which is a suspected abnormal data point The triangle formed by the left neighbor point 1 and the right neighbor point 2 is the suspected abnormal data point The characteristic triangle of Figure 2 The leftmost triangle is the triangle adjacent to the left of the characteristic triangle, that is, the adjacent characteristic triangle of the characteristic triangle. Figure 2 The rightmost triangle in is the triangle adjacent to the feature triangle on the right, that is, the adjacent feature triangle of the feature triangle.
[0038] By obtaining the difference between the internal angles of the feature triangle and the adjacent feature triangle, the suspected abnormal data points are obtained. If the difference of the inner angle is small, that is, the local similarity is greater, it means that there are suspected abnormal data points. There is a high probability of a change pattern, and there are suspected abnormal data points The lower the probability of being an abnormal data point, the greater the probability of being a normal data point.
[0039] Among them, suspected abnormal data points The acquisition of local similarity includes: For any adjacent characteristic triangle, respectively calculate the absolute value of the difference between the two internal angles of the characteristic triangle and the adjacent characteristic triangle in each same direction, and obtain the average value of all the absolute values of the difference; The average value is used as the independent variable of the cosine function to obtain the similarity value between the characteristic triangle and the adjacent characteristic triangles. According to the similarity value between the characteristic triangle and each of the adjacent characteristic triangles, the average value of the similarity value is obtained as the local similarity degree of the suspected abnormal data point.
[0040] In one embodiment, the upper inner angle of the triangle is taken as the first inner angle, the left inner angle is taken as the second inner angle, and the right inner angle is taken as the third inner angle. For the tth adjacent characteristic triangle, the suspected abnormal data point is calculated. The absolute value of the difference between the I-th internal angle of the characteristic triangle of the t-th adjacent characteristic triangle is used to obtain the suspected abnormal data point. The local similarity degree is calculated as follows: in, Indicates suspected abnormal data points The local similarity of represents the I-th interior angle of the characteristic triangle, represents the I-th internal angle of the t-th adjacent characteristic triangle, cos() represents the cosine function, Represents the absolute value symbol.
[0041] It should be noted that The smaller the value, the more likely the abnormal data point is. The smaller the difference between the Ith interior angle of the characteristic triangle of and the Ith interior angle of the tth adjacent characteristic triangle, the more equal the angles of the two interior angles are. The smaller the value of is, the smaller the angle difference between the three internal angles of the characteristic triangle and the tth adjacent characteristic triangle is, that is, the more similar the characteristic triangle is to the tth adjacent characteristic triangle. Therefore, when When the value of is closer to 0, the value of the cosine function cos() is closer to 1, and the suspected abnormal data point The local similarity The closer the value of is to 1, the more likely the abnormal data point is. The data in its local range has a similar change trend and a certain change pattern, rather than a sudden change. The corresponding suspected abnormal data points The probability of being an abnormal data point is lower.
[0042] (2) Obtaining characteristic persistence indicators of the suspected abnormal data points within a preset time range, including: In the residual term, a first total number of all data points of the suspected abnormal data points within a preset time range is counted, a second total number of all suspected abnormal data points of the suspected abnormal data points within the preset time range is counted, and the ratio between the first total number and the second total number is calculated to obtain a characteristic persistence index of the suspected abnormal data point.
[0043] In one implementation, the preset time range is set to 5, which is not limited here. The implementer can set it according to the implementation scenario, and obtain suspected abnormal data points in the residual term. The data points included in the 5 minutes before and after the time point are counted. The number of all suspected abnormal data points included in these 10 minutes is counted. The suspected abnormal data points are obtained based on the number of all suspected abnormal data points. The characteristic persistence index is calculated as follows: in, Indicates suspected abnormal data points The characteristic persistence index of Indicates the preset time range, Indicates suspected abnormal data points The time period before the current time point , Indicates suspected abnormal data points The time period after the current time point , Represents suspected abnormal data points in the residual term The time period before and after the current time point The number of all data points in, that is, the first total number, Represents suspected abnormal data points in the residual term The time period before and after the current time point The number of all suspected abnormal data points in , that is, the second total number.
[0044] It is worth mentioning that The larger the value, the more likely the data point is to be abnormal. The time period before and after the current time point The more suspected abnormal data contained in the data, the more likely it is that data that cannot be fully captured by the trend term and seasonal term of the STL algorithm will appear in the current time period. The value range of is (0, 1), The closer it is to 1, the more abnormal the data point is. The higher the feature persistence, the more likely the corresponding suspected abnormal data point is. The probability of being an abnormal data point is lower.
[0045] (3) Calculating the non-abnormality degree of the suspected abnormal data point based on the local similarity degree and the characteristic persistence index, including: The local similarity degree and the characteristic persistence index of the suspected abnormal data point are weightedly summed to obtain the non-abnormal degree of the suspected abnormal data point.
[0046] In one embodiment, based on the suspected abnormal data point The local similarity and characteristic persistence index of the data points are used to obtain the suspected abnormal data points. The calculation formula of non-abnormal degree is: in, Indicates suspected abnormal data points The degree of non-anomalousness, represents the first weight value, represents the second weight value, Indicates suspected abnormal data points The local similarity of Indicates suspected abnormal data points Characteristic persistence indicator.
[0047] It should be noted that setting is 0.85, set is 0.15, there is no limit here, the implementer can set it according to the implementation scenario, and then The value range of is (0, 1), The larger the value or The larger the value of The closer it is to 1, the more likely it is that there are abnormal data points in that time period. The data persistence is high and there are suspected abnormal data points There is a certain change pattern, not a sudden change, suspected abnormal data points The more likely it is a normal data point.
[0048] The data detection module 14 is used to obtain the non-abnormal degree of all suspected abnormal data points in the residual term, obtain the abnormal data points in the residual term according to the non-abnormal degree of all suspected abnormal data points in the residual term, and perform anomaly detection on the AI network security operation and maintenance data according to the abnormal data points.
[0049] According to the abnormality judgment module 13, the non-abnormality degree of all the suspected abnormal data points in the residual term is obtained.
[0050] Since some data in the residual term cannot be fully captured by the seasonal term or trend term in the STL algorithm, these data may be caused by the inherent periodicity or seasonal factors in the data and are not abnormal data points. Therefore, the real abnormal data points should be screened out according to the non-abnormality degree of each suspected abnormal data point in the residual term.
[0051] Wherein, obtaining the abnormal data points in the residual term according to the non-abnormal degree of all the suspected abnormal data points in the residual term includes: For any non-abnormal degree, if the non-abnormal degree is within a preset non-abnormal degree range, the suspected abnormal data point corresponding to the non-abnormal degree is marked as an abnormal data point.
[0052] In one embodiment, the preset non-abnormal degree range is set to [0, 0.9]. There is no limitation here. The implementer can set it according to the implementation scenario. For any suspected abnormal data point in the residual term, if the non-abnormal degree of the suspected abnormal data point is within [0, 0.9], the suspected abnormal data point is marked as an abnormal data point; if the non-abnormal degree of the suspected abnormal data point is within (0.9, 1), the suspected abnormal data point is marked as a normal data point.
[0053] Similarly, the non-abnormal degree of each suspected abnormal data point in the residual term is analyzed to obtain all abnormal data points in the residual term, and then the AI network security operation and maintenance data is detected for anomalies based on the abnormal data points. Anomaly detection includes but is not limited to confirming whether the abnormal data point represents a network security incident or problem, and promptly feeding back the problem to the system. The system will generate an alarm or notify relevant personnel of the occurrence of the abnormal situation. At the same time, the data will be recorded and archived for subsequent analysis; then the predetermined response process is initiated, the anomaly is deeply analyzed to determine its root cause, and necessary repair operations are performed to generate an abnormality report for management personnel to review and make decisions.
[0054] It is worth noting that the focus of the present invention is on how to filter out abnormal data points and normal data points in the residual terms obtained by decomposing time series data using the STL algorithm. Anomaly detection of AI network security operation and maintenance data based on abnormal data points belongs to the prior art and will not be repeated here.
[0055] In summary, the present invention provides an AI network security operation and maintenance data processing system based on machine learning, including a data processing module, which is used to obtain a data indicator involved in AI network security operation and maintenance within a preset time period, and obtain a data sequence of the data indicator, and perform STL decomposition on the data sequence to obtain a corresponding residual term; a data analysis module, which is used to obtain the degree of deviation of each data point in the residual term according to the distribution characteristics of all data points in the residual term, and obtain at least one suspected abnormal data point according to the degree of deviation of each data point in the residual term; an abnormal judgment module, which is used to obtain the local similarity of any suspected abnormal data point according to the change of data points in the local range of the suspected abnormal data point, obtain the characteristic persistence index of the suspected abnormal data point within a preset time range, and calculate the non-abnormal degree of the suspected abnormal data point according to the local similarity and the characteristic persistence index; a data detection module, which is used to obtain the non-abnormal degree of all suspected abnormal data points in the residual term, obtain the abnormal data points in the residual term according to the non-abnormal degree of all suspected abnormal data points in the residual term, and perform abnormality detection on the AI network security operation and maintenance data according to the abnormal data points. Among them, by obtaining the degree of deviation of all data points in the residual term, data points with higher deviation are screened out as suspected abnormal data points. Because abnormal data points often show drastic changes or fluctuations, and have irregular characteristics in the change trend, according to these characteristics, for the suspected abnormal data points screened out above, the local similarity of the suspected abnormal data points is calculated by analyzing the distribution characteristics of the data in the local range of the suspected abnormal data points, and the persistence of the suspected abnormal data points in a certain period of time (that is, the characteristic persistence index) is analyzed. The non-abnormal degree of the suspected abnormal data points is obtained through the local similarity and persistence of the suspected abnormal data points, and the suspected abnormal data points that meet the non-abnormal degree requirements are found and defined as abnormal data points. Then, anomaly detection is performed on the AI network security operation and maintenance data based on the abnormal data points, so as to improve the accuracy of anomaly detection on the AI network security operation and maintenance data using the STL algorithm.
[0056] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.
Claims
1. AI network security operation and maintenance data processing system based on machine learning, characterized by: The AI network security operation and maintenance data processing system based on machine learning includes: A data processing module, used to obtain a data indicator involved in AI network security operation and maintenance within a preset period of time, obtain a data sequence of the data indicator, perform STL decomposition on the data sequence, and obtain a corresponding residual term; A data analysis module, configured to obtain the degree of deviation of each data point in the residual term according to the distribution characteristics of all data points in the residual term, and obtain at least one suspected abnormal data point according to the degree of deviation of each data point in the residual term; An abnormality judgment module is used to obtain, for any suspected abnormal data point, a local similarity degree of the suspected abnormal data point according to changes in data points within a local range of the suspected abnormal data point, obtain a characteristic persistence index of the suspected abnormal data point within a preset time range, and calculate the non-abnormality degree of the suspected abnormal data point according to the local similarity degree and the characteristic persistence index; The data detection module is used to obtain the non-abnormal degree of all suspected abnormal data points in the residual term, obtain the abnormal data points in the residual term according to the non-abnormal degree of all suspected abnormal data points in the residual term, and perform anomaly detection on the AI network security operation and maintenance data according to the abnormal data points.
2. The AI network security operation and maintenance data processing system based on machine learning according to claim 1 is characterized in that: The data analysis module obtains the degree of deviation of each data point in the residual term according to the distribution characteristics of all data points in the residual term, including: Obtaining the data mean of all data points in the residual term, and constructing a horizontal straight line in the residual term as a residual line according to the data mean; For any data point in the residual term, the vertical distance from the data point to the residual line is calculated, the vertical distance from each data point in the residual term to the residual line is calculated to obtain the average distance, and the ratio of the vertical distance to the average distance is used as the independent variable of the hyperbolic tangent function to obtain the degree of deviation of the data point.
3. The AI network security operation and maintenance data processing system based on machine learning according to claim 1 is characterized in that: The data analysis module obtains at least one suspected abnormal data point according to the degree of deviation of each data point in the residual term, including: For any data point, if the deviation degree of the data point is within a preset deviation degree range, the data point is regarded as a suspected abnormal data point.
4. The AI network security operation and maintenance data processing system based on machine learning according to claim 1 is characterized in that: The abnormality judgment module obtains the local similarity of the suspected abnormal data point according to the change of the data point in the local range of the suspected abnormal data point, including: The data point adjacent to the suspected abnormal data point on the left is taken as the left neighboring point, the data point adjacent to the suspected abnormal data point on the right is taken as the right neighboring point, and the suspected abnormal data point, the left neighboring point and the right neighboring point are connected to obtain a characteristic triangle; Respectively acquiring a triangle adjacent to the left and a triangle adjacent to the right of the characteristic triangle as adjacent characteristic triangles of the characteristic triangle; According to the difference in structural features between the adjacent characteristic triangles and the characteristic triangle, the local similarity of the suspected abnormal data points is obtained.
5. The AI network security operation and maintenance data processing system based on machine learning according to claim 4 is characterized in that: The abnormality judgment module obtains the local similarity of the suspected abnormal data points according to the structural feature difference between the adjacent characteristic triangle and the characteristic triangle, including: For any adjacent characteristic triangle, respectively calculate the absolute value of the difference between the two internal angles of the characteristic triangle and the adjacent characteristic triangle in each same direction, and obtain the average value of all the absolute values of the difference; The average value is used as the independent variable of the cosine function to obtain the similarity value between the characteristic triangle and the adjacent characteristic triangles. According to the similarity value between the characteristic triangle and each of the adjacent characteristic triangles, the average value of the similarity value is obtained as the local similarity degree of the suspected abnormal data point.
6. The AI network security operation and maintenance data processing system based on machine learning according to claim 1 is characterized in that: The abnormality judgment module obtains the characteristic persistence index of the suspected abnormal data point within a preset time range, including: In the residual term, a first total number of all data points of the suspected abnormal data points within a preset time range is counted, a second total number of all suspected abnormal data points of the suspected abnormal data points within the preset time range is counted, and the ratio between the first total number and the second total number is calculated to obtain a characteristic persistence index of the suspected abnormal data point.
7. The AI network security operation and maintenance data processing system based on machine learning according to claim 1 is characterized in that: The abnormality judgment module calculates the non-abnormality degree of the suspected abnormal data point according to the local similarity degree and the characteristic persistence index, including: The local similarity degree and the characteristic persistence index of the suspected abnormal data point are weightedly summed to obtain the non-abnormal degree of the suspected abnormal data point.
8. The AI network security operation and maintenance data processing system based on machine learning according to claim 1 is characterized in that: The data detection module obtains abnormal data points in the residual term according to the non-abnormal degree of all suspected abnormal data points in the residual term, including: For any non-abnormal degree, if the non-abnormal degree is within a preset non-abnormal degree range, the suspected abnormal data point corresponding to the non-abnormal degree is marked as an abnormal data point.
Citation Information
Cited By
Universe identity authentication method, device and system
CN120639304A
A real-time temperature monitoring method in high-frequency heating anchor cable production process
CN122692861A
A real-time temperature monitoring method in high-frequency heating anchor cable production process
CN122692861B
