Abnormality identification method, device, equipment, medium and program based on safety monitoring

Abstract

The application provides an abnormality identification method, device, equipment, medium and program based on security monitoring. The method comprises: acquiring real-time monitoring data, the real-time monitoring data representing network data generated by a user end; performing feature extraction according to the real-time monitoring data to obtain a target feature vector, the target feature vector representing a data deviation degree of the real-time monitoring data compared with historical monitoring data; acquiring a target detection dimension, and based on the target detection dimension and the target feature vector, obtaining a target feature matrix corresponding to the target detection dimension, the target feature matrix representing a data deviation degree between current network data and historical network data of the target detection dimension; based on the target feature matrix and a historical feature matrix, obtaining a real-time detection result corresponding to the target detection dimension, the real-time detection result representing an abnormality of the real-time monitoring data. The method of the application can more quickly realize abnormality detection by reducing the amount of data to be processed.

Description

Technical Field

[0001] This application relates to the field of data security, and in particular to an anomaly identification method, apparatus, device, medium and program based on security monitoring. Background Technology

[0002] With the development of the internet, web applications have become an important part of people's daily lives and work. However, the complexity and openness of web applications have also brought enormous security challenges. Front-end security issues such as XSS (Cross-Site Scripting) and CSRF (Cross-Site Request Forwarding) are commonplace, posing a serious threat to user privacy and data security. Traditional front-end security monitoring methods mainly rely on predefined rules and static analysis, which have significant limitations and cannot effectively cope with increasingly complex and ever-changing security threats.

[0003] One related technology is a front-end security detection approach that combines static and dynamic analysis. This method integrates static code analysis and dynamic behavior monitoring. By performing static analysis during the development phase and dynamic monitoring during runtime, potential security issues can be detected. Static analysis primarily detects security vulnerabilities in the code, while dynamic monitoring uses instrumentation techniques to monitor the execution of the front-end code at runtime.

[0004] However, dynamic monitoring consumes a lot of resources, and the process of directly judging front-end security based on the amount of dynamic monitoring data may take a long time.

[0005] Therefore, how to improve the speed of anomaly detection by reducing the amount of data to be processed is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] This application provides an anomaly identification method, apparatus, device, medium, and program based on security monitoring to solve the technical problem of slow anomaly detection speed.

[0007] In a first aspect, this application provides an anomaly identification method based on security monitoring, comprising:

[0008] Acquire real-time monitoring data, wherein the real-time monitoring data represents network data generated by the user terminal;

[0009] Based on the real-time monitoring data, feature extraction is performed to obtain multiple target feature vectors, wherein the target feature vectors represent the degree of data offset of the real-time monitoring data compared with the historical monitoring data.

[0010] Obtain the target detection dimension, and based on the target detection dimension and the target feature vector, obtain the target feature matrix corresponding to the target detection dimension, wherein the target feature matrix represents the degree of data offset between the current network data and the historical network data of the target detection dimension;

[0011] Based on the target feature matrix and the historical feature matrix, the real-time detection result corresponding to the target detection dimension is obtained, wherein the real-time detection result represents the abnormal situation of the real-time monitored data.

[0012] Optionally, in the method described above, the real-time monitoring data includes real-time sub-data corresponding to multiple dimensions. Feature extraction is performed based on the real-time monitoring data to obtain multiple target feature vectors, including:

[0013] For each dimension, based on the historical sub-data corresponding to the dimension, the data distribution information corresponding to the dimension is obtained, wherein the data distribution information represents the distribution of historical network data in different time periods;

[0014] Based on the data distribution information and the real-time sub-data, a data offset is obtained, wherein the data offset represents the degree of data offset of the real-time sub-data compared to the historical sub-data;

[0015] For each monitoring type, a target feature vector corresponding to the monitoring type is obtained based on the data offset corresponding to the monitoring type, wherein the monitoring type includes several dimensions.

[0016] Optionally, in the method described above, the data distribution information includes feature string groups corresponding to multiple dimensions, and the data offset is obtained based on the data distribution information and the real-time sub-data, including:

[0017] For each dimension, N-gram similarity is calculated based on the real-time sub-data and the feature string group corresponding to the dimension to obtain multiple association similarities corresponding to the dimension, wherein the association similarity represents the similarity between the feature string in the feature string group and the real-time sub-data;

[0018] The data offset is obtained based on the multiple association similarities, wherein the data offset characterizes the distribution characteristics of the multiple association similarities.

[0019] Optionally, the method described above, based on the target detection dimension and the target feature vector, obtains the target feature matrix corresponding to the target detection dimension, including:

[0020] Historical abnormal data corresponding to the target detection dimension is obtained, and the target detection dimension is queued based on the historical abnormal data to obtain a target dimension queue, wherein the target dimension queue includes multiple target detection dimensions arranged according to abnormal frequency;

[0021] Based on the target dimension queue and the target feature vector, the target feature matrix is ​​obtained.

[0022] Optionally, the method described above, based on the target detection dimension and the target feature vector, obtains the target feature matrix corresponding to the target detection dimension, including:

[0023] Obtain the target time range, and determine the target feature vector set based on the target feature vector and the target time range, wherein the target feature vector set includes several target feature vectors within the target time range;

[0024] The target feature matrix is ​​obtained based on the target feature vector set.

[0025] Optionally, the method described above, based on the target feature matrix and the historical feature matrix, obtains the real-time detection result corresponding to the target detection dimension, including:

[0026] Obtain the target feature values ​​corresponding to the target feature matrix;

[0027] Obtain the historical feature values ​​corresponding to the historical feature matrix;

[0028] An initial detection result is obtained based on the historical feature values ​​and the target feature values, wherein the initial detection result represents the similarity relationship between the target feature matrix and the historical feature matrix;

[0029] The real-time detection result is obtained based on multiple initial detection results, wherein the real-time detection result represents the central tendency of the multiple initial detection results.

[0030] Secondly, this application provides an anomaly identification device based on security monitoring, comprising:

[0031] The acquisition module is used to acquire real-time monitoring data, wherein the real-time monitoring data represents network data generated by the user terminal;

[0032] The first processing module is used to extract features based on the real-time monitoring data to obtain multiple target feature vectors, wherein the target feature vectors characterize the degree of data offset between the real-time monitoring data and the historical monitoring data.

[0033] The second processing module is used to obtain the target detection dimension and, based on the target detection dimension and the target feature vector, obtain the target feature matrix corresponding to the target detection dimension, wherein the target feature matrix represents the degree of data offset between the current network data and the historical network data of the target detection dimension.

[0034] The analysis module is used to obtain the real-time detection results corresponding to the target detection dimension based on the target feature matrix and the historical feature matrix, wherein the real-time detection results represent the abnormal situation of the real-time monitored data.

[0035] Optionally, the real-time monitoring data includes real-time sub-data corresponding to multiple dimensions. The first processing module, when performing feature extraction based on the real-time monitoring data to obtain multiple target feature vectors, is used to:

[0036] For each dimension, based on the historical sub-data corresponding to the dimension, the data distribution information corresponding to the dimension is obtained, wherein the data distribution information represents the distribution of historical network data in different time periods;

[0037] Based on the data distribution information and the real-time sub-data, a data offset is obtained, wherein the data offset represents the degree of data offset of the real-time sub-data compared to the historical sub-data;

[0038] For each monitoring type, a target feature vector corresponding to the monitoring type is obtained based on the data offset corresponding to the monitoring type, wherein the monitoring type includes several dimensions.

[0039] Optionally, the data distribution information includes feature string groups corresponding to multiple dimensions. The first processing module, when executing the process of obtaining the data offset based on the data distribution information and the real-time sub-data, is used to:

[0040] For each dimension, N-gram similarity is calculated based on the real-time sub-data and the feature string group corresponding to the dimension to obtain multiple association similarities corresponding to the dimension, wherein the association similarity represents the similarity between the feature string in the feature string group and the real-time sub-data;

[0041] The data offset is obtained based on the multiple association similarities, wherein the data offset characterizes the distribution characteristics of the multiple association similarities.

[0042] Optionally, the second processing module described above, when performing the process of obtaining the target feature matrix corresponding to the target detection dimension based on the target detection dimension and the target feature vector, is used to:

[0043] Historical abnormal data corresponding to the target detection dimension is obtained, and the target detection dimension is queued based on the historical abnormal data to obtain a target dimension queue, wherein the target dimension queue includes multiple target detection dimensions arranged according to abnormal frequency;

[0044] Based on the target dimension queue and the target feature vector, the target feature matrix is ​​obtained.

[0045] Optionally, the second processing module described above, when performing the process of obtaining the target feature matrix corresponding to the target detection dimension based on the target detection dimension and the target feature vector, is used to:

[0046] Obtain the target time range, and determine the target feature vector set based on the target feature vector and the target time range, wherein the target feature vector set includes several target feature vectors within the target time range;

[0047] The target feature matrix is ​​obtained based on the target feature vector set.

[0048] Optionally, the analysis module described above, when executing the real-time detection result corresponding to the target detection dimension based on the target feature matrix and the historical feature matrix, is used to:

[0049] Obtain the target feature values ​​corresponding to the target feature matrix;

[0050] Obtain the historical feature values ​​corresponding to the historical feature matrix;

[0051] An initial detection result is obtained based on the historical feature values ​​and the target feature values, wherein the initial detection result represents the similarity relationship between the target feature matrix and the historical feature matrix;

[0052] The real-time detection result is obtained based on multiple initial detection results, wherein the real-time detection result represents the central tendency of the multiple initial detection results.

[0053] Thirdly, an electronic device includes: a processor, and a memory communicatively connected to the processor;

[0054] The memory stores computer-executed instructions;

[0055] The processor executes computer execution instructions stored in the memory to implement the anomaly identification method based on security monitoring as described in the first aspect.

[0056] Fourthly, a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the anomaly identification method based on security monitoring as described in the first aspect.

[0057] Fifthly, a computer program product includes a computer program that, when executed by a processor, implements the anomaly detection method based on security monitoring as described in the first aspect.

[0058] This application provides an anomaly identification method, apparatus, device, medium, and program based on security monitoring. It acquires real-time monitoring data to obtain network data generated by the user terminal; extracts features from the real-time monitoring data to obtain extracted target feature vectors, thereby determining the data offset between the real-time monitoring data and historical monitoring data, achieving initial compression of the dynamic monitoring data; acquires a target detection dimension, and based on the target detection dimension and the extracted target feature vectors, obtains a target feature matrix corresponding to the target detection dimension, thereby determining the data offset between the current network data and historical network data for the target detection dimension, achieving secondary compression of the dynamic monitoring data; and obtains real-time detection results corresponding to the target detection dimension more quickly based on the historical feature matrix and the target feature matrix obtained after the two compressions, achieving the beneficial effect of faster anomaly identification. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0060] Figure 1 A schematic diagram illustrating an application scenario for the anomaly identification method based on security monitoring provided in this application;

[0061] Figure 2 A flowchart illustrating the anomaly identification method based on security monitoring provided in this application;

[0062] Figure 3 Another flowchart illustrating the anomaly identification method based on security monitoring provided in this application;

[0063] Figure 4 A schematic diagram of the structure of the anomaly identification device based on security monitoring provided in this application;

[0064] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0065] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0066] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0068] As the user interface for a website or application, the security of the web front-end is directly related to user data security. With the continuous evolution of network security methods, monitoring the web front-end can effectively improve system security and prevent potential security threats.

[0069] Therefore, rule-based front-end security monitoring methods have emerged. Common methods include using Content Security Policy (CSP), security headers (such as X-Content-Type-Options, X-Frame-Options, etc.), and static analysis tools (such as the ESLint plugin) to detect and prevent XSS and CSRF attacks. Front-end security monitoring methods based on predefined rule sets and static code analysis are simple to implement, have clear rules, and offer good defense against known attack methods such as XSS and CSRF. However, they cannot handle unknown attacks. When an unknown attack occurs, the corresponding defense method needs to be manually written into the rule set, thus lacking flexibility and unable to adapt to dynamically changing security threats in real time.

[0070] Dynamic monitoring of real-time data can solve the problem of being unable to detect unknown attacks, which is based on a front-end security monitoring method that combines static and dynamic analysis. This method combines static code analysis and dynamic behavior monitoring. Static analysis is performed during the development phase, and dynamic monitoring is performed at runtime to detect potential security issues. Static analysis mainly detects security vulnerabilities in the code, while dynamic monitoring monitors the execution of front-end code at runtime through instrumentation techniques. The above method combines the advantages of static and dynamic analysis, enabling dual detection of potential security issues during development and runtime. However, dynamic monitoring consumes more resources, and the process of directly judging front-end security based on the amount of dynamic monitoring data can be time-consuming.

[0071] Therefore, how to improve the speed of anomaly detection by reducing the amount of data to be processed is a technical problem that urgently needs to be solved by those skilled in the art.

[0072] like Figure 1 As shown, the security monitoring device obtains dynamic data from the server, then performs security anomaly analysis based on the dynamic data, and sends the analysis results to the terminal.

[0073] This application provides an anomaly identification method based on security monitoring, aiming to solve the above-mentioned technical problems of the prior art, and is executed by an electronic device. The electronic device can be a server or a terminal device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet computer, laptop computer, desktop computer, etc., but is not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this.

[0074] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0075] like Figure 2 As shown, an anomaly identification method based on security monitoring may specifically include steps S101 to S104, wherein:

[0076] S101. Obtain real-time monitoring data, wherein the real-time monitoring data represents network data generated by the user terminal.

[0077] The aforementioned real-time monitored data includes user behavior data and network request data. User behavior data refers to user actions such as clicks and input on the server. Network request data refers to network requests initiated by user terminals corresponding to the server.

[0078] S102. Based on the real-time monitoring data, feature extraction is performed to obtain multiple target feature vectors, where the target feature vectors represent the degree of data offset between the real-time monitoring data and the historical monitoring data.

[0079] The real-time monitoring data includes real-time sub-data corresponding to multiple dimensions. These dimensions can include the user's origin region, the domain name from which the user came, the user's stay time on the website, and the bounce rate.

[0080] Specifically, for each dimension, historical sub-data corresponding to the dimension is obtained, and the data offset between the historical sub-data and the real-time sub-data is determined. The historical sub-data includes historical data such as the time users spend on the website and bounce rate. The monitoring type is obtained, and the data offset corresponding to the monitoring type is used as vector elements to obtain the target feature vector corresponding to the monitoring type. The arrangement of vector elements in the target feature vector can be random or preset by technicians and stored in electronic devices. The monitoring type includes multiple dimensions.

[0081] For example, monitoring type 002 is the situation of a user visiting website A. The situation of visiting website A includes dimensions such as the source region, the search engine used by the user, and the time spent on the website. Historical sub-data corresponding to each dimension is obtained. For each dimension, the offset between historical sub-data and real-time sub-data is determined to obtain the data offset corresponding to that dimension. All data offsets corresponding to monitoring type 002 are combined to obtain the target feature vector corresponding to monitoring type 002.

[0082] S103. Obtain the target detection dimension, and based on the target detection dimension and the target feature vector, obtain the target feature matrix corresponding to the target detection dimension. The target feature matrix represents the degree of data offset between the current network data and the historical network data of the target detection dimension.

[0083] The target detection dimensions can be any dimension such as the user's origin region, the referring domain, the user's time spent on the website, and the bounce rate, which can be obtained by the terminal. For example, the terminal obtains the monitoring targets of its users and sends the monitoring targets to the electronic device. After receiving the monitoring targets, the electronic device processes them to obtain the target detection dimensions. Among them, the terminal's users represent users with security monitoring purposes, and the server's users represent users whose network usage security is being protected.

[0084] The target feature matrix includes target feature vectors corresponding to each of the multiple target detection dimensions. When the target feature vectors are column vectors, the target feature matrix is ​​a group of column vectors containing multiple target feature vectors; when the target feature vectors are row vectors, the target feature matrix is ​​a group of row vectors containing multiple target feature vectors.

[0085] It should be noted that the arrangement of target feature vectors in the target feature matrix can be random or can be preset by technicians and stored in electronic devices. This embodiment does not specifically limit this arrangement.

[0086] S104. Based on the target feature matrix and the historical feature matrix, obtain the real-time detection results corresponding to the target detection dimension, wherein the real-time detection results represent the abnormal situation of the real-time monitored data.

[0087] Specifically, the target feature matrix and the historical feature matrix are operated on based on a preset matrix processing method to obtain the matrix operation result. The preset matrix processing method can be the eigenvalue method or the determinant method, as long as it can determine whether the two matrices are similar. This embodiment does not specify it further.

[0088] The anomaly identification method based on security monitoring provided in this application obtains real-time monitoring data to obtain network data generated by the user terminal; it extracts features from the real-time monitoring data to obtain the extracted target feature vector, thereby determining the data offset between the real-time monitoring data and historical monitoring data, achieving the first compression of dynamic monitoring data; it obtains the target detection dimension, and based on the target detection dimension and the extracted target feature vector, it obtains the target feature matrix corresponding to the target detection dimension, thereby determining the data offset between the current network data and historical network data of the target detection dimension, achieving the second compression of dynamic monitoring data; based on the historical feature matrix and the target feature matrix obtained after the two compressions, it obtains the real-time detection result corresponding to the target detection dimension more quickly, thus achieving the beneficial effect of faster identification of anomalies.

[0089] In one possible implementation, the real-time monitoring data includes real-time sub-data corresponding to multiple dimensions. In step S102, feature extraction is performed based on the real-time monitoring data to obtain multiple target feature vectors. Specifically, this may include steps S1021 to S1022, wherein:

[0090] S1021. For each dimension, based on the historical sub-data corresponding to the dimension, obtain the data distribution information corresponding to the dimension, wherein the data distribution information represents the distribution of historical network data in different time periods.

[0091] Specifically, the data types of the aforementioned historical sub-data are identified to obtain the corresponding historical data types. The historical data types are numbers or strings, and numbers include integers and floating-point numbers. When the historical data type is a number, the data distribution information is obtained based on the normal distribution method. When the historical data type is a string, the data distribution information is obtained based on string classification.

[0092] When the historical sub-data is numerical, the data distribution information is a historical distribution map of the historical sub-data. When the historical sub-data is a string, the data distribution information includes feature string groups corresponding to multiple dimensions.

[0093] The data distribution information obtained based on the normal distribution method specifically includes: obtaining the historical sample size corresponding to the historical sub-data, and determining whether the historical sample size is greater than a preset sample size threshold, preferably 5000; if yes, then obtaining the distribution result based on the Kolmogorov-Smirnov test algorithm and the historical sub-data; if no, then obtaining the distribution result based on the Shapiro-Wilk test algorithm and the historical sub-data, wherein the distribution result is either yes or no; when the distribution result is yes, it indicates that the historical sub-data conforms to a normal distribution, and obtaining the normal distribution function corresponding to the historical sub-data based on the normal distribution function extraction algorithm, wherein the normal distribution function extraction algorithm includes, but is not limited to, calculating the expected value and standard deviation of the historical sub-data, and the independent variable and dependent variable corresponding to the normal distribution function are the specific values ​​of time and historical sub-data, respectively; obtaining the above normal distribution graph based on the normal distribution function, thereby obtaining the data distribution information.

[0094] Obtaining data distribution information based on string classification specifically includes: acquiring the historical time corresponding to the historical sub-data, and grouping the historical sub-data based on the historical time to obtain the historical data groups corresponding to different historical time periods; for each historical data group, performing keyword recognition on each historical sub-data to obtain the target keywords corresponding to the historical sub-data, wherein the keyword recognition can be achieved based on preset keywords and matching algorithms, and the preset keywords can be set information such as domain names; determining the string classification identifier corresponding to the target keywords, and using the string classification identifier as the feature string corresponding to the dimension of the aforementioned historical sub-data, thereby obtaining the data distribution information.

[0095] S1022. Based on the data distribution information and real-time sub-data, obtain the data offset, whereby the data offset represents the degree of data deviation of the real-time sub-data compared to the historical sub-data.

[0096] The data offset is a numerical value.

[0097] When the historical sub-data is numerical, obtain the real-time time corresponding to the real-time sub-data, and based on the real-time time and real-time sub-data, determine the real-time coordinates of the real-time sub-data in the normal distribution plot. Here, the real-time time represents the moment when the electronic device receives the real-time sub-data, and the real-time coordinates are: (real-time time, real-time sub-data). Based on the normal distribution curve corresponding to the normal distribution plot, obtain the midpoint of the area between the normal distribution curve and the horizontal axis. The midpoint of the area can be obtained through two-dimensional integration. Determine the straight-line distance between the midpoint of the area and the real-time coordinates, and use the above straight-line distance as the data offset. The straight-line distance can be obtained based on the coordinates corresponding to the midpoint of the area, the real-time coordinates, and the Euclidean distance (Euclidean metric) formula.

[0098] When the historical sub-data is a string, the data offset is determined based on the N-gram algorithm, data distribution information, and real-time sub-data, as detailed in S10221 and S10222.

[0099] S1023. For each monitoring type, obtain the target feature vector corresponding to the monitoring type based on the data offset corresponding to the monitoring type. The monitoring type includes several dimensions.

[0100] Specifically, the vector element arrangement rules corresponding to the target monitoring type are obtained. The vector element arrangement rules represent the arrangement order of multiple dimensions included in the monitoring type. The arrangement order can be random or in a fixed order, which is not specifically limited in this embodiment. Based on the vector element arrangement rules, the arrangement order corresponding to each dimension under the monitoring type is determined. According to the data offset and arrangement order corresponding to each dimension under the monitoring type, the target feature vector is obtained. The target feature vector includes the data offsets corresponding to the monitoring type and each of the multiple dimensions arranged according to the arrangement order.

[0101] The anomaly identification method based on security monitoring provided in this application determines the distribution of historical network data in different time periods by obtaining data distribution information based on historical sub-data; the data offset can be obtained based on the data distribution information and real-time sub-data, thereby determining the degree of data offset of real-time sub-data compared to historical sub-data; finally, a more accurate target feature vector can be obtained based on the data offset corresponding to each dimension included in the monitoring type.

[0102] In one possible implementation, when the historical sub-data is a string, the data distribution information includes feature string groups corresponding to multiple dimensions. In step S1022, based on the data distribution information and the real-time sub-data, the data offset is obtained, which may specifically include steps S10221 and S10222, wherein:

[0103] S10221. For each dimension, perform N-gram similarity calculation based on the real-time sub-data and the feature string group corresponding to the dimension to obtain multiple association similarities corresponding to the dimension. Among them, the association similarity represents the similarity between the feature string in the feature string group and the real-time sub-data.

[0104] Specifically, the N value is obtained, which can be 2 or 3, and the N value represents the length of the text segmentation unit. For example, if the string is "abcdef" and the N value is 2, then "abcdef" is segmented into "ab", "cd", and "ef". Based on the N value and the real-time sub-data, multiple real-time word segments corresponding to the real-time sub-data are obtained. Based on the N value and the feature string, multiple historical word segments corresponding to the feature string are obtained. For the feature string and the real-time sub-data, based on multiple real-time word segments, multiple historical word segments, and the Similarity calculation formula, the association similarity corresponding to each feature string is obtained.

[0105] Similarity calculation formula: Similarity=|GN(S)|+|GN(T)|-2*|GN(S)∩GN(T)|, where Similarity is the association similarity, |GN(S)| is the number of multiple real-time word segments, |GN(T)| is the number of multiple historical word segments, and |GN(S)∩GN(T)| is the number of word segments that are the same as those in multiple historical word segments and multiple real-time word segments.

[0106] S10222. Based on multiple association similarities, obtain the data offset, where the data offset represents the distribution characteristics of multiple association similarities.

[0107] When the historical sub-data is a string, the data offset is the average of the above-mentioned multiple association similarities.

[0108] It is important to note that related technologies determine whether there is an offset between real-time and historical sub-data by checking whether historical sub-data is identical to the historical sub-data. For example, when the URL of the same website changes due to web-related reasons, the real-time and historical sub-data may differ, leading to an error in determining the data offset.

[0109] The anomaly identification method based on security monitoring provided in this application determines the text similarity between the feature strings and real-time sub-data by obtaining multiple association similarities corresponding to the dimension based on the feature string groups corresponding to the fact sub-data and the dimension. Based on the above text similarity, a more accurate data offset can be obtained.

[0110] In one possible implementation, in S103 above, based on the target detection dimension and the target feature vector, the target feature matrix corresponding to the target detection dimension is obtained, which may specifically include S103a1 and S103a2, wherein:

[0111] S103a1. Obtain historical abnormal data corresponding to the target detection dimensions, and queue the target detection dimensions based on the historical abnormal data to obtain a target dimension queue, wherein the target dimension queue includes multiple target detection dimensions arranged according to the abnormal frequency.

[0112] Specifically, based on historical anomaly data, the anomaly frequency corresponding to each target detection dimension is determined. Based on these anomaly frequencies, the target detection dimensions are queued to obtain a target dimension queue. The earlier a target detection dimension appears in the queue, the higher its anomaly frequency.

[0113] S103a2. Based on the target dimension queue and the target feature vector, the target feature matrix is ​​obtained.

[0114] The target feature vector can be a row vector or a column vector. Specifically, when the target feature vector is a row vector, the anomaly frequency of the target feature vector in the target feature matrix decreases sequentially from top to bottom. When the target feature vector is a column vector, the anomaly frequency of the target feature vector in the target feature matrix decreases sequentially from left to right.

[0115] The anomaly identification method based on security monitoring provided in this application determines the historical anomaly situation of the target detection dimension by acquiring historical anomaly data corresponding to the target detection dimension; then, the target detection dimension is queued based on the historical anomaly situation to obtain a target dimension queue arranged according to the real-time anomaly frequency; then, the target feature vector is arranged based on the target dimension queue arranged according to the real-time anomaly frequency to obtain a target feature matrix that can perceive anomaly changes, thereby improving the accuracy of the target feature matrix.

[0116] In one possible implementation, in S103 above, based on the target detection dimension and the target feature vector, the target feature matrix corresponding to the target detection dimension is obtained, which may specifically include S10.3b1 and S103b2, wherein:

[0117] S103b1. Obtain the target time range, and determine the target feature vector set based on the target feature vector and the target time range, wherein the target feature vector set includes several target feature vectors within the target time range.

[0118] The target time range represents a time range determined based on a preset unit of time. For example, if the preset unit of time is 5 seconds and the current time is 11:50:22, then the target time range is from 11:50:17 to 11:50:22.

[0119] It is important to note that the target feature vector represents the degree of data deviation between the real-time monitored data and the historical monitored data. Furthermore, the real-time monitored data is constantly being updated, and therefore, the historical monitored data is also constantly being updated. Therefore, by determining the target time range and then identifying the target feature vector within that time range from all target feature vectors, the stability of the target feature vector comparison process can be ensured.

[0120] Real-time monitored data refers to network data generated by the server in real time, and the time corresponding to the real-time monitored data is the time when the electronic device receives the real-time monitored data. The target feature vector is obtained based on the real-time monitored data corresponding to the same time, and the target feature matrix includes target feature vectors that conform to the target time range.

[0121] S103b2. Based on the target feature vector set, obtain the target feature matrix.

[0122] The target feature matrix includes multiple sets of target feature vectors arranged in chronological order.

[0123] Specifically, the vector time value corresponding to each target feature vector in the target feature vector set is obtained, where the time value represents the time when the electronic device receives the earliest real-time data to be monitored in the target feature vector; based on all the vector time values ​​corresponding to the target feature vector set, the vector set time is obtained, where the vector set time is the latest vector time value; all target feature vector sets corresponding to this dimension are arranged based on the vector set time to obtain the target feature matrix, where the multiple target feature vector sets in the target feature matrix have progressively later vector set times from front to back.

[0124] The anomaly identification method based on security monitoring provided in this application determines the time range for comparison by obtaining a target time range, where the comparison refers to the comparison between historical network data and current network data; and in the real-time data that can be invoked...

[0125] In one possible implementation, S104 above, based on the target feature matrix and the historical feature matrix, obtains the real-time detection result corresponding to the target detection dimension, specifically including S1041 to S1044, wherein:

[0126] S1041. Obtain the target feature values ​​corresponding to the target feature matrix.

[0127] S1042. Obtain the historical feature values ​​corresponding to the historical feature matrix.

[0128] The methods for obtaining the target feature values ​​and historical feature values ​​include, but are not limited to, the power method, the inverse power method, the QR method, the Jacobi method, the Givens rotation method, the Householder transformation method, and the Lanczos method. It should be noted that the method corresponding to the target feature value and the method corresponding to the historical feature value can be the same or different; this embodiment does not specifically limit the methods.

[0129] S1043. Based on the historical feature values ​​and the target feature values, the initial detection results are obtained, wherein the initial detection results represent the similarity relationship between the target feature matrix and the historical feature matrix.

[0130] Specifically, it determines whether the historical feature values ​​and the target feature values ​​are the same; if so, it indicates that the target feature matrix is ​​normal compared to the historical feature matrix, and the initial detection result is determined to be matrix similar; otherwise, it indicates that the target feature matrix is ​​abnormal compared to the historical feature matrix, and the initial detection result is determined to be matrix dissimilar.

[0131] S1044. Based on multiple initial detection results, a real-time detection result is obtained, wherein the real-time detection result represents the central tendency of multiple initial detection results.

[0132] Specifically, based on the multiple initial detection results, the number of similar and dissimilar items are obtained. The number of similar items is the number of matrices that are similar according to the multiple initial detection results, and the number of dissimilar items is the number of matrices that are dissimilar according to the multiple initial detection results. Based on the number of similar and dissimilar items, a quantitative relationship is obtained, which represents the size relationship between the number of similar and dissimilar items. The quantitative relationship is greater than or equal to or less than. Greater than or equal to indicates that the number of similar items is large, and less than indicates that the number of dissimilar items is large. When the quantitative relationship is greater than or equal to, the real-time detection result is determined to be normal; when the quantitative relationship is less than, the real-time detection result is determined to be abnormal.

[0133] Determine the median of the above initial detection results, and determine whether the median of the above results exceeds the preset result threshold. If it does, the real-time detection result is determined to be abnormal; otherwise, the real-time detection result is determined to be normal.

[0134] The following example illustrates the anomaly identification method based on security monitoring provided in the above embodiments. Figure 3 As shown, it includes the following steps:

[0135] S201. Obtain multiple real-time sub-data, wherein the multiple real-time sub-data represent network data generated by the user terminal corresponding to multiple dimensions.

[0136] S202. For each dimension, based on the historical sub-data corresponding to the dimension, obtain the data distribution information corresponding to the dimension, and obtain the feature string group corresponding to each of the multiple dimensions based on the data distribution information. The data distribution information represents the distribution of historical network data in different time periods.

[0137] S203. Based on the N-gram algorithm, the feature string group corresponding to the real-time sub-data and the dimension, the data offset is obtained, where the data offset represents the degree of data offset of the real-time sub-data compared with the historical sub-data.

[0138] Specifically, for each dimension, N-gram similarity is calculated based on the real-time sub-data and the feature string group corresponding to the dimension to obtain multiple association similarities for the dimension. The association similarity represents the similarity between the feature string in the feature string group and the real-time sub-data. Based on the multiple association similarities, the data offset is obtained, where the data offset represents the distribution characteristics of the multiple association similarities.

[0139] S204. For each monitoring type, obtain the target feature vector corresponding to the monitoring type based on the data offset corresponding to the monitoring type. The monitoring type includes several dimensions, and the target feature vector represents the degree of data offset between the real-time monitored data and the historical monitored data.

[0140] S205. Obtain the target detection dimension, and based on the target detection dimension and the target feature vector, obtain the target feature matrix corresponding to the target detection dimension. The target feature matrix represents the degree of data offset between the current network data and the historical network data of the target detection dimension.

[0141] In one possible implementation, the target feature matrix corresponding to the target detection dimension is obtained based on the target detection dimension and the target feature vector. Specifically, this includes: obtaining historical abnormal data corresponding to the target detection dimension, and queuing the target detection dimension based on the historical abnormal data to obtain a target dimension queue, wherein the target dimension queue includes multiple target detection dimensions arranged according to abnormal frequency; and obtaining the target feature matrix based on the target dimension queue and the target feature vector.

[0142] In another possible implementation, the target feature matrix corresponding to the target detection dimension is obtained based on the target detection dimension and the target feature vector. Specifically, this includes: obtaining the target time range, and determining the target feature vector set according to the target feature vector and the target time range, wherein the target feature vector set includes several target feature vectors within the target time range; and obtaining the target feature matrix according to the target feature vector set.

[0143] In another possible implementation, the target feature matrix corresponding to the target detection dimension is obtained based on the target detection dimension and the target feature vector. Specifically, this includes: obtaining the target time range and determining the element feature matrix based on the target feature vector and the target time range, wherein the element feature matrix includes several target feature vectors within the target time range; obtaining historical abnormal data corresponding to the target detection dimension and queuing the target detection dimension based on the historical abnormal data to obtain a target dimension queue, wherein the target dimension queue includes multiple target detection dimensions arranged according to abnormal frequency; and obtaining the target feature matrix based on the target dimension queue and the element feature matrix.

[0144] S206. Obtain the feature values ​​corresponding to the target feature matrix and the historical feature matrix respectively, and obtain the initial detection result corresponding to each historical matrix based on the above feature values. The initial detection result represents the similarity relationship between the target feature matrix and the historical feature matrix.

[0145] S207. Based on multiple initial detection results, a real-time detection result is obtained, wherein the real-time detection result represents the abnormal situation of the real-time monitored data, and the real-time detection result represents the central tendency of multiple initial detection results.

[0146] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0147] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0148] Figure 4 A schematic diagram of the anomaly detection device based on security monitoring provided in this application is shown below. Figure 4 As shown, the anomaly identification device 40 based on security monitoring provided in this embodiment includes:

[0149] The acquisition module 401 is used to acquire real-time monitoring data, wherein the real-time monitoring data represents network data generated by the user terminal;

[0150] The first processing module 402 is used to extract features based on real-time monitoring data to obtain multiple target feature vectors, wherein the target feature vectors represent the degree of data offset between real-time monitoring data and historical monitoring data.

[0151] The second processing module 403 is used to obtain the target detection dimension and, based on the target detection dimension and the target feature vector, obtain the target feature matrix corresponding to the target detection dimension, wherein the target feature matrix represents the degree of data offset between the current network data and the historical network data of the target detection dimension.

[0152] Analysis module 404 is used to obtain real-time detection results corresponding to the target detection dimension based on the target feature matrix and the historical feature matrix. The real-time detection results represent the abnormal situation of the real-time monitored data.

[0153] Optionally, the real-time monitoring data includes real-time sub-data corresponding to multiple dimensions. The first processing module 402, when performing feature extraction based on the real-time monitoring data to obtain multiple target feature vectors, is used to:

[0154] For each dimension, based on the historical sub-data corresponding to the dimension, the data distribution information corresponding to the dimension is obtained. The data distribution information represents the distribution of historical network data in different time periods.

[0155] Based on the data distribution information and real-time sub-data, the data offset is obtained, where the data offset represents the degree of data deviation of the real-time sub-data compared with the historical sub-data.

[0156] For each monitoring type, the target feature vector corresponding to the monitoring type is obtained based on the data offset corresponding to the monitoring type. The monitoring type includes several dimensions.

[0157] Optionally, the data distribution information includes feature string groups corresponding to multiple dimensions. The first processing module 402, when executing the process of obtaining the data offset based on the data distribution information and real-time sub-data, is used to:

[0158] For each dimension, N-gram similarity is calculated based on the real-time sub-data and the feature string group corresponding to the dimension to obtain multiple association similarities for the dimension. The association similarity represents the similarity between the feature string in the feature string group and the real-time sub-data.

[0159] Based on multiple association similarities, the data offset is obtained, where the data offset represents the distribution characteristics of multiple association similarities.

[0160] Optionally, the second processing module 403, when performing the process of obtaining the target feature matrix corresponding to the target detection dimension based on the target detection dimension and the target feature vector, is used to:

[0161] Obtain historical anomaly data corresponding to the target detection dimensions, and queue the target detection dimensions based on the historical anomaly data to obtain the target dimension queue, wherein the target dimension queue includes multiple target detection dimensions arranged according to anomaly frequency;

[0162] Based on the target dimension queue and the target feature vector, the target feature matrix is ​​obtained.

[0163] Optionally, the second processing module 403, when performing the process of obtaining the target feature matrix corresponding to the target detection dimension based on the target detection dimension and the target feature vector, is used to:

[0164] Obtain the target time range, and determine the target feature vector set based on the target feature vector and the target time range, wherein the target feature vector set includes several target feature vectors within the target time range;

[0165] Based on the target feature vector set, the target feature matrix is ​​obtained.

[0166] Optionally, the analysis module 404, when executing the real-time detection results corresponding to the target detection dimension based on the target feature matrix and the historical feature matrix, is used to:

[0167] Obtain the target feature values ​​corresponding to the target feature matrix;

[0168] Obtain the historical feature values ​​corresponding to the historical feature matrix;

[0169] Based on historical feature values ​​and target feature values, an initial detection result is obtained, where the initial detection result represents the similarity relationship between the target feature matrix and the historical feature matrix;

[0170] Based on multiple initial detection results, real-time detection results are obtained, whereby the real-time detection results characterize the central tendency of the multiple initial detection results.

[0171] The anomaly identification device based on security monitoring provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0172] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0173] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0174] When an integrated unit / module is implemented in hardware, that hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc.

[0175] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0176] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0177] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0178] Unless otherwise specified, processor 501 can be any suitable hardware processor, such as CPU, GPU, FPGA, DSP, and ASIC. Unless otherwise specified, memory 502 can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0179] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0180] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the above-described method for updating redundant fields in a distributed system.

[0181] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for updating redundant fields in a distributed system.

[0182] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not contradict each other, they should be considered within the scope of this specification.

[0183] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0184] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for identifying an anomaly based on security monitoring, characterized by, The method comprises: acquiring real-time monitoring data, wherein the real-time monitoring data represents network data generated by a user terminal, and the real-time monitoring data comprises a plurality of real-time sub-data corresponding to respective dimensions; performing feature extraction based on the real-time monitoring data to obtain a plurality of target feature vectors, comprising: for each dimension, obtaining data distribution information corresponding to the dimension based on historical sub-data corresponding to the dimension, wherein the data distribution information represents the distribution of historical network data in different time periods; obtaining a data offset based on the data distribution information and the real-time sub-data, wherein the data offset represents the data offset degree of the real-time sub-data compared with historical sub-data; for each monitoring type, obtaining a target feature vector corresponding to the monitoring type based on the data offset corresponding to the monitoring type, wherein the monitoring type comprises a plurality of dimensions, and the target feature vector represents the data offset degree of real-time monitoring data compared with historical monitoring data; acquiring a target detection dimension, and obtaining a target feature matrix corresponding to the target detection dimension based on the target detection dimension and the target feature vector, wherein the target feature matrix represents the data offset degree between the current network data of the target detection dimension and the historical network data; obtaining a real-time detection result corresponding to the target detection dimension based on the target feature matrix and a historical feature matrix, wherein the real-time detection result represents the abnormal situation of real-time monitoring data.

2. The method of claim 1, wherein, The data distribution information comprises a plurality of feature string groups corresponding to respective dimensions, obtaining a data offset based on the data distribution information and the real-time sub-data, comprising: for each dimension, performing N-gram similarity calculation based on the real-time sub-data and the feature string group corresponding to the dimension to obtain a plurality of correlation similarities corresponding to the dimension, wherein the correlation similarity represents the similarity between the feature string in the feature string group and the real-time sub-data; obtaining the data offset based on the plurality of correlation similarities, wherein the data offset represents the distribution characteristics of the plurality of correlation similarities. 3.The security monitoring based anomaly identification method of claim 1, wherein, obtaining a target feature matrix corresponding to the target detection dimension based on the target detection dimension and the target feature vector, comprising: acquiring historical abnormal data corresponding to the target detection dimension, and queuing the target detection dimension based on the historical abnormal data to obtain a target dimension queue, wherein the target dimension queue comprises a plurality of target detection dimensions arranged according to abnormal frequency; obtaining the target feature matrix based on the target dimension queue and the target feature vector. 4.The security monitoring based anomaly identification method of claim 1, wherein, obtaining a target feature matrix corresponding to the target detection dimension based on the target detection dimension and the target feature vector, comprising: acquiring a target time range, and determining a target feature vector set based on the target feature vector and the target time range, wherein the target feature vector set comprises a plurality of target feature vectors within the target time range; obtaining the target feature matrix based on the target feature vector set.

5. The method of claim 1, wherein, obtaining real-time detection results corresponding to the target detection dimensions based on the target feature matrix and the historical feature matrix, including: obtaining target feature values corresponding to the target feature matrix; obtaining historical feature values corresponding to the historical feature matrix; obtaining initial detection results according to the historical feature values and the target feature values, wherein the initial detection results represent the similarity between the target feature matrix and the historical feature matrix; obtaining the real-time detection results according to a plurality of the initial detection results, wherein the real-time detection results represent the central tendency of the plurality of the initial detection results.

6. An abnormality identification device based on security monitoring, characterized by, including: an acquisition module, configured to acquire real-time monitoring data, wherein the real-time monitoring data represent network data generated by a user end, and the real-time monitoring data include real-time sub-data corresponding to a plurality of dimensions respectively; a first processing module, configured to perform feature extraction according to the real-time monitoring data, and obtain a plurality of target feature vectors, wherein the target feature vectors represent the data offset degree of the real-time monitoring data compared with historical monitoring data; a second processing module, configured to obtain target detection dimensions, and obtain a target feature matrix corresponding to the target detection dimensions based on the target detection dimensions and the target feature vectors, including: for each of the dimensions, obtaining data distribution information corresponding to the dimension according to historical sub-data corresponding to the dimension, wherein the data distribution information represents the distribution of historical network data in different time periods; obtaining data offset amounts according to the data distribution information and the real-time sub-data, wherein the data offset amounts represent the data offset degree of the real-time sub-data compared with historical sub-data; and for each monitoring type, obtaining a target feature vector corresponding to the monitoring type according to data offset amounts corresponding to the monitoring type, wherein the monitoring type includes a plurality of dimensions, and the target feature matrix represents the data offset degree between current network data of the target detection dimensions and historical network data. an analysis module, configured to obtain real-time detection results corresponding to the target detection dimensions based on the target feature matrix and the historical feature matrix, wherein the real-time detection results represent abnormal conditions of the real-time monitoring data.

7. An electronic device, comprising: including: a processor, and a memory connected with the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method according to any one of claims 1 to 5.

9. A computer program product, including a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images