High-frequency file reading monitoring system based on log analysis

Abstract

The present invention relates to the fields of log analysis and information security, and in particular to a high-frequency file reading monitoring system based on log analysis. The method comprises: constructing an access risk context through log collection and auxiliary library scoring, and performing behavioral feature extraction and normalization processing to obtain a structured file reading behavior instance; introducing a static risk score and an exponential adjustment factor to perform context-aware distance optimization processing to obtain a static risk optimization factor; constructing a dynamic pattern risk score and an exponential amplification factor to perform feature vector behavioral pattern analysis to obtain a dynamic risk optimization factor; and fusing the static and dynamic risk optimization factors to perform a final optimized distance calculation processing to obtain a fully risk-weighted behavioral instance similarity, and applying the optimized distance-based K-Means clustering algorithm to perform behavioral pattern recognition and anomaly analysis, thereby improving the security monitoring capability for high-frequency access behaviors in large-scale systems.

Description

Technical Field

[0001] The present invention relates to the technical field of log analysis and information security, and in particular to a high-frequency file reading monitoring system based on log analysis. Background Art

[0002] In various information systems, files are the storage medium for core information such as configuration, data, and code. Effectively monitoring access to these files is fundamental to ensuring system security, conducting compliance checks, and diagnosing operational issues. Access logs generated during system operation detail information such as the time a file was accessed, the identity of the accessor, the program used, and the specific file path, providing raw data for monitoring and analysis.

[0003] Analyzing log data to identify unusual file access patterns is a common practice in information security and system operations. Frequent file reads, particularly those that occur frequently, are a key focus of monitoring and analysis, as they can indicate normal business operations or potential risks (data theft attempts, malicious probing). Accurately distinguishing between these two distinct types of high-frequency reads is crucial for improving the effectiveness of monitoring systems.

[0004] To address this challenge, data clustering methods are often used in existing technologies, such as The general process of this method is: first, extract the features that can quantitatively describe the reading behavior from the log data and construct a multi-dimensional feature vector; then apply The algorithm divides the reading behavior instances into different clusters based on the similarity between the feature vectors; finally, by interpreting the formed clusters, it is expected to distinguish between pattern clusters representing normal operations and pattern clusters indicating potential anomalies. Within the algorithm framework, the standard way to measure the similarity between two behavioral feature vectors is to calculate the standard Euclidean distance between them. This distance metric reflects the geometric proximity of two vectors in the feature space, that is, the numerical similarity.

[0005] However, in the scenario of monitoring high-frequency file reading, it is not feasible to directly apply the standard Euclidean distance Clustering suffers from a core technical flaw: the standard Euclidean distance, when measuring the similarity of file read behaviors, relies entirely on the numerical differences in feature vectors, inherently failing to incorporate contextual information crucial for risk assessment. This inherent lack of risk context manifests itself in several ways: it cannot determine the inherent sensitivity of a file based on its path (access to critical system files should be treated differently from access to ordinary log files); it cannot assess the trustworthiness of a visitor using user identity or process origin information (the behavior of a known service process carries different risks than that of an unknown script); and it struggles to distinguish between differences in risk implied by feature combination patterns (the same high-frequency read behavior carries significantly different risks from a stable single-service model versus a decentralized, multi-user detection model).

[0006] Since the standard Euclidean distance does not include the key information that determines the risk level, the distance it calculates only reflects the geometric proximity of the behaviors in terms of apparent numerical characteristics, but cannot accurately reflect the actual potential risk gap between the two reading behaviors. The direct consequence is The clustering process cannot effectively distinguish between high-frequency benign reads and high-frequency reads that are potentially anomalous or malicious. This is because behaviors with very different risk connotations are mistakenly grouped together due to similar feature values. This severely impacts the accuracy and reliability of subsequent anomaly detection, making the entire monitoring system prone to false positives and false negatives, and failing to meet the requirements of precise security monitoring. Therefore, an improved distance metric is urgently needed that can overcome the limitations of the standard Euclidean distance and effectively incorporate key risk context into the similarity assessment of file read behaviors. Summary of the Invention

[0007] In view of this, the present invention aims to propose a high-frequency file reading monitoring system based on log analysis.

[0008] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0009] A high-frequency file reading monitoring system based on log analysis includes the following steps:

[0010] Step S1: Construct access risk context through log collection and auxiliary library scoring, and perform behavioral feature extraction and normalization processing to obtain structured file reading behavior instances;

[0011] Step S2: By introducing a static risk score and an exponential adjustment factor, context-aware distance optimization is performed to obtain a static risk optimization factor;

[0012] Step S3: By constructing a dynamic model risk score and an index amplification factor, characteristic vector behavior pattern analysis is performed to obtain a dynamic risk optimization factor;

[0013] Step S4: By fusing the static and dynamic risk optimization factors, the final optimized distance calculation is performed to obtain the behavior instance similarity after comprehensive risk weighting;

[0014] Step S5: By applying the K-Means clustering algorithm based on optimized distance to perform behavior pattern recognition and anomaly analysis, risk warning information of high-frequency file reading is obtained.

[0015] Furthermore, according to the access risk context constructed through log collection and auxiliary library scoring and behavioral feature extraction and normalization processing, a structured file reading behavior instance is obtained, which specifically includes:

[0016] The log agent deployed in the target information system continuously collects file access audit log data. The collected file access audit log entries include timestamp, accessed file path, user ID performing the access operation, process ID initiating the access, and operation type.

[0017] In the central log processing system, log data undergoes preliminary screening, retaining only log records with the "read" operation type. Simultaneously, the system loads a sensitive file path pattern library and a visitor credibility information library. The sensitive file path pattern library contains sensitive paths defined by regular expressions and their corresponding sensitivity scores, while the visitor credibility information library records the credibility scores corresponding to user IDs and process IDs.

[0018] After completing log screening and auxiliary library loading, the system divides the "read" type logs into non-overlapping time windows of fixed length in chronological order. In each time window, the system aggregates the unique file paths that have been read. The aggregation process includes counting the number of reads of the path, the number of independent user IDs, the number of independent process IDs, and calculating the standard deviation of the time intervals of all read operation timestamps; the multidimensional features in the aggregation result are used as the original feature vector of the file in the time window; the original feature vector of the file in the time window is normalized to obtain a normalized feature vector;

[0019] For each file and each time window, the normalized feature vector of the file in the time window, the file path, and the user information and process information used to assess the visitor risk are used as structured file reading behavior instances.

[0020] Furthermore, by introducing the static risk score and the exponential adjustment factor, the context-aware distance optimization process is performed to obtain the static risk-weighted behavioral distance, which specifically includes:

[0021] Performing static risk assessment on the structured file reading behavior instance through file sensitivity score and visitor credibility score to obtain the first static risk assessment factor;

[0022] By performing relative risk assessment on structured file reading behavior instances, a static risk optimization factor is obtained.

[0023] Furthermore, a static risk assessment is performed on the structured file reading behavior instance based on the file sensitivity score and the visitor credibility score to obtain a first static risk assessment factor, which specifically includes:

[0024] Obtain a structured file reading behavior instance, and for the structured file reading behavior instance, read the accessed file path in the instance, the visitor and process information of the operator performing the operation; obtain the set file sensitivity scoring function and visitor credibility scoring function; input the visitor and process information of the operator performing the operation in the structured file reading behavior instance into the visitor credibility scoring function, and obtain the output as the visitor credibility of the instance; input the file path in the structured file reading behavior instance into the file sensitivity scoring function, and use the corresponding output as the file sensitivity of the instance; multiply the result of the calculation obtained by subtracting the constant 1 from the visitor credibility of the instance by the file sensitivity, and use the corresponding calculation result as the first static risk assessment factor.

[0025] Furthermore, by performing relative risk assessment on the structured file reading behavior instances, a static risk optimization factor is obtained, specifically including:

[0026] Obtain any two structured file reading behavior instances, and use the larger first static risk assessment factor among the first static risk assessment factors corresponding to the any two structured file reading behavior instances as the static risk assessment of the two structured file reading behavior instances; and use the mapping result of the exponential function mapping with the natural constant e as the base for the static risk assessment as the static risk optimization factor.

[0027] Furthermore, by constructing the dynamic model risk score and the index amplification factor, characteristic vector behavior pattern analysis is performed to obtain a dynamic risk optimization factor, which specifically includes:

[0028] By evaluating the access source dispersion and behavior instability based on the normalized feature vector, a dynamic pattern risk analysis is performed on the structured file reading behavior instance to obtain the first dynamic pattern risk assessment factor;

[0029] By performing dynamic risk comparison and evaluation between structured file reading behavior instances, a dynamic risk optimization factor is obtained.

[0030] Furthermore, according to the access source dispersion and behavior instability assessment based on the normalized feature vector, a dynamic pattern risk analysis is performed on the structured file reading behavior instance to obtain a first dynamic pattern risk assessment factor, which specifically includes:

[0031] Obtain a normalized feature vector of a structured file reading behavior instance, extract the feature dimension that is preset to represent the number of independent access users and the feature dimension that represents the number of independent access processes in the normalized feature vector, and assign them as the user access degree feature value and the process access degree feature value respectively; obtain a weighting coefficient for calculating the access source dispersion, multiply the user access degree feature value and the process access degree feature value by the corresponding weighting coefficient respectively, and sum the products and divide them by the sum of the weighting coefficients, and the obtained value is used as the access source dispersion of the instance; at the same time, extract the feature dimension that is preset to represent the standard deviation of the reading frequency from the normalized feature vector, assign it as the behavioral instability feature value, and use the feature value as the behavioral instability of the instance; multiply the access source dispersion and the behavioral instability, and use the corresponding calculation result as the first dynamic mode risk assessment factor.

[0032] Furthermore, according to the dynamic risk comparison and evaluation between the structured file reading behavior instances, a dynamic risk optimization factor is obtained, which specifically includes:

[0033] Obtain any two structured file reading behavior instances, and use the larger first dynamic mode risk assessment factor among the first dynamic mode risk assessment factors corresponding to the two structured file reading behavior instances as the dynamic risk assessment value of the two structured file reading behavior instances; and use the mapping result of the dynamic risk assessment value using an exponential function with the natural constant e as the base as the dynamic risk optimization factor.

[0034] Furthermore, the final optimized distance calculation process is performed by fusing the static and dynamic risk optimization factors to obtain the fully risk-weighted behavior instance similarity, specifically including:

[0035] Obtaining any two structured file reading behavior instances, and calculating a result of multiplying the static risk optimization factor and the dynamic risk optimization factor of the two structured file reading behavior instances as the distance optimization factor of the two structured file reading behavior instances;

[0036] Obtaining normalized feature vectors of the two structured file reading behavior instances, and using the Euclidean distance between the normalized feature vectors of the two structured file reading behavior instances as a basic distance of the two structured file reading behavior instances;

[0037] A result of multiplying the distance optimization factor of the two structured file reading behavior instances by the basic distance of the two structured file reading behavior instances is used as the behavior instance similarity after comprehensive risk weighting.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] The high-frequency file reading monitoring system based on log analysis described in the present invention, by constructing a structured file reading behavior instance and introducing static risk factors and dynamic risk factors on this basis to perform dual optimization on the behavior similarity measurement, can fully integrate the importance of the file itself and the dynamic characteristics of the access behavior, thereby achieving more accurate identification of potential abnormal high-frequency reading behavior. The present invention first quantifies the static sensitivity of the file and the credibility of the visitor through the sensitive path scoring function and the visitor credibility scoring function, and constructs a static risk assessment factor and an exponential static risk optimization factor, which makes up for the defect of traditional distance measurement that ignores the static risk background; further, the present invention forms a dynamic pattern risk score by combining the dispersion of access sources and the instability of access frequency in the behavior feature vector, and constructs a corresponding dynamic risk optimization factor, thereby effectively capturing the "dynamic abnormal pattern hidden under the data structure that represents normal behavior", significantly improving the ability to identify complex attack methods or abnormal system operations. By jointly applying the static risk optimization factor and the dynamic risk optimization factor to the standard Euclidean distance calculation process, the present invention realizes a risk-awareness enhanced distance measurement method. This method has higher discrimination and clustering stability in K-Means cluster analysis, effectively reducing phenomena such as misclustering and missed judgments. At the same time, the present invention can significantly improve the security monitoring capability for high-frequency access behaviors in large-scale systems by automatically identifying and alerting high-risk clusters, loose clusters and outlier instances. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which constitute part of the present invention, are provided to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are provided to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0041] Figure 1 This is a flow chart of a method for a high-frequency file reading monitoring system based on log analysis according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0043] In the description of the present invention, it should be noted that the terms "upper," "lower," "inner," and "back" and other terms indicating orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0044] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0045] See also Figure 1 , is a method flow chart of a high-frequency file reading monitoring system based on log analysis provided by the first embodiment of the present invention, such as Figure 1 As shown, a high-frequency file reading monitoring system based on log analysis may include:

[0046] S1, through log collection and auxiliary library scoring, builds access risk context and performs behavioral feature extraction and normalization to obtain structured file reading behavior instances.

[0047] The log agent deployed in the target information system continuously collects file access audit log data. The collected file access audit log entries include timestamp, accessed file path, user ID performing the access operation, process ID initiating the access, and operation type.

[0048] In the central log processing system, log data undergoes preliminary screening, retaining only log records with the "read" operation type. Simultaneously, the system loads a sensitive file path pattern library and a visitor credibility information library. The sensitive file path pattern library contains sensitive paths defined by regular expressions and their corresponding sensitivity scores, while the visitor credibility information library records the credibility scores corresponding to user IDs and process IDs.

[0049] Among them, the sensitive file path pattern library contains a series of predefined regular expressions for matching sensitive file or directory paths, and associates a fixed sensitivity score with each pattern. In this embodiment, the library contains the following example rules and scores: "^ / etc / shadow$" ->1.0, "^ / etc / passwd$" ->0.9, ".*\\.key$" ->0.8, "^ / var / log / secure$" ->0.7, " / database_secrets / .*" ->0.9, other default paths ->0.1. Based on this library, a file sensitivity scoring function is implemented: for the input file path, the function matches the regular expressions in the library in priority order and returns the sensitivity score corresponding to the first successful matching rule; if no rule is matched, the default score of 0.1 is returned. The specific path score is determined according to the actual usage scenario and is not required.

[0050] The visitor credibility information library integrates user trust level information and process trust level information. The user trust level information stores the mapping from user ID to trust score. In this embodiment, the visitor credibility is set as root -> 0.2, service_backup -> 0.9, user_admin -> 0.4, and other ordinary users -> 0.5. The process trust level information stores the mapping from process ID proc to trust score ( / usr / sbin / sshd -> 0.9, / usr / bin / backup_agent -> 0.9, / bin / bash -> 0.6, / tmp / .* (matches processes executed under / tmp) -> 0.1). Based on this library, the visitor credibility score function is obtained: for the input username and process name , which looks up the corresponding user trust score and process trust score respectively, and then calculates the final visitor credibility score The minimum of the two values. This strategy ensures that the trustworthiness of the combination is determined by the least trustworthy component. The specific score is also determined by the actual usage scenario and is not required.

[0051] After completing log screening and auxiliary library loading, the system divides the "read" type logs into non-overlapping time windows of fixed length in chronological order. In each time window, the system aggregates the unique file paths that have been read. The aggregation process includes counting the number of reads of the path, the number of independent user IDs, the number of independent process IDs, and calculating the standard deviation of the time intervals of all read operation timestamps; the multidimensional features in the aggregation result are used as the original feature vector of the file in the time window; the original feature vector of the file in the time window is normalized to obtain a normalized feature vector;

[0052] For each file and each time window, the normalized feature vector of the file in the time window, the file path, and the user information and process information used to assess the visitor risk are used as structured file reading behavior instances.

[0053] S2, by introducing static risk score and exponential adjustment factor, performs context-aware distance optimization processing to obtain static risk optimization factor.

[0054] In the example of computing the two file reading behaviors by using the standard Euclidean distance (denoted as and )’s eigenvector (denoted as and ), only the geometric proximity of these vectors in the numerical feature space is considered. However, this calculation method completely ignores the static contextual information in which the behavior occurs, which is crucial for assessing potential risks. Specifically, standard distance fails to reflect the sensitivity of the accessed file itself and the trustworthiness of the user and process performing the read operation. Even if the numerical characteristics of a read of a system password file, such as the number of reads and the number of accessing users, are exactly the same as those of a read of a common application log file, the inherent security risk level is significantly higher. Similarly, a high-frequency read performed by a known and trusted system backup service has completely different risk implications from a high-frequency read with the same numerical characteristics performed by a script from an unknown source running in a temporary directory. Standard Euclidean distance fails to capture these key differences based on static context, resulting in behaviors with very different risks being incorrectly clustered together.

[0055] To address the blind spot of the standard Euclidean distance in the context of static risk, this step first aims to calculate a static risk score This score is designed to quantify the inherent risk level of each file read behavior instance, independent of short-term behavior patterns. The calculation of this score requires two key types of contextual information, which are obtained from the log record itself and the associated knowledge base:

[0056] File path information : Used to determine the sensitivity level of the accessed file. Each file path is assigned a sensitivity score by matching or querying with a predefined sensitive file path pattern library (including rules for key operating system directories, key storage locations, directories storing user privacy data, etc.). This score reflects the importance or confidentiality of the document's content.

[0057] Visitor information (user ID and process ID ): Used to evaluate the trustworthiness of the entity performing the read operation. By querying the user permission database and the known process reputation library (for example, marking known system services, trusted third-party applications, potential malware or unknown processes), a trustworthiness score can be assigned to the combination of user and process. . This rating reflects the reliability of the access source.

[0058] Based on the obtained file sensitivity score and visitor credibility score, a comprehensive static risk score is calculated The calculation logic follows a core principle: when the two conditions of accessing high-sensitivity files and low-trust visitors are met or tend to be met at the same time, the static risk score should increase significantly. Conversely, when accessing low-sensitivity files or operations performed by high-trust visitors, the static risk score is lower. This design ensures Able to effectively quantify the basic risk level determined by static background factors.

[0059] When getting each behavior instance and Static risk score and Afterwards, the goal of this step is to calculate a static risk adjustment factor , this factor will be used to adjust and The basic Euclidean distance between .

[0060] Specifically, first, a static risk assessment is performed on the structured file reading behavior instance using the file sensitivity score and the visitor credibility score to obtain the first static risk assessment factor;

[0061] Then, a static risk optimization factor is obtained by performing relative risk assessment on the structured file reading behavior instances.

[0062] The static risk assessment of the structured file reading behavior instance is performed using the file sensitivity score and the visitor credibility score to obtain the first static risk assessment factor, which specifically includes:

[0063] Obtain a structured file reading behavior instance, and for the structured file reading behavior instance, read the accessed file path in the instance, the visitor and process information of the operator performing the operation; obtain the set file sensitivity scoring function and visitor credibility scoring function; input the visitor and process information of the operator performing the operation in the structured file reading behavior instance into the visitor credibility scoring function, and obtain the output as the visitor credibility of the instance; input the file path in the structured file reading behavior instance into the file sensitivity scoring function, and use the corresponding output as the file sensitivity of the instance; multiply the result of the calculation obtained by subtracting the constant 1 from the visitor credibility of the instance by the file sensitivity, and use the corresponding calculation result as the first static risk assessment factor.

[0064] In one embodiment, assuming that The visitor credibility of a structured file reading behavior instance is ;No. The file sensitivity of a structured file reading behavior instance is , then The calculation expression of the first static risk assessment factor of a structured file reading behavior instance is:

[0065]

[0066] in, Indicates the A first static risk assessment factor for a structured file reading behavior instance; Indicates the The file sensitivity of a structured file reading behavior instance; Indicates the The visitor's credibility for a structured file reading behavior instance.

[0067] It should be noted that this design (The range is ), can directly quantify the baseline risk level determined by static context, as its calculation logic ensures that the risk score only increases significantly when a highly sensitive file is accessed by a low-trust accessor. This directly compensates for the lack of this key information in the standard Euclidean distance.

[0068] After quantifying the static risk score of each behavior instance These scores can then be used to construct a static risk optimization factor for adjusting the distance. Specifically, by performing a relative risk assessment between instances of structured file reading behavior, the static risk optimization factor is obtained, which includes:

[0069] Obtain any two structured file reading behavior instances, and use the larger first static risk assessment factor among the first static risk assessment factors corresponding to the any two structured file reading behavior instances as the static risk assessment of the two structured file reading behavior instances; and use the mapping result of the exponential function mapping with the natural constant e as the base for the static risk assessment as the static risk optimization factor.

[0070] In one embodiment, the The structured file reading behavior instance and the The calculation expression of the static risk optimization factor of a structured file reading behavior instance is:

[0071]

[0072] in, Indicates the The structured file reading behavior instance and the Static risk optimization factor for a structured file reading behavior instance; Indicates the A first static risk assessment factor for a structured file reading behavior instance; Indicates the The first static risk assessment factor of a structured file reading behavior instance.

[0073] It should be noted that the static risk adjustment factor constructed by the above logic , successfully integrated two key static risk context information, file sensitivity and visitor credibility, which were originally ignored by standard distance, into the distance measurement in a quantitative, high-risk-focused and nonlinearly amplified manner, thus overcoming the core defects of existing technologies in this regard and laying the foundation for subsequent more accurate file reading behavior clustering analysis.

[0074] S3, by constructing a dynamic pattern risk score and an index amplification factor, conducts characteristic vector behavior pattern analysis and obtains a dynamic risk optimization factor.

[0075] Although the static risk adjustment factor introduced in the previous steps Taking into account the two key static risk contexts of file sensitivity and visitor credibility, the shortcomings of the standard Euclidean distance in this regard are corrected, but this does not completely solve the limitations of distance metrics in risk assessment. Specifically, even if two file reading behaviors and Having similar static risk backgrounds, their dynamic behavior patterns during execution are reflected in their characteristic vectors and There may still be significant differences in the specific distribution and combination of values, and these differences themselves may contain different potential risks. For example, a stable, high-frequency read executed by a single process (common in benign background tasks) and a high-frequency read involving multiple different users and processes with drastic frequency fluctuations (more likely to be associated with anomaly detection or system failures) may reveal completely different risk implications in their dynamic patterns, even if the static risks are similar. The adjusted distance metric is still based on Euclidean distance, so it is still not sensitive enough to identify the risk differences indicated by this specific pattern composed of feature combinations, resulting in dynamic patterns with very different risks being mistakenly judged as similar.

[0076] In order to make up for this shortcoming, that is, to solve the problem of insufficient distinguishing ability caused by the inability to effectively evaluate the risk of dynamic behavior patterns when static risks are similar, the present invention further optimizes the distance metric. This requires the introduction of a second adjustment factor, namely the dynamic risk adjustment factor The core function of this factor is to deeply analyze the characteristic vector of file reading behavior The dynamic patterns themselves are quantified to identify anomalies or risk tendencies inherent in them. Key dynamic characteristics that indicate pattern risk must first be identified and quantified, such as the dispersion of access sources and the instability of their behavior. High dispersion and high instability, especially when these two factors coexist, are often associated with higher potential risks.

[0077] Based on this understanding, the feature vector of each behavior instance needs to be Calculating a dynamic model risk score The design of this scoring function needs to comprehensively consider the above-mentioned dynamic feature dimensions, aiming to quantify the degree to which the behavior pattern deviates from the "normal, stable, and centralized" state, or the degree of abnormality presented by the pattern itself. For example, the scoring logic should ensure that when the feature vector shows high dispersion and high instability at the same time, The scores were significantly higher than when only a single feature was abnormal or the pattern was stable.

[0078] To construct such an effective dynamic risk adjustment factor , the first task is to Feature vectors within minute time windows Calculate a quantitative indicator that can reflect its dynamic model risk, defined as the dynamic model risk score The score must be calculated based on the feature vector The key dimensions and their combinations that can indicate pattern abnormality. To eliminate the influence of different feature scale differences, these dimensions need to be normalized before calculation (mapped to interval), the normalized eigenvector is In this embodiment, the calculation of the feature vector is based on a preset time window, and the time window size is set to 5 minutes. The key features that can reflect the risk of dynamic patterns include:

[0079] Dispersion of access sources : The normalized number of independent users , number of independent processes In this embodiment, it is designed to be a weighted average of these features: ,in is the preset weight coefficient. High The value is shown here The access sources within the minute window are wide-ranging and the pattern tends to be more dispersed.

[0080] Temporal instability of behavior : Use the normalized standard deviation of read frequency To measure: .high The value is shown here The behavior within the minute window is irregular and sudden, and the pattern tends to be more unstable.

[0081] Since the risk of dynamic patterns is often reflected in the coordinated occurrence of multiple abnormal features (e.g. Diversity and volatility within a 2-minute window is generally more suspicious than either just dispersion or just volatility), so the dynamic pattern risk score The calculation should reflect this combination effect. The indicators reflecting different risk aspects are multiplied together so that the final risk score will only increase significantly when multiple indicators are high. Based on this consideration, The calculation formula is designed as:

[0082]

[0083] in, Indicates the A first dynamic mode risk assessment factor of a normalized feature vector of a structured file reading behavior instance; Indicates the Evaluation of the access source dispersion of structured file reading behavior instances; Indicates the Behavioral instability evaluation of a structured file reading behavior instance.

[0084] It should be noted that the value( and All in interval, Also there interval), which can quantize the eigenvector In this The risk tendency of the dynamic pattern itself displayed within the minute time window, especially giving higher scores to high-risk patterns such as "dispersed and unstable", thus solving the problem that existing distance metrics are insensitive to the risks of such patterns.

[0085] After successfully quantifying each behavior instance in the corresponding Dynamic pattern risk scoring of feature vectors within minute windows These scores can then be used to construct dynamic risk adjustment factors that further adjust for distance .

[0086] Obtain any two structured file reading behavior instances, and use the larger first dynamic mode risk assessment factor among the first dynamic mode risk assessment factors corresponding to the two structured file reading behavior instances as the dynamic risk assessment value of the two structured file reading behavior instances; and use the mapping result of the dynamic risk assessment value using an exponential function with the natural constant e as the base as the dynamic risk optimization factor.

[0087] In one embodiment, the The structured file reading behavior instance and the The calculation expression of the dynamic risk optimization factor of a structured file reading behavior instance is:

[0088]

[0089] in, Indicates the The structured file reading behavior instance and the Dynamic risk optimization factor for each instance of structured file reading behavior; Indicates the A first dynamic mode risk assessment factor of a normalized feature vector of a structured file reading behavior instance; Indicates the The first dynamic pattern risk assessment factor of the normalized feature vector of a structured file reading behavior instance.

[0090] It should be noted that the construction logic of the static risk adjustment factor is similar to that of the static risk adjustment factor, considering the need to focus on higher-risk models and adopt a nonlinear amplification mechanism. The calculation is based on the two behaviors involved in the comparison and The risk score for the dynamic pattern with the higher risk score (corresponding to the risk score of the respective time window) is then processed using a natural exponential function. This ensures that if only one of the two behaviors has a higher risk, the adjustment effect is amplified, and the higher the risk, the faster the adjustment (magnification factor) grows.

[0091] S4, by fusing static and dynamic risk optimization factors, performs the final optimization distance calculation to obtain the behavior instance similarity after comprehensive risk weighting.

[0092] Obtain any two structured file reading behavior instances, and use the result of multiplying the static risk optimization factor and the dynamic risk optimization factor of the two structured file reading behavior instances as the distance optimization factor of the two structured file reading behavior instances;

[0093] Obtaining normalized feature vectors of each of the two structured file reading behavior instances, and using the Euclidean distance between the normalized feature vectors of the two structured file reading behavior instances as the basic distance of the two structured file reading behavior instances;

[0094] The result of multiplying the distance optimization factor of two structured file reading behavior instances by the basic distance of the two structured file reading behavior instances is used as the behavior instance similarity after comprehensive risk weighting.

[0095] S5, by applying the K-Means clustering algorithm based on optimized distance to perform behavioral pattern recognition and anomaly analysis, risk warning information of high-frequency file reading is obtained.

[0096] Obtain the fully risk-weighted behavior instance similarity of any two structured file reading behavior instances, and use the behavior instance similarity between the two instances as the optimized distance between the two instances.

[0097] Completed the optimization distance between behavior instances After the calculation of The clustering algorithm performs pattern recognition on file reading behavior, thereby realizing the monitoring of high-frequency reading. Specifically, the steps For a specific time window Output all file reading behavior instances do The input data of the algorithm. During the iteration process, the centroid of each cluster is calculated by calculating all the data points currently assigned to the cluster The normalized eigenvector of (to ensure effective operation even when dealing with mean centroids without native context information).

[0098] In application Before the algorithm, the number of clusters needs to be determined, that is value. The selection of the value adopts the commonly used "elbow rule", which is an existing method and will not be described in detail here.

[0099] implement The algorithm uses the optimized distance to iterate until the cluster assignment converges. Behavior instances are divided into Clusters The clustering results are then analyzed to identify potential abnormal high-frequency reading patterns. The focus of the analysis is to identify clusters whose characteristics are significantly different from known normal patterns, or outliers that do not belong to any stable cluster. and ) is significantly higher than other clusters, indicating that there is a high probability of abnormal behavior. Similarly, a single behavior instance far away from the centroid of all clusters Also considered a potential anomaly.

[0100] When a cluster or an outlier instance is judged as abnormal according to the above rules, the system generates a corresponding alarm. The alarm information includes the associated file path , visitor information and behavioral characteristics to support subsequent safety investigations. In this way, the present invention improves The application effect of clustering in high-frequency file reading monitoring improves the accuracy of distinguishing benign from abnormal behaviors.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A high-frequency file reading monitoring system based on log analysis, characterized in that: The high-frequency file reading monitoring system comprises the following steps: Step S1: Construct access risk context through log collection and auxiliary library scoring, and perform behavioral feature extraction and normalization processing to obtain structured file reading behavior instances; Step S2: By introducing a static risk score and an exponential adjustment factor, context-aware distance optimization is performed to obtain a static risk optimization factor, specifically including: Performing static risk assessment on the structured file reading behavior instance through file sensitivity score and visitor credibility score to obtain the first static risk assessment factor; By performing relative risk assessment on the instances of structured file reading behavior, a static risk optimization factor is obtained; By performing relative risk assessment on structured file reading behavior instances, a static risk optimization factor is obtained, including: Obtain any two structured file reading behavior instances, and use the larger first static risk assessment factor of the first static risk assessment factors corresponding to the two structured file reading behavior instances as the static risk assessment of the two structured file reading behavior instances; and use the result of mapping the static risk assessment to an exponential function with a natural constant e as the base as the static risk optimization factor; Step S3: By constructing a dynamic model risk score and an index amplification factor, performing a characteristic vector behavior pattern analysis, and obtaining a dynamic risk optimization factor; according to the above-mentioned construction of a dynamic model risk score and an index amplification factor, performing a characteristic vector behavior pattern analysis, and obtaining a dynamic risk optimization factor, specifically includes: By evaluating the access source dispersion and behavior instability based on the normalized feature vector, a dynamic pattern risk analysis is performed on the structured file reading behavior instance to obtain the first dynamic pattern risk assessment factor; By performing dynamic risk comparison and evaluation between instances of structured file reading behavior, a dynamic risk optimization factor is obtained; Step S4: By fusing the static and dynamic risk optimization factors, the final optimized distance calculation is performed to obtain the behavior instance similarity after comprehensive risk weighting; Step S5: By applying the K-Means clustering algorithm based on optimized distance to perform behavior pattern recognition and anomaly analysis, risk warning information of high-frequency file reading is obtained.

2. The high-frequency file reading monitoring system based on log analysis according to claim 1 is characterized in that: According to the above, access risk context is constructed through log collection and auxiliary library scoring, and behavioral feature extraction and normalization are performed to obtain a structured file reading behavior instance, which specifically includes: The log agent deployed in the target information system continuously collects file access audit log data. The collected file access audit log entries include timestamp, accessed file path, user ID performing the access operation, process ID initiating the access, and operation type. In the central log processing system, log data undergoes preliminary screening, retaining only log records with the "read" operation type. Simultaneously, the system loads a sensitive file path pattern library and a visitor credibility information library. The sensitive file path pattern library contains sensitive paths defined by regular expressions and their corresponding sensitivity scores, while the visitor credibility information library records the credibility scores corresponding to user and process IDs. After completing log filtering and auxiliary library loading, the system divides the "read" type logs into non-overlapping time windows of fixed length in chronological order. In each time window, the system aggregates the unique file paths that have been read. The aggregation process includes counting the number of reads of the path, the number of independent user IDs, the number of independent process IDs, and calculating the standard deviation of the time interval of all read operation timestamps; the multidimensional features in the aggregation result are used as the original feature vector of the file in the time window; the original feature vector of the file in the time window is normalized to obtain the normalized feature vector; For each file and each time window, the normalized feature vector of the file in the time window, the file path, and the user information and process information used to assess the visitor risk are used as structured file reading behavior instances.

3. The high-frequency file reading monitoring system based on log analysis according to claim 1 is characterized in that: According to the above, by introducing a static risk score and an exponential adjustment factor, a context-aware distance optimization process is performed to obtain a static risk-weighted behavioral distance.

4. The high-frequency file reading monitoring system based on log analysis according to claim 1 is characterized in that: According to the static risk assessment of the structured file reading behavior instance using the file sensitivity score and the visitor credibility score, a first static risk assessment factor is obtained, which specifically includes: Obtain a structured file reading behavior instance, and for the structured file reading behavior instance, read the accessed file path in the instance, the visitor and process information of the operator performing the operation; obtain the set file sensitivity scoring function and visitor credibility scoring function; input the visitor and process information of the operator performing the operation in the structured file reading behavior instance into the visitor credibility scoring function, and obtain the output as the visitor credibility of the instance; input the file path in the structured file reading behavior instance into the file sensitivity scoring function, and use the corresponding output as the file sensitivity of the instance; multiply the result of the calculation obtained by subtracting the constant 1 from the visitor credibility of the instance by the file sensitivity, and use the corresponding calculation result as the first static risk assessment factor.

5. The high-frequency file reading monitoring system based on log analysis according to claim 1 is characterized in that: According to the access source dispersion and behavior instability assessment based on the normalized feature vector, a dynamic pattern risk analysis is performed on the structured file reading behavior instance to obtain a first dynamic pattern risk assessment factor, which specifically includes: Obtain a normalized feature vector of a structured file reading behavior instance, extract the feature dimension that is preset to represent the number of independent access users and the feature dimension that represents the number of independent access processes in the normalized feature vector, and assign them as the user access degree feature value and the process access degree feature value respectively; obtain a weighting coefficient for calculating the access source dispersion, multiply the user access degree feature value and the process access degree feature value by the corresponding weighting coefficient respectively, and sum the products and divide them by the sum of the weighting coefficients, and the obtained value is used as the access source dispersion of the instance; at the same time, extract the feature dimension that is preset to represent the standard deviation of the reading frequency from the normalized feature vector, assign it as the behavioral instability feature value, and use the feature value as the behavioral instability of the instance; multiply the access source dispersion and the behavioral instability, and use the corresponding calculation result as the first dynamic mode risk assessment factor.

6. The high-frequency file reading monitoring system based on log analysis according to claim 1 is characterized in that: According to the above, by performing dynamic risk comparison and evaluation between structured file reading behavior instances, a dynamic risk optimization factor is obtained, which specifically includes: Obtain any two structured file reading behavior instances, and use the larger first dynamic mode risk assessment factor among the first dynamic mode risk assessment factors corresponding to the two structured file reading behavior instances as the dynamic risk assessment value of the two structured file reading behavior instances; and use the mapping result of the dynamic risk assessment value using an exponential function with the natural constant e as the base as the dynamic risk optimization factor.

7. The high-frequency file reading monitoring system based on log analysis according to claim 1 is characterized in that: According to the above, the final optimized distance calculation process is performed by fusing the static and dynamic risk optimization factors to obtain the behavior instance similarity after comprehensive risk weighting, specifically including: Obtaining any two structured file reading behavior instances, and calculating a result of multiplying the static risk optimization factor and the dynamic risk optimization factor of the two structured file reading behavior instances as the distance optimization factor of the two structured file reading behavior instances; Obtaining normalized feature vectors of the two structured file reading behavior instances, and using the Euclidean distance between the normalized feature vectors of the two structured file reading behavior instances as a basic distance of the two structured file reading behavior instances; A result of multiplying the distance optimization factor of the two structured file reading behavior instances by the basic distance of the two structured file reading behavior instances is used as the behavior instance similarity after comprehensive risk weighting.

Images