Network security posture assessment system based on data analysis

Abstract

The application relates to the technical field of situation awareness, in particular to a network security situation assessment system based on data analysis, which comprises a structure jump identification module, an evolution trend analysis module, a repeated access judgment module, an access density statistical module and an abnormal set identification module.In the application, the level change of a request behavior path is continuously identified and combined, the request sequence with a jump structure in the access path is extracted, the change direction and continuity of the jump behavior are analyzed through sequence modeling in combination with the evolution trend of a response parameter in the time dimension, the abnormal request set with an obvious trend deviation is identified, the time interval and the repeated degree of the source IP request behavior are statistically judged on the basis, the frequent abnormal access event is aggregated and identified, and the abnormal access high-incidence area is dynamically constructed in combination with the local dense degree of the access frequency, so that the access convergence focusing trend of the target resource can be identified.

Description

Technical Field

[0001] This invention relates to the field of situational awareness technology, and in particular to a network security situational assessment system based on data analysis. Background Technology

[0002] The field of situational awareness technology involves the collection, fusion, understanding, and prediction of various types of data in complex information environments, including information collection, processing, information fusion, multi-dimensional visualization, dynamic prediction, and decision support.

[0003] Among them, the network security situation assessment system based on data analysis refers to a system that assesses the current state of the network and determines the threat level by collecting network security-related data such as attack logs, access records, and abnormal behavior logs, and using statistical analysis, rule matching, or classification algorithms.

[0004] Current technologies for network security situation assessment primarily rely on static rule matching or classification algorithms with fixed indicators to statistically process log data. They lack dynamic mining of changes in the internal path structure of request behavior, which can lead to identification delays or misjudgments when dealing with cross-level jumps or non-linear request behavior. When encountering high-frequency mutated access from source IPs within a short period of time, it is difficult to extract abnormal evolution characteristics from the access path structure, nor can it determine whether such requests have concentrated access targets or trends. In addition, when attack behavior is characterized by a stealthy approach with subtle path changes but frequent repetition, existing methods often ignore the dynamic characteristics of behavior aggregation and access density, resulting in coarse-grained situation assessment and failure to capture concentrated trend areas formed by abnormal access in a timely manner. Consequently, when facing the gradual accumulation of attack intensity, early identification and defense deployment are not achieved, affecting the sensitivity and effectiveness of overall situation awareness. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a network security situation assessment system based on data analysis.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a network security situation assessment system based on data analysis includes:

[0007] The structural jump identification module monitors the server access logs to obtain the request sequence of the corresponding client, identifies the jump request behavior between URL paths based on each type of request URL path in the request sequence, and constructs a structural jump request sequence.

[0008] The evolution trend analysis module analyzes the evolution trend of each jump request behavior in the structural jump request sequence over the entire request sequence, filters out request behavior records with abnormal evolution trends, and obtains a set of abnormal trends.

[0009] The repeated access determination module obtains the source IP and request timestamp of the abnormal request behavior records in the trend anomaly set, determines the repeated abnormal access behavior, and obtains the repeated abnormal access event group.

[0010] The access density statistics module counts the access frequency of the target server based on the repeated abnormal access event groups, and identifies abnormal access density areas based on the access frequency.

[0011] The anomaly concentration identification module determines the anomaly concentration situation based on whether the abnormal access density area and the URL path jump position in the structural jump request sequence point to the same server resource location.

[0012] As a further aspect of the present invention, the structural jump request sequence includes path jump request type, jump level change range, and corresponding response status attributes; the trend anomaly set includes jump behavior change direction, anomaly trend duration, and trend deviation magnitude; the repeated anomaly access event group includes repeated behavior occurrence count, repeated request time distribution, and concentrated source IP of anomaly behavior; the anomaly access density region includes anomaly access frequency segment, server corresponding time interval, and cumulative access behavior magnitude; and the anomaly access concentration situation includes target resource aggregation location, path aggregation distribution, and server pointing consistency.

[0013] As a further aspect of the present invention, the structural transition recognition module includes:

[0014] The request aggregation submodule retrieves the client IP, target server identifier, URL path, request timestamp, response status code, and response duration from the access log, aggregates them into a continuous request record sequence by combining client and server, and arranges them in chronological order to generate a continuous client request sequence.

[0015] The path change judgment submodule extracts the number of path levels and calculates the level change magnitude between adjacent paths based on the URL path of each record in the continuous request sequence of the client. Combined with the corresponding response status code and response duration, it performs path structure jump judgment on the continuous records and obtains the structure change judgment result.

[0016] The jump sequence construction submodule calls the access behavior identified in the structural change determination result, extracts the corresponding URL path, request timestamp, response status code and response duration, and combines them into a structural jump request sequence while maintaining the original time order.

[0017] As a further aspect of the present invention, the evolutionary trend analysis module includes:

[0018] The response feature extraction submodule calls the response status code and response duration corresponding to each request record in the structural jump request sequence, extracts the response parameters according to the record arrangement order, and obtains a set of response change data groups.

[0019] The time trajectory construction submodule constructs a time sequence trajectory of the jump request behavior based on the response change data set and the request timestamp, and establishes a response state evolution trajectory set by combining the recurrence frequency of the response status code and the continuous change direction of the response duration.

[0020] The trend anomaly identification submodule calls the trend of response status code aggregation degree and response duration change of each trajectory in the response state evolution trajectory set, establishes the change pattern between sequences based on the conditional random field algorithm, performs joint judgment on the state aggregation intensity and duration increase rate in the evolution pattern, filters the jump request behavior records that meet the evolution trend anomaly conditions, and generates a trend anomaly set.

[0021] As a further aspect of the present invention, the repeated access determination module includes:

[0022] The source aggregation submodule obtains the source IP and request timestamp of each abnormal request behavior record in the trend anomaly set, classifies and organizes the request behavior records into source access sequences according to the source IP, arranges the corresponding abnormal access actions according to the time order, and generates a source access sequence set.

[0023] The behavior interval analysis submodule calls the request timestamp data in the source access sequence set, calculates the time interval between adjacent request behaviors, and combines it with the response status code of the corresponding record to determine whether there is a combination of behaviors with tight consecutive request intervals and consistent response status codes in the source sequence, and obtains the behavior repetition analysis results.

[0024] The repeated behavior identification submodule, based on the behavior repetition analysis results, filters groups of consecutive access records with a time interval less than the interval threshold and consistent response status codes, performs behavior aggregation judgment by source IP, marks the behavior groups that meet the repeated access conditions, and generates repeated abnormal access event groups.

[0025] As a further aspect of the present invention, the access density statistics module includes:

[0026] The server access extraction submodule calls the target server identifier and request timestamp corresponding to each request record in the repeated abnormal access event group, sorts the request time data according to the target server, and establishes a server access time sequence set.

[0027] The access frequency calculation submodule calculates the number of requests to each target server within a unit time period based on the server access time series set, and constructs the target server access frequency series.

[0028] The density region identification submodule calls the access frequency of each server in the target server access frequency sequence for the corresponding time period, uses the density peak clustering algorithm to cluster the access frequency change trend, identifies the degree of concentration of access behavior in the time period, marks the region of sudden increase in access frequency, and generates abnormal access density regions.

[0029] As a further aspect of the present invention, the anomaly centralized identification module includes:

[0030] The path feature extraction submodule calls the URL path recorded in the structure jump request sequence, extracts the path level jump position and calculates the path character similarity, aggregates the path structure according to the source client, and obtains the path aggregated feature data group;

[0031] The resource pointing judgment submodule performs classification and comparison on the target server identifier based on the path aggregation feature data group to determine whether there are multiple client request paths that tend to point to the resource location of the same server. Combined with the server list in the abnormal access density area, it filters the server behavior combination that the path aggregation target falls into the density area and generates resource pointing focused matching results.

[0032] The centralized situation output submodule calls all server behavior records in the resource pointing focus matching results that have consistent pointing and clear aggregation trends, marks them as a set of target servers with similar access structures and concentrated frequency, and generates an abnormal access centralized situation.

[0033] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0034] In this invention, by continuously identifying and combining the hierarchical changes in the request behavior path, request sequences with jump structures in the access path can be extracted. Combined with the evolution trend of response parameters in the time dimension, the direction and persistence of jump behavior changes are analyzed by sequence modeling, thereby identifying abnormal request sets with obvious trend deviations. On this basis, the time interval and repetition of the source IP request behavior are statistically judged to achieve the aggregation and identification of frequent abnormal access events. At the same time, combined with the local density of access frequency, high-incidence areas of abnormal access are dynamically constructed. Furthermore, by combining the consistency characteristics of path pointing features and resource ownership location, the convergence and focusing trend of access to target resources is identified, realizing the full-chain identification path from structural jumps to access concentration. This processing logic can effectively reveal potential network attack concentration points and abnormal behavior outbreak areas, thereby improving the accuracy of situation assessment. Attached Figure Description

[0035] Figure 1 This is a system flowchart of the present invention;

[0036] Figure 2 This is a flowchart of the structural transition recognition module of the present invention;

[0037] Figure 3 This is a flowchart of the evolution trend analysis module of the present invention;

[0038] Figure 4 This is a flowchart of the duplicate access determination module of the present invention;

[0039] Figure 5 This is a flowchart of the access density statistics module of the present invention;

[0040] Figure 6 This is a flowchart of the anomaly centralized identification module of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0043] Please see Figure 1 Data-driven cybersecurity situation assessment systems include:

[0044] The structural jump identification module monitors the server access logs to obtain the request sequence of the corresponding client, identifies the jump request behavior between URL paths based on each type of request URL path in the request sequence, and constructs a structural jump request sequence.

[0045] The evolution trend analysis module analyzes the evolution trend of each type of jump request behavior in the structural jump request sequence over the entire request sequence, filters out request behavior records with abnormal evolution trends, and obtains a set of trend anomalies.

[0046] The duplicate access determination module obtains the source IP and request timestamp of the abnormal request behavior records in the trend anomaly set, determines the duplicate abnormal access behavior, and obtains the duplicate abnormal access event group.

[0047] The access density statistics module counts the access frequency of the target server based on repeated abnormal access event groups, and identifies abnormal access density areas based on the access frequency.

[0048] The anomaly concentration identification module determines the anomaly concentration situation based on whether the URL path jump positions in the abnormal access density area and the structural jump request sequence point to the same server resource location.

[0049] The structural jump request sequence includes the path jump request type, the jump level change range, and the corresponding response status attributes. The trend anomaly set includes the jump behavior change direction, the duration of the abnormal trend, and the trend deviation magnitude. The repeated abnormal access event group includes the number of repeated behaviors, the time distribution of repeated requests, and the source IP of the abnormal behavior concentration. The abnormal access density area includes the abnormal access frequency segment, the server corresponding time interval, and the cumulative magnitude of access behavior. The abnormal access concentration situation includes the target resource aggregation location, the path aggregation distribution, and the server pointing consistency.

[0050] Please see Figure 2 The structural transition recognition module includes:

[0051] The request aggregation submodule retrieves the client IP, target server identifier, URL path, request timestamp, response status code, and response duration from the access log, aggregates them into a continuous request record sequence by combining client and server, and arranges them in chronological order to generate a continuous client request sequence.

[0052] Processing access logs first requires reading the original log file or database table, parsing line by line to extract the client IP address, target server identifier, requested URL path, request timestamp, server response status code, and response duration. For each access record, the log structure is parsed using regular expressions to extract fields. For example, an Apache log can be broken down into "123.45.67.89--[01 / Jul / 2025:13:00:12+0000]'GET / api / user / loginHTTP / 1.1'200150". Through string splitting and regular expression matching, the IP address is extracted as "123.45.67.89", the path as " / api / user / login", the status code as 200, and the response duration as 150ms. After extraction, the logs need to be sorted by client IP address and target server identifier. The system aggregates and combines server requests, using a hash table in the data structure. Each IP-server pair serves as the key, and all request records are compiled into an ordered list. This list is then sorted in ascending order by the request timestamp field. This can be achieved by converting the timestamp to UNIX time and then sorting by integer value. For example, sorting three records with timestamps of "2025-07-03 08:30:15", "2025-07-03 08:30:18", and "2025-07-03 08:31:02" generates a sequential request sequence. For instance, if user IP "123.45.67.89" accesses "server01", its sequential request sequence might be " / login→ / dashboard→ / settings". Each path includes the time, status code, and response time, and these are recorded sequentially in the data table.

[0053] The path change judgment submodule extracts the number of path levels and calculates the level change magnitude between adjacent paths based on the URL path of each record in the continuous request sequence of the client. Combined with the corresponding response status code and response duration, it performs path structure jump judgment on the continuous records and obtains the structure change judgment result.

[0054] The URL path of each record is obtained from the continuous request sequence from the client. The path level is calculated by extracting the URL path field for each record. The path level is defined as the number of non-empty segments after the " / " character in the URL is used. The extraction process can use string splitting to process the URL. For example, the path " / api / user / login" is split into three segments ["api", "user", "login"] using the split(" / ") method, with a level of 3. Another record, " / api / v1 / user / details", is split into four segments ["api", "v1", "user", "details"], with a level of 4. The path level of each record in the sequence is extracted sequentially to form a path level sequence, such as [3, 4, 5, 3, 2]. Then, the change in path level between adjacent path levels is calculated sequentially. , No. Article and No. Variation between bars ,in, : indicates the first The absolute value of the difference between the path hierarchy of a path record and its next record. : indicates the first in the sequence The number of path levels corresponding to each request record (i.e., the number of non-empty segments obtained after dividing the URL by " / "); : indicates the first in the sequence The number of path levels in the request record, where " "Indicates the current index" Based on that, move one position to the right, that is, to the next adjacent record; : Represents the absolute value of the difference between the number of levels of two records.

[0055] Suppose that the number of levels for the path sequence " / index(1), / user / login(2), / user / login / detail(3), / error(1)" are [1, 2, 3, 1], the change range is calculated as follows:

[0056] ,Right now , ;

[0057] ,Right now , ;

[0058] ,Right now , .

[0059] The paths and magnitudes of change in this sequence are integrated into a structure:

[0060] " / index" → " / user / login": Level change range = 1;

[0061] " / user / login" → " / user / login / detail": Level change range = 1;

[0062] " / user / login / detail" → " / error": Level change range = 2.

[0063] Each path pair also needs to be comprehensively judged based on the corresponding response status code and response time. If the response status code is an error code such as 500, 404, or 403, or the response time exceeds the set baseline value (e.g., the threshold is set to 1000ms), then the path pair is determined to be a "jump path". For example, in the above example, the " / user / login / detail" → " / error" path has a status code of 500 and a response time of 1200ms, which meets the jump condition. And the status code is 500 and the response time is [time missing]. Therefore, the path transition is valid. The baseline value of 1000ms is based on statistical analysis of the normal request response time distribution. For example, if 80% of the response times in the sample fall within the 0–900ms range, then 1000ms as the transition threshold has practical distinguishing power. By iterating through all adjacent records in the path sequence, calculating the hierarchical changes, and combining them with the judgment conditions, a complete list of structural transition judgment sequences is finally obtained.

[0064] The jump sequence construction submodule calls the access behavior identified in the structural change judgment result, extracts the corresponding URL path, request timestamp, response status code and response duration, and combines them into a structural jump request sequence while maintaining the original time order.

[0065] After receiving the structural change determination results, all records corresponding to the access behaviors determined to be jumps are extracted, including their URL paths, request timestamps, response status codes, and response durations. These records are arranged and combined according to the original request time sequence to maintain temporal consistency. For example, if three request records are identified from the structural change: the first is " / dashboard / settings", timestamp "2025-07-03 09:00:12", status code 500, response duration 1800ms; the second is " / admin / user / edit", time "2025-07-03 09:00:35", status code 403, response duration 1300ms; and the third is " / logs / error / view", time "2025-07-03 09:01:02", status code 200, response duration 1100ms, then these are combined into a structural jump request sequence, arranged in chronological order in a list, and finally form a complete structural jump sequence record set.

[0066] Please see Figure 3 The evolutionary trend analysis module includes:

[0067] The response feature extraction submodule calls the response status code and response duration corresponding to each request record in the structural jump request sequence, extracts the response parameters according to the record arrangement order, and obtains a set of response change data groups.

[0068] From the structural jump request sequence, each record's response status code and response duration are read sequentially. The extraction order strictly follows the record arrangement order, without reordering or filtering, to maintain the integrity of the original behavior chain. During extraction, a field mapping method can be used to establish separate status code and response duration sequences. For example, extract the first record with status code 500 and duration 1280ms, the second record with status code 403 and duration 1100ms, and the third record with status code 200 and duration 890ms. The status codes and durations are paired to form response change data groups. Each group contains two elements: a status code, indicating the server's response type for the request; and a response time, expressed in milliseconds. After extraction, the data is integrated into a set of response change data groups according to the original request order. An example execution process is: group 1 is [500, 1280], group 2 is [403, 1100], group 3 is [200, 1280], group 2 is [403, 1100], group 3 is [200, 1280], group 403 is [403, 1100], group 500 is [403, 1100], group 6 is [403, 1100], group 7 is [403, 1100], group 8 is [403, 1100], group 9 is [403, 1100], group 1 ...

[890] , thus obtaining the set of response change data groups.

[0069] The time trajectory construction submodule constructs a time sequence trajectory of the jump request behavior based on the response change data set and the request timestamp, and establishes a response state evolution trajectory set by combining the recurrence frequency of the response status code and the continuous change direction of the response duration.

[0070] After receiving the set of response change data groups, a timeline node is first constructed based on the request timestamp corresponding to each data group. The timestamps can be arranged in raw millisecond format. For example, record 1 has a timestamp of "2025-07-03 09:00:15.128", record 2 has a timestamp of "2025-07-03 09:00:17.300", and record 3 has a timestamp of "2025-07-03". "09:00:18.150" binds this timeline to each response change data group to form a time series trajectory of the jump request. The trajectory is constructed by setting the status code and response duration combination point corresponding to each time point. In the visualization, a two-dimensional trajectory line graph can be constructed, with one axis representing time and the other axis representing the superposition of status code and response time changes. Further analysis is performed on the repetition frequency of status codes in the sequence. For example, if status code 500 appears twice and 200 appears once in 3 requests, it can be marked as a status aggregation point. At the same time, the direction of response duration change is detected. If the response time increases or decreases with each request, it constitutes a trend direction. For example, the first record takes 1280ms, the second takes 1100ms, and the third takes 890ms, showing a downward trend. This change direction is recorded as "decreasing". Together with the status code aggregation, it forms a response state evolution trajectory set. The trajectory set is a collection of trajectories constructed from all jump records in the sequence according to three dimensions: time expansion, status code marking, and duration change trend.

[0071] The trend anomaly identification submodule calls the trend of the clustering degree of response status codes and the change of response duration for each trajectory in the response state evolution trajectory set, establishes the change pattern between sequences based on the conditional random field algorithm, performs joint judgment on the state aggregation intensity and duration rise rate in the evolution pattern, filters the jump request behavior records that meet the evolution trend anomaly conditions, and generates a trend anomaly set.

[0072] After obtaining the response state evolution trajectory set, the status code clustering strength and response duration variation trend of each trajectory are extracted first. These two factors are then used as joint feature inputs. Status code clustering strength is defined as the proportion of the number of times a specific status code appears in a trajectory relative to the length of that trajectory. Response duration variation trend is defined as the average growth rate of response time for consecutive records. For example, if the status code sequence in three records of a trajectory is 500, 500, 200, the clustering strength is... The corresponding response times are 1280ms, 1400ms, and 1650ms, with response time changes of 120ms and 250ms respectively, and an average growth rate of [missing data]. .

[0073] Using the formula of the conditional random field model:

[0074] ;

[0075] in, : Indicates that in a given feature sequence Predicting label sequences under the given conditions The conditional probability; : Input sequence, representing the joint feature set of the current jump trajectory at each time point, where: : indicates the first in the trajectory Input feature pairs at each time step, including status code clustering and response time growth rate; : This is the time step index, representing the current position in the sequence, with a value range of 100%. ; : This represents the total length of the trajectory sequence, indicating the total number of time steps it contains; Output the label sequence, where: : indicates the first Whether a time step is abnormal or not, the value is 0 (normal) or 1 (abnormal); : Normalization factor, used to normalize the exponential sum of all possible combinations of label sequences, so that the result is a probability distribution; Weight parameters corresponding to the feature function: : Control state characteristic function The degree of impact on the overall probability; : Control transfer characteristic function The extent of the impact;

[0076] State feature function, used to evaluate the features at the current time point. With tags The degree of matching is determined by the following conditions: clustering degree > 0.6 and response speed > 150ms. If the value is 1, then the value is 1; otherwise, the value is 0. : Transition feature function, used to represent the label at the previous time step With the current tag The transfer relationship between them, for example: if and This indicates a new abnormal state has occurred; if and This indicates that the abnormal state continues; : Indicates the current time point The label value at the previous time point, where: : Indicates the previous time step whose time step index is 1 less than the current time step. Note that when... This item is in its initial state and needs to be set during model initialization.

[0077] : Corresponding state characteristic function This represents the input features (such as clustering and response rate) at the current moment and their relationship with the label. The degree of matching. Weight The larger the value, the greater the influence of the state features on the final anomaly judgment, meaning the model relies more heavily on the feature values ​​at the current moment to determine whether it is an anomaly. This applies to scenarios where features such as status codes and response times play a decisive role in anomaly identification in abnormal requests.

[0078] : Corresponding transition characteristic function This represents the transition relationship between the label at the previous time step and the label at the current time step. Weight The larger the value, the greater the impact of the transition feature on the outcome, meaning the model places more emphasis on the persistence or suddenness of abnormal behavior. For example, the continuation of the abnormal state ( ) or sudden change ( (This) will be given greater importance.

[0079] Given a trajectory length of 3 and a feature sequence of:

[0080] Step 1: Status code 500, clustering degree = 0.67, response time = 1280ms;

[0081] Step 2: Status code 500, clustering degree = 0.67, response time = 1400ms;

[0082] Step 3: Status code 200, clustering degree = 0.67, response time = 1650ms.

[0083] The duration differences are 120ms and 250ms respectively, with an average growth rate of 185ms, which meets the abnormal growth threshold (150ms). The status code clustering also meets the set threshold (0.6). Therefore, let the label sequence... The model score is calculated as follows:

[0084] ,because The conditions are met, and ;

[0085] Similarly;

[0086] ,because This indicates the start of an exception;

[0087] The abnormal continuation;

[0088] Assumption , ;

[0089] Total score: .

[0090] The probability of the label sequence is obtained after normalization. The highest score indicates an abnormal trend trajectory. All trajectories with high scores meeting this characteristic are grouped into the abnormal trend set.

[0091] In the process of trend anomaly identification, the core computational steps include extracting the clustering degree of the response status codes and the average growth rate of response duration for each trajectory. These features are used as input, and a conditional random field model is used to score whether each time point in the trajectory is in an anomalous state. The calculation process first determines whether the features at each time point meet the anomaly conditions, such as a clustering degree greater than 0.6 and a response growth rate greater than 150ms. If these conditions are met, the scoring function increases. Then, further scoring is performed based on the transition relationship between labels, such as from normal to anomalous or the continuation of anomalousness. Finally, the state feature scores and transition feature scores are weighted and summed to form the total score of the trajectory label sequence. By scoring and normalizing all possible label sequences, the label sequence with the highest score is selected as the prediction result. This calculation result indicates which time points in the current trajectory are identified as trend anomaly points. Finally, a set of jump request records with an abnormal evolution trend is selected for the system to monitor and mark abnormal behavior.

[0092] Please see Figure 4 The duplicate access detection module includes:

[0093] The source aggregation submodule obtains the source IP and request timestamp of each abnormal request behavior record in the trend anomaly set, categorizes and organizes the request behavior records into source access sequences according to the source IP, arranges the corresponding abnormal access actions according to the time order, and generates a source access sequence set.

[0094] The source IP and request timestamp of each abnormal request behavior record are obtained from the trend anomaly set. By examining the source IP of each record one by one, request records from the same source IP are categorized, and all abnormal requests from the same IP are arranged in chronological order to form an access sequence for that source. For example, assuming that the timestamps of request records from the same IP are "2025-07-03 10:00:00", "2025-07-03 10:01:00", and "2025-07-03 10:03:00", they form a group of access behaviors in chronological order, recording abnormal requests issued by that IP at different times. Similarly, by performing similar operations on all source IPs, a source access sequence set is finally generated, which contains all abnormal access records sorted by time.

[0095] The behavior interval analysis submodule calls the request timestamp data in the source access sequence set, calculates the time interval between adjacent request behaviors, and combines it with the response status code of the corresponding record to determine whether there is a combination of behaviors with tight consecutive request intervals and consistent response status codes in the source sequence, and obtains the behavior repetition analysis results.

[0096] The request timestamp data for each record is extracted from the source access sequence set, and the time interval between adjacent request actions is calculated. For example, assuming two adjacent records have request timestamps of "2025-07-03 10:01:00" and "2025-07-03 10:03:00", the time interval between them is 2 minutes. Based on this, the response status code of each record is also used to determine the continuity of the action. For example, if the response status codes of the first two records are both 200, then these two requests are considered to likely belong to the same action pattern. Based on the matching of the calculated interval time and response status codes, further analysis is conducted to determine if there are action combinations within the source sequence with small intervals between consecutive requests and consistent status codes. For example, if the time intervals of multiple requests are all less than 1 minute and the response status codes are consistent, then this combination will be marked as having a high degree of action repetition.

[0097] The repeated behavior identification submodule filters groups of consecutive access records with a time interval less than the interval threshold and a consistent response status code based on the behavior repetition analysis results. It then performs behavior aggregation judgment by source IP, marks the behavior groups that meet the repeated access conditions, and generates repeated abnormal access event groups.

[0098] Based on the behavioral interval analysis results, groups of consecutive access records with time intervals less than a set threshold (e.g., 1 minute) and consistent response status codes are further filtered out. For each filtered behavioral combination, they are clustered and judged based on the source IP. If the request behavior of the same source IP meets the condition of repeated access, these behaviors are marked as repeated abnormal access events, forming a repeated abnormal access event group. For example, if multiple requests from a certain IP address occur consecutively within 1 minute, and their response status codes are 200, and these behaviors occur repeatedly, then these requests will be identified as abnormal access events, and the repeated access behavior of that source IP will be recorded. Finally, through filtering and marking, a set of abnormal events focusing on high-frequency repeated access behaviors is obtained.

[0099] Please see Figure 5 Access density statistics module includes:

[0100] The server access extraction submodule calls the target server identifier and request timestamp corresponding to each request record in the duplicate abnormal access event group, sorts the request time data by target server, and establishes a server access time sequence set.

[0101] The process begins by retrieving the target server identifier and request timestamp for each request record from the repeated abnormal access event group. First, the target server identifier and corresponding timestamp for each record are paired, and the request time data is categorized and organized according to the target server. For example, suppose a target server receives multiple requests at different times, with timestamps of "2025-07-03 11:00:00", "2025-07-03 11:05:00", and "2025-07-03 11:08:00". These request records are categorized as access time sequences for that target server, arranged chronologically. By performing similar operations on all servers, the request timestamps of each server are combined into an independent access time sequence, ultimately resulting in a complete set of server access time sequences.

[0102] The access frequency calculation submodule calculates the number of requests to each target server within a unit of time based on the server access time series set, and constructs the target server access frequency series.

[0103] By iterating through the set of server access time series, the number of requests to each target server within a unit of time is counted. The unit of time can be set to 1 minute, 5 minutes, or a longer range; the specific time period depends on the granularity requirements of anomaly detection. For example, for a target server's access time series, if 5 requests occur within 1 minute, then the server was accessed 5 times within 1 minute. By iterating through all time periods and calculating the number of requests within each time period, an access frequency sequence is ultimately constructed for each target server. This access frequency sequence reflects the request load intensity of the server in different time periods and can be used to subsequently detect which time periods have abnormally high access frequencies.

[0104] The density region identification submodule calls the access frequency of each server in the target server access frequency sequence for the corresponding time period, uses the density peak clustering algorithm to cluster the access frequency change trend, identifies the degree of concentration of access behavior in the time period, marks the areas of sudden increase in access frequency, and generates abnormal access density regions.

[0105] Based on the access frequency sequence of the target server, the density peak clustering algorithm is used to cluster the access frequency change trend of each server, identifying concentrated areas of access frequency within certain time periods and marking them as abnormal access density areas. The specific steps of the clustering process are as follows: Access frequency data for each time period is extracted from the access frequency sequence. Assuming the request frequency of a certain server in different time periods is [1, 3, 5, 6, 2, 8, 10, 15], these data represent the access frequency of the server in each time period. The density peak clustering algorithm clusters based on the local density and relative distance of data points. The density of each data point is represented by the number of points close to it, while the distance between that point and other points is used to determine the cluster boundary.

[0106] First, the access frequency density for each time period is calculated. For example, if the request frequency for a certain time period is 10, and several adjacent time periods within that time period also have high frequencies, then the density of that time period is high; conversely, the density is low. Next, the difference in access frequency between each time period and other time periods is calculated. For example, if the access frequency for one time period is 10, while the access frequency for another time period is 1, then the difference between them is large. Based on density and distance, the density peak clustering algorithm divides clustering regions according to the relative density and mutual distance of data points. Suppose that after calculation, a sudden increase in access frequency is found in a certain period (such as an interval with a frequency of 10 or higher), this period will be identified as a "high density" region and marked as an abnormal access density region. In this way, time periods with sudden increases in access frequency can be identified. These areas of sudden increase usually indicate abnormal access behavior, such as DDoS attacks or sudden traffic bursts. Finally, the clustering algorithm will mark the time periods with significantly concentrated frequency and sudden increases in access as abnormal access density regions. Marking these density regions facilitates subsequent anomaly detection and event response.

[0107] For example, if a server has access frequencies of 10 and 15 in the time periods [6, 7] and [7, 8] respectively, and the access frequencies of the time periods surrounding these time periods (such as [5, 6], [8, 9]) are low, then the access frequency during this period will be considered to be a sudden increase and thus marked as an area of ​​abnormal access density.

[0108] Please see Figure 6 The anomaly centralized identification module includes:

[0109] The path feature extraction submodule calls the URL path recorded in the structural jump request sequence, extracts the path level jump position and calculates the path character similarity, aggregates the path structure according to the source client, and obtains the path aggregated feature data group;

[0110] The module retrieves the URL path of each record from the structural jump request sequence and extracts the path hierarchy jump position. The path hierarchy jump position refers to the point in the request sequence where the path hierarchy number changes significantly. For example, if a request path is " / api / user / login" with a hierarchy number of 3, and the next request path is " / api / user / details" with a hierarchy number of 4, the jump position occurs at the point where the path jumps from 3 to 4. For each pair of consecutive request paths, the module calculates their path character similarity using a method such as Jaccard similarity to calculate the string similarity between adjacent paths.

[0111] Formula for calculating path character similarity:

[0112] ;

[0113] in, and It is a collection of two paths (the elements in the path string can be the various segments of the path, such as the segments "api", "user", and "login" in " / api / user / login"). Represents two sets of paths and The size of the intersection of the paths is the number of path segments they collectively contain. Represents a set of paths and The size of the union of the paths is the total number of path segments.

[0114] There are two paths: path A: " / api / user / login", path B: " / api / user / settings", represented in segments: the segments of path A are... The segments of path B are: .

[0115] Calculate the intersection and union: Intersection: (Both path A and path B contain these two path segments), Union: Calculate Jaccard similarity: Therefore, the character similarity between path A and path B is 0.5, indicating that their path structures are 50% similar. Next, by aggregating the paths by source client, all request paths from the same client are summarized, and a path aggregation feature data set is generated based on the path's hierarchical jump positions and similarity. The path aggregation feature data set includes each client's access behavior, path hierarchy changes, and path character similarity.

[0116] The resource pointing judgment submodule performs classification and comparison on the target server identifier based on the path aggregation feature data group to determine whether there are multiple client request paths that tend to point to the same server's resource location. Combined with the server list in the abnormal access density area, it filters the server behavior combination that the path aggregation target falls into the density area and generates resource pointing focused matching results.

[0117] Based on path aggregation feature data sets, target server identifiers are categorized and compared. First, target server identifier information is extracted from each path aggregation feature data set and then categorized. If multiple clients' request paths tend to point to the same server resource location, it indicates a concentration of access behavior among these clients. For example, if path aggregation data sets from multiple different clients show that their request target servers all point to the same resource, this may be due to a shared resource or public service. Next, combining the server list in the abnormal access density region, the submodule filters out server behavior combinations where path aggregation targets fall into the density region. Servers within the density region typically indicate that servers in that region have experienced high-frequency requests within a certain period, making them more susceptible to abnormal behavior. Finally, resource-pointing focused matching results are generated, reflecting which servers and access behavior combinations are most closely related to the abnormal density region.

[0118] The centralized situation output submodule calls the resource pointer to focus on matching results and all server behavior records that point to the same server and have a clear aggregation trend. These records are marked as a set of target servers with similar access structures and concentrated frequency, and an abnormal access centralized situation is generated.

[0119] The module receives data from resource-targeted matching results, analyzes it, and marks server behavior records that show consistent targeting and clear aggregation trends. These records indicate that multiple clients frequently access the same server within the same time period, forming a pattern of convergent access structures. Through centralized analysis of these records, the submodule marks these server behaviors as a set of target servers with concentrated frequency, meaning that these servers play a centralized role in abnormal access behavior. These servers often face sudden requests from multiple clients, potentially indicating abnormal events such as DDoS attacks, crawler activity, or abnormal traffic. After generating the abnormal access concentration situation, a clear view is provided, showing which servers have the most concentrated access requests in abnormal access behavior, thereby helping system administrators identify and take appropriate countermeasures.

[0120] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A network security situation assessment system based on data analysis, characterized in that, The system includes: The structural jump identification module monitors the server access logs to obtain the request sequence of the corresponding client, identifies the jump request behavior between URL paths based on each type of request URL path in the request sequence, and constructs a structural jump request sequence. The evolution trend analysis module analyzes the evolution trend of each jump request behavior in the structural jump request sequence over the entire request sequence, filters out request behavior records with abnormal evolution trends, and obtains a set of abnormal trends. The repeated access determination module obtains the source IP and request timestamp of the abnormal request behavior records in the trend anomaly set, determines the repeated abnormal access behavior, and obtains the repeated abnormal access event group. The access density statistics module counts the access frequency of the target server based on the repeated abnormal access event groups, and identifies abnormal access density areas based on the access frequency. The anomaly concentration identification module determines the anomaly concentration situation based on whether the abnormal access density area and the URL path jump position in the structural jump request sequence point to the same server resource location. The structural transition recognition module includes: The request aggregation submodule retrieves the client IP, target server identifier, URL path, request timestamp, response status code, and response duration from the access log, aggregates them into a continuous request record sequence by combining client and server, and arranges them in chronological order to generate a continuous client request sequence. The path change judgment submodule extracts the number of path levels and calculates the level change magnitude between adjacent paths based on the URL path of each record in the continuous request sequence of the client. Combined with the corresponding response status code and response duration, it performs path structure jump judgment on the continuous records and obtains the structure change judgment result. The jump sequence construction submodule calls the access behavior identified in the structural change determination result, extracts the corresponding URL path, request timestamp, response status code and response duration, and combines them into a structural jump request sequence while maintaining the original time order.

2. The network security situation assessment system based on data analysis according to claim 1, characterized in that, The structural jump request sequence includes path jump request type, jump level change range, and corresponding response status attributes. The trend anomaly set includes jump behavior change direction, anomaly trend duration, and trend deviation magnitude. The repeated anomaly access event group includes the number of repeated behaviors, repeated request time distribution, and concentrated source IP of the anomaly behavior. The anomaly access density region includes anomaly access frequency segment, server corresponding time interval, and cumulative access behavior magnitude. The anomaly access concentration situation includes target resource aggregation location, path aggregation distribution, and server pointing consistency.

3. The network security situation assessment system based on data analysis according to claim 1, characterized in that, The evolution trend analysis module includes: The response feature extraction submodule calls the response status code and response duration corresponding to each request record in the structural jump request sequence, extracts the response parameters according to the record arrangement order, and obtains a set of response change data groups. The time trajectory construction submodule constructs a time sequence trajectory of the jump request behavior based on the response change data set and the request timestamp, and establishes a response state evolution trajectory set by combining the recurrence frequency of the response status code and the continuous change direction of the response duration. The trend anomaly identification submodule calls the trend of response status code aggregation degree and response duration change of each trajectory in the response state evolution trajectory set, establishes the change pattern between sequences based on the conditional random field algorithm, performs joint judgment on the state aggregation intensity and duration increase rate in the evolution pattern, filters the jump request behavior records that meet the evolution trend anomaly conditions, and generates a trend anomaly set.

4. The network security situation assessment system based on data analysis according to claim 3, characterized in that, The duplicate access determination module includes: The source aggregation submodule obtains the source IP and request timestamp of each abnormal request behavior record in the trend anomaly set, classifies and organizes the request behavior records into source access sequences according to the source IP, arranges the corresponding abnormal access actions according to the time order, and generates a source access sequence set. The behavior interval analysis submodule calls the request timestamp data in the source access sequence set, calculates the time interval between adjacent request behaviors, and combines it with the response status code of the corresponding record to determine whether there is a combination of behaviors with tight consecutive request intervals and consistent response status codes in the source sequence, and obtains the behavior repetition analysis results. The repeated behavior identification submodule, based on the behavior repetition analysis results, filters groups of consecutive access records with a time interval less than the interval threshold and consistent response status codes, performs behavior aggregation judgment by source IP, marks the behavior groups that meet the repeated access conditions, and generates repeated abnormal access event groups.

5. The network security situation assessment system based on data analysis according to claim 4, characterized in that, The access density statistics module includes: The server access extraction submodule calls the target server identifier and request timestamp corresponding to each request record in the repeated abnormal access event group, sorts the request time data according to the target server, and establishes a server access time sequence set. The access frequency calculation submodule calculates the number of requests to each target server within a unit time period based on the server access time series set, and constructs the target server access frequency series. The density region identification submodule calls the access frequency of each server in the target server access frequency sequence for the corresponding time period, uses the density peak clustering algorithm to cluster the access frequency change trend, identifies the degree of concentration of access behavior in the time period, marks the region of sudden increase in access frequency, and generates abnormal access density regions.

6. The network security situation assessment system based on data analysis according to claim 5, characterized in that, The anomaly centralized identification module includes: The path feature extraction submodule calls the URL path recorded in the structure jump request sequence, extracts the path level jump position and calculates the path character similarity, aggregates the path structure according to the source client, and obtains the path aggregated feature data group; The resource pointing judgment submodule performs classification and comparison on the target server identifier based on the path aggregation feature data group to determine whether there are multiple client request paths that tend to point to the resource location of the same server. Combined with the server list in the abnormal access density area, it filters the server behavior combination that the path aggregation target falls into the density area and generates resource pointing focused matching results. The centralized situation output submodule calls all server behavior records in the resource pointing focus matching results that have consistent pointing and clear aggregation trends, marks them as a set of target servers with similar access structures and concentrated frequency, and generates an abnormal access centralized situation.

Images