Network security situation assessment system based on data analysis

Abstract

The invention relates to the technical field of situation awareness, in particular to a network security situation assessment system based on data analysis, which comprises a structure jump identification module, an evolution trend analysis module, a repeated access judgment module, an access density statistics module and an abnormal centralized identification module. According to the method, the request sequence with the hopping structure in the access path can be extracted by continuously identifying and combining the hierarchical change of the request behavior path, and the sequence modeling analysis is performed on the change direction and continuity of the hopping behavior in combination with the evolution trend of the response parameter in the time dimension; on the basis, statistical judgment is carried out on the time interval and the repetition degree of source IP request behaviors, aggregation recognition of frequent abnormal access events is achieved, and meanwhile, an abnormal access high-incidence area is dynamically constructed in combination with the local density degree of access frequency. And the access convergence focusing trend of the target resource can be identified.

Description

Technical Field

[0001] The present invention relates to the technical field of situation awareness, and in particular to a network security situation assessment system based on data analysis. Background Art

[0002] The field of situational awareness technology involves the collection, fusion, understanding and prediction of various types of data in complex information environments, including information collection, processing, information fusion, multi-dimensional visualization, dynamic prediction and decision support.

[0003] Among them, the network security situation assessment system based on data analysis refers to a system that collects network security related data such as attack logs, access records, abnormal behavior logs, etc. and uses statistical analysis rule matching or classification algorithms to evaluate the current status of the network and judge the threat level.

[0004] When conducting network security situation assessments, existing technologies mainly rely on static rule matching or fixed indicator classification algorithms to perform statistical processing on log data. They lack dynamic mining of changes in the internal path structure of request behaviors, resulting in recognition lags or misjudgments when processing cross-level jumps or nonlinear request behaviors. When encountering situations where the source IP performs high-frequency mutated access in a short period of time, it is difficult to extract abnormal evolution characteristics from the access path structure, nor is it possible to determine whether such requests have concentrated access targets or concentrated trends. In addition, when the attack behavior manifests itself as a hidden method with slight path changes but frequent repetitions, existing methods often ignore the dynamic characteristics of behavior aggregation and access density, resulting in coarse situation judgment granularity and failure to timely capture the concentrated trend areas formed by abnormal accesses. In the face of the gradual accumulation of attack intensity, early identification and defense deployment cannot be achieved, affecting the sensitivity and effectiveness of the overall situation awareness. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a network security situation assessment system based on data analysis.

[0006] In order to achieve the above objectives, the present invention adopts the following technical solutions: A network security situation assessment system based on data analysis includes: The structural jump identification module monitors the server access log to obtain the corresponding client's request sequence. Based on each type of request URL path in the request sequence, it identifies the jump request behavior between URL paths and constructs the structural jump request sequence. An evolution trend analysis module analyzes the evolution trend of each jump request behavior in the structure jump request sequence over the entire request sequence, filters request behavior records with abnormal evolution trends, and obtains a trend anomaly set; A repeated access determination module obtains the source IP and request timestamp of the abnormal request behavior record in the trend anomaly set, determines the repeated abnormal access behavior, and obtains a repeated abnormal access event group; An access density statistics module, which counts the access frequency of the target server according to the repeated abnormal access event group, and identifies the abnormal access density area according to the access frequency; The abnormal concentration identification module obtains the abnormal access concentration situation according to whether the abnormal access density area and the URL path jump position in the structure jump request sequence point to the resource location of the same server.

[0007] As a further solution of the present invention, the structural jump request sequence includes the path jump request type, the jump level change range, and the corresponding response status attribute; the trend anomaly set includes the jump behavior change direction, the abnormal trend duration, and the trend deviation amplitude; the repeated abnormal access event group includes the number of repeated behavior occurrences, the repeated request time distribution, and the concentrated source IP of the abnormal behavior; the abnormal access density area includes the access frequency abnormal segment, the server corresponding time interval, and the access behavior cumulative amplitude; the abnormal access concentration situation includes the target resource collection location, the path aggregation distribution, and the server pointing consistency.

[0008] As a further solution of the present invention, the structural transition identification module includes: The request aggregation submodule obtains the client IP, target server ID, URL path, request timestamp, response status code, and response duration in the access log, aggregates them into a continuous request record sequence based on the client and server combination, and arranges them in chronological order to generate a client continuous request sequence; The path change judgment submodule extracts the number of path levels and calculates the level change amplitude between adjacent paths based on the URL path of each record in the client's continuous request sequence. Combined with the corresponding response status code and response duration, it judges the path structure jump of the continuous records and obtains the structure change judgment result; The jump sequence construction submodule calls the access behavior identified in the structural change judgment result, extracts the corresponding URL path, request timestamp, response status code and response duration, and combines them into a structural jump request sequence while maintaining the original time sequence.

[0009] As a further solution of the present invention, the evolution trend analysis module includes: The response feature extraction submodule calls the response status code and response duration corresponding to each request record in the structure jump request sequence, extracts the response parameters in the order of record arrangement, and obtains a response change data group set; The time trajectory construction submodule constructs a time series trajectory of the jump request behavior according to the response change data set and the request timestamp, and establishes a response state evolution trajectory set by combining the recurrence frequency of the response status code and the continuous change direction of the response duration; The trend anomaly identification submodule calls the response status code aggregation degree and response duration change trend of each trajectory in the response state evolution trajectory set, establishes an inter-sequence change pattern based on the conditional random field algorithm, performs a joint judgment on the state aggregation intensity and duration increase rate in the evolution pattern, screens the jump request behavior records that meet the evolution trend anomaly conditions, and generates a trend anomaly set.

[0010] As a further solution of the present invention, the repeated access determination module includes: The source aggregation submodule obtains the source IP and request timestamp of each abnormal request behavior record in the trend anomaly set, classifies the request behavior records into source access sequences according to the source IP, arranges the corresponding abnormal access actions in chronological order, and generates a source access sequence set; The behavior interval analysis submodule calls the request timestamp data in the source access sequence set, calculates the time interval between adjacent request behaviors, and combines the response status codes of the corresponding records to determine whether there is a behavior combination with close consecutive request intervals and consistent response status codes in the source sequence, thereby obtaining the behavior repetition analysis results; The repeated behavior identification submodule, based on the behavior repetition analysis results, screens the continuous access record groups with time intervals less than the interval threshold and consistent response status codes, performs behavior aggregation judgment based on the source IP, marks the behavior combinations that meet the repeated access conditions, and generates repeated abnormal access event groups.

[0011] As a further solution of the present invention, the access density statistics module includes: The server access extraction submodule calls the target server identifier and request timestamp corresponding to each request record in the repeated abnormal access event group, classifies and sorts the request time data by target server, and establishes a server access time series set; The access frequency calculation submodule counts the number of requests to each target server within a unit time period according to the server access time series set, and constructs a target server access frequency sequence; The density area identification submodule calls the access frequency of each server in the target server access frequency sequence for the corresponding time period, uses the density peak clustering algorithm to cluster the access frequency change trend, identifies the concentration of access behavior in the time period, marks the area with sudden increase in access frequency, and generates abnormal access density area.

[0012] As a further solution of the present invention, the abnormal centralized identification module includes: A path feature extraction submodule calls the URL path recorded in the structure jump request sequence, extracts the path level jump position and calculates the path character similarity, aggregates the path structure according to the source client, and obtains a path aggregation feature data group; The resource pointing judgment submodule performs classification comparison on the target server identifier based on the path aggregation feature data group to determine whether there are multiple client request paths tending to point to the resource location of the same server. In combination with the server list in the abnormal access density area, the module selects the server behavior combination whose path aggregation target falls into the density area to generate a resource pointing focus matching result. The concentrated situation output submodule calls the server behavior records that all point to the same server and have a clear aggregation trend in the resource pointing focus matching results, marks them as a target server set with similar access structure and concentrated frequency, and generates an abnormal access concentration situation.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by continuously identifying and combining the hierarchical changes of the request behavior path, it is possible to extract the request sequence with a jump structure in the access path, and combine the evolution trend of the response parameters in the time dimension to perform sequence modeling analysis on the change direction and continuity of the jump behavior, thereby identifying a set of abnormal requests with obvious trend deviations. On this basis, the time interval and repetition degree of the source IP request behavior are statistically judged to achieve aggregated identification of frequent abnormal access events. At the same time, combined with the local density of access frequency, a high-incidence area of ​​abnormal access is dynamically constructed, and further combined with the consistency characteristics of the path pointing characteristics and the resource ownership location, the access convergence and focusing trend of the target resource is identified, realizing the full-chain identification path from structural jump to access concentration situation. This processing logic can effectively reveal potential network attack concentration points and abnormal behavior outbreak areas, thereby improving the accuracy of situation assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a system flow chart of the present invention; Figure 2 This is a flow chart of the structural transition identification module of the present invention; Figure 3 This is a flow chart of the evolution trend analysis module of the present invention; Figure 4 This is a flow chart of a repeated access determination module of the present invention; Figure 5 This is a flow chart of the access density statistics module of the present invention; Figure 6 This is a flow chart of the abnormal centralized identification module of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0017] See also Figure 1 , the network security situation assessment system based on data analysis includes: The structural jump identification module monitors the server access log to obtain the corresponding client's request sequence. Based on each type of request URL path in the request sequence, it identifies the jump request behavior between URL paths and constructs the structural jump request sequence. The evolution trend analysis module analyzes the evolution trend of each jump request behavior in the structure jump request sequence over the entire request sequence, filters request behavior records with abnormal evolution trends, and obtains a trend anomaly set; Repeated access determination module obtains the source IP and request timestamp of abnormal request behavior records in the trend anomaly set, determines repeated abnormal access behavior, and obtains repeated abnormal access event groups; The access density statistics module counts the access frequency of the target server based on the repeated abnormal access event group and identifies the abnormal access density area based on the access frequency; The abnormal concentration identification module obtains the abnormal access concentration situation based on whether the abnormal access density area and the URL path jump position in the structure jump request sequence point to the resource location of the same server; The structural jump request sequence includes the path jump request type, the jump level change range, and the corresponding response status attributes. The trend anomaly set includes the jump behavior change direction, the abnormal trend duration, and the trend deviation amplitude. The repeated abnormal access event group includes the number of repeated behaviors, the repeated request time distribution, and the concentrated source IP of abnormal behaviors. The abnormal access density area includes the abnormal access frequency segment, the corresponding server time interval, and the cumulative amplitude of access behavior. The abnormal access concentration situation includes the target resource collection location, the path aggregation distribution, and the server pointing consistency.

[0018] See also Figure 2 , the structural jump recognition module includes: The request aggregation submodule obtains the client IP, target server ID, URL path, request timestamp, response status code, and response duration in the access log, aggregates them into a continuous request record sequence based on the client and server combination, and arranges them in chronological order to generate a client continuous request sequence; To process the access log, you first need to read the original log file or database table, parse the client IP address, target server identifier, requested URL path, request timestamp, server response status code and response duration line by line, and parse the log structure and extract fields through regular expressions for each access record. For example, an Apache log can be decomposed into "123.45.67.89--[01 / Jul / 2025:13:00:12+0000]'GET / api / user / loginHTTP / 1.1'200150". Through string segmentation and regular matching, the IP "123.45.67.89", path " / api / user / login", status code 200, and response time 150ms are extracted respectively. After the extraction is completed, you need to sort by client IP and target server. The servers are combined and aggregated, and a hash table record is used in the data structure. Each IP-server pair is a key value. All request records are constructed into an ordered list, and then the list is sorted in ascending order according to the request timestamp field. This can be achieved by converting the timestamp to UNIX time and then sorting by integer value. For example, after sorting the three records with the times of "2025-07-03 08:30:15", "2025-07-03 08:30:18", and "2025-07-03 08:31:02", they are arranged in sequence to generate a continuous request sequence. For example, when user IP "123.45.67.89" accesses "server01", its continuous request sequence may be " / login→ / dashboard→ / settings". Each path has time, status code and response time, and the order is kept in the data table.

[0019] The path change judgment submodule extracts the number of path levels and calculates the level change amplitude between adjacent paths based on the URL path of each record in the client's continuous request sequence. It then judges the path structure jump of the continuous records based on the corresponding response status code and response duration to obtain the structure change judgment result. Get the URL path of each record from the client's continuous request sequence, and count its path hierarchy by extracting the URL path fields one by one. The path hierarchy is defined as the number of non-empty segments after the " / " character in the URL is delimited. The URL can be processed by string segmentation during the extraction process. For example, the path " / api / user / login" is split into three segments ["api", "user", "login"] by the split(" / ") method, with a hierarchy of 3. Another record " / api / v1 / user / details" is split into four segments ["api", "v1", "user", "details"], with a hierarchy of 4. The path hierarchy of each record in the sequence is extracted in sequence to form a path hierarchy sequence, such as: [3, 4, 5, 3, 2], and then the change amplitude between adjacent path hierarchies is calculated in sequence. , No. Article and Variation between bars ,in, :Indicates the The path level change between a path record and the next record (the absolute value of the difference); :Indicates the first The number of path levels corresponding to the request record (that is, the number of non-empty segments obtained after dividing the URL by " / "); :Indicates the first The number of path levels of the request record, where " " indicates that at the current index Move back one position based on , that is, the next adjacent record; : Indicates the absolute value of the difference between the levels of two records.

[0020] Assume that for the path sequence “ / index(1), / user / login(2), / user / login / detail(3), / error(1)”, the number of levels is [1, 2, 3, 1], and the change range is calculated as follows: ,Right now 、 ; ,Right now 、 ; ,Right now 、 .

[0021] Integrate the paths and change amplitudes in the sequence into a structure: “ / index” → “ / user / login”: level change = 1; " / user / login" → " / user / login / detail": level change = 1; “ / user / login / detail” → “ / error”: level change = 2.

[0022] Each path pair must also be comprehensively judged based on the corresponding response status code and response time. If the response status code is an abnormal code such as 500, 404, or 403, or the response time exceeds the set benchmark value (for example, the threshold is set to 1000ms), then the path pair is judged as a "jump path". For example, the corresponding status code of the path " / user / login / detail" → " / error" in the above example is 500 and the response time is 1200ms, which meets the jump condition ( The status code is 500 and the response takes time ), so the path transition is established. The baseline value of 1000ms is based on a statistical analysis of the distribution of normal request response times. For example, if 80% of the response times in a sample fall between 0 and 900ms, then 1000ms is a practical threshold for transition discrimination. By looping through all adjacent records in the path sequence, calculating the level change, and combining it with the judgment criteria, a complete list of structural transition determination sequences is ultimately derived.

[0023] The jump sequence construction submodule calls the access behavior identified in the structural change judgment result, extracts the corresponding URL path, request timestamp, response status code and response duration, and combines them into a structural jump request sequence while maintaining the original time sequence; After receiving the structural change determination result, all corresponding records of access behaviors determined to be jumps are extracted, including their URL path, request timestamp, response status code, and response duration. These records are arranged and combined according to the original request time sequence to maintain temporal consistency. For example, three request records are identified from the structural change: the first is " / dashboard / settings", with a timestamp of "2025-07-03 09:00:12", a status code of 500, and a response duration of 1800ms; the second is " / admin / user / edit", with a time stamp of "2025-07-03 09:00:35", a status code of 403, and a response duration of 1300ms; and the third is " / logs / error / view", with a time stamp of "2025-07-03 09:01:02", a status code of 200, and a response duration of 1100ms. These records are then combined into a structural jump request sequence and arranged in a list in chronological order, ultimately forming a complete structural jump sequence record set.

[0024] See also Figure 3 , the evolution trend analysis module includes: The response feature extraction submodule calls the response status code and response duration corresponding to each request record in the structure jump request sequence, extracts the response parameters in the order of the records, and obtains a set of response change data groups; From the structure jump request sequence, read the response status code and response duration contained in each record one by one. The extraction order is strictly processed according to the record arrangement order, and no reordering or filtering is performed to maintain the integrity of the original behavior chain. When extracting, the field mapping method can be used to establish the status code sequence and response duration sequence respectively. For example, the first record is extracted with a status code of 500 and a duration of 1280ms, the second record is extracted with a status code of 403 and a duration of 1100ms, and the third record is extracted with a status code of 200 and a duration of 890ms. The status code and duration are combined in pairs to form a response change data group. Each group contains two elements: one is the status code, which indicates the feedback type of the server processing the request; the other is the response time, which indicates the server processing time in milliseconds. After the extraction is completed, it is integrated into a response change data group set according to the original request order. The exemplary execution process is: the first group is [500, 1280], the second group is [403, 1100], and the third group is [200, 890], and obtain a set of response change data groups.

[0025] The time trajectory construction submodule builds a time series trajectory of the jump request behavior based on the response change data set and the request timestamp. It also builds a response state evolution trajectory set by combining the recurrence frequency of the response status code and the continuous change direction of the response duration. After receiving the response change data set, first construct a timeline node based on the request timestamp corresponding to each set of data. The timestamps can be arranged in the original millisecond time format. For example, the timestamp of record 1 is "2025-07-03 09:00:15.128", the timestamp of record 2 is "2025-07-03 09:00:17.300", and the timestamp of record 3 is "2025-07-03 09:00:18.150". This timeline is bound to each response change data group to form a time series trajectory of the jump request. The trajectory is constructed by setting the status code and response duration combination point corresponding to each time point. In the visualization, a two-dimensional trajectory line chart can be constructed, with one axis representing time and the other axis representing the superposition of status code and response time changes. The repetition frequency of the status code in the sequence is further analyzed. For example, if the status code 500 appears twice and 200 appears once in three requests, it can be marked as a state aggregation point. At the same time, the direction of response duration change is detected. If the response time increases or decreases one by one, it constitutes a trend direction. For example, if the first record takes 1280ms, the second takes 1100ms, and the third takes 890ms, showing a downward trend, the change direction is recorded as "downward". Together with the status code aggregation, a response state evolution trajectory set is formed. The trajectory set is a collection of trajectories for all jump records in the sequence constructed according to the three dimensions of time expansion, status code labeling, and duration change trend.

[0026] The trend anomaly identification submodule uses the response status code aggregation degree and response duration change trend of each trajectory in the response status evolution trajectory set to establish the inter-sequence change pattern based on the conditional random field algorithm. It performs a joint judgment on the state aggregation strength and duration increase rate in the evolution pattern, selects the jump request behavior records that meet the evolution trend anomaly conditions, and generates a trend anomaly set. After obtaining the response state evolution trajectory set, the state code aggregation intensity and response time variation trend of each trajectory are first extracted, and the two are input as joint features. The state code aggregation intensity is defined as the ratio of the number of times a specific state code appears in the trajectory to the length of the trajectory, and the response time variation trend is defined as the average growth rate of the response time of consecutive records. For example, if the state code sequence in 3 records of a trajectory is 500, 500, and 200, the aggregation degree is , the corresponding response time is 1280ms, 1400ms, 1650ms, the response time changes are 120ms and 250ms respectively, and the average growth rate is .

[0027] Using the conditional random field model formula: ; in, :Indicates that in a given feature sequence Under the premise of predicting the label sequence The conditional probability of : Input sequence, representing the joint feature set of the current jump trajectory at each time point, where: :Indicates the first The input feature pairs of time steps include the state code aggregation and response time growth rate; : is the time step index, indicating the current position in the sequence, and its value range is ; : is the total length of the trajectory sequence, indicating the total number of time steps included; : Output label sequence, where: :Indicates the Whether the time step is trend abnormal, the value is 0 (normal) or 1 (abnormal); : Normalization factor, used to normalize the exponential sum of all possible tag sequence combinations so that the result is a probability distribution; : Weight parameter corresponding to the characteristic function: : Control state characteristic function The degree of impact on the overall probability; : Control transfer characteristic function the extent of the impact; : State feature function, used to evaluate the features at the current time point With label If the degree of matching is greater than 0.6 and the response speed is greater than 150ms and , then the value is 1, otherwise it is 0; : Transfer feature function, used to represent the label of the previous moment With the current label For example, if and , indicating that an abnormal state has newly appeared; if and , indicating that the abnormal state continues; : Indicates the current time point The label value at the previous time point, where: : Indicates the previous moment when the time step index is 1 less than the current one. Note that when This item is in the initial state and needs to be set by model initialization.

[0028] : Corresponding state characteristic function , which represents the input features (such as aggregation and response speed) and labels at the current moment The degree of match. Weight The larger the value, the greater the impact of the status features on the final anomaly judgment, which means that the model relies more on the feature value at the current moment to determine whether it is an anomaly. This is applicable to scenarios where features such as status code and response time play a decisive role in anomaly identification in abnormal requests.

[0029] : Corresponding transfer characteristic function , which represents the transfer relationship between the label at the previous moment and the label at the current moment. Weight The larger the value is, the greater the impact of the transfer feature on the result, which means that the model pays more attention to the persistence or suddenness of abnormal behavior. For example, the continuation of the abnormal state ( ) or a sudden change ( ) will be given greater importance.

[0030] Assume that the trajectory length is 3 and the characteristic sequence is: Step 1: Status code 500, aggregation = 0.67, response time = 1280ms; Step 2: Status code 500, aggregation = 0.67, response time = 1400ms; Step 3: Status code 200, aggregation = 0.67, response time = 1650ms.

[0031] The duration differences are 120ms and 250ms respectively, with an average speed increase of 185ms, which meets the abnormal speed increase threshold (150ms), and the state code concentration also meets the set threshold (0.6). , the model score is calculated as follows: ,because Satisfy the conditions, and ; , similarly; ,because , indicating the start of an anomaly; , abnormal continuation; Assumptions , ; The total score is: .

[0032] After normalization, the probability of the label sequence is obtained The largest one is identified as a trend anomaly trajectory. All trajectories with high scores that meet this feature are classified into the trend anomaly set.

[0033] The core computational steps in trend anomaly identification include extracting the clustering of response status codes and the average growth rate of response duration for each trajectory. These features are used as input, and a conditional random field model is used to score each time point in the trajectory as an abnormal state. The computational process first determines whether the features at each time point meet abnormality criteria, such as a clustering greater than 0.6 and a response growth rate greater than 150ms. If these criteria are met, the scoring function is increased. Further scoring is performed based on transition relationships between labels, such as from normal to abnormal or continuation of abnormality. Finally, the weighted sum of the state feature score and the transition feature score forms the total score for the trajectory label sequence. By scoring and normalizing all possible label sequences, the label sequence with the highest score is selected as the prediction result. This calculation result indicates which time points in the current trajectory are identified as trend anomalies, ultimately filtering out a set of jump request records with abnormal evolution trends, which the system uses to monitor and flag abnormal behavior.

[0034] See also Figure 4 , the repeated access determination module includes: The source aggregation submodule obtains the source IP and request timestamp of each abnormal request behavior record in the trend anomaly set, classifies the request behavior records into source access sequences based on the source IP, arranges the corresponding abnormal access actions in chronological order, and generates a source access sequence set; The source IP and request timestamp of each abnormal request behavior record are obtained from the trend anomaly set. By checking the source IP of each record one by one, the request records with the same source IP are classified and all abnormal requests from the same IP are arranged in timestamp order to form an access sequence for that source. For example, if the request records for the same IP have the timestamps "2025-07-03 10:00:00", "2025-07-03 10:01:00", and "2025-07-03 10:03:00", they form a set of access behaviors in chronological order, recording the abnormal requests issued by this IP at different times. Similarly, by performing similar operations on all source IPs, a source access sequence set is ultimately generated, which contains all abnormal access records sorted by time.

[0035] The behavior interval analysis submodule calls the request timestamp data in the source access sequence set, calculates the time interval between adjacent request behaviors, and combines the response status codes of the corresponding records to determine whether there is a behavior combination with close consecutive request intervals and consistent response status codes in the source sequence, thereby obtaining the behavior repetition analysis results; Extract the request timestamp data for each record from the source access sequence set and calculate the time interval between adjacent request behaviors. For example, assuming the request timestamps of two adjacent records are "2025-07-03 10:01:00" and "2025-07-03 10:03:00," the time interval between them is 2 minutes. Based on this, the response status code of each record is also used to determine the continuity of the behavior. For example, if the response status code of the first two records is 200, it is considered that these two requests may belong to the same behavior pattern. Based on the matching of the calculated interval time and the response status code, further analysis is conducted within the source sequence to determine whether there are any behavior combinations with small intervals between consecutive requests and consistent status codes. For example, if the time interval between multiple requests is less than 1 minute and the response status code is consistent, this combination is marked as having a high degree of behavior repetition.

[0036] The repeated behavior identification submodule, based on the behavior repetition analysis results, screens groups of consecutive access records with time intervals less than the interval threshold and consistent response status codes, aggregates behaviors by source IP address, and marks behaviors that meet the repeated access criteria to generate repeated abnormal access event groups. Based on the results of the behavior interval analysis, we further filter out those continuous access record groups whose time intervals are less than the set interval threshold (such as 1 minute) and whose response status codes are consistent. For each filtered behavior combination, it is clustered and judged according to the source IP. If the request behavior of the same source IP meets the conditions of repeated access, these behaviors are marked as repeated abnormal access events to form a repeated abnormal access event group. At this time, assuming that multiple requests from a certain IP address occur continuously within 1 minute, and their response status codes are 200, and these behaviors appear repeatedly, these requests will be identified as abnormal access events, and the repeated access behavior of the source IP will be recorded. Finally, through screening and marking, a group of abnormal events focusing on high-frequency repeated access behaviors is obtained.

[0037] See also Figure 5 , access density statistics modules include: The server access extraction submodule calls the target server identifier and request timestamp corresponding to each request record in the repeated abnormal access event group, organizes the request time data by target server, and establishes a server access time series set; Obtain the target server identifier and request timestamp of each request record from the repeated abnormal access event group. First, pair the target server identifier and the corresponding timestamp of each record, and classify and organize the request time data by target server. For example, suppose a target server receives access from multiple requests at different times, with the timestamps "2025-07-03 11:00:00", "2025-07-03 11:05:00", and "2025-07-03 11:08:00" respectively. These request records are classified as the access time series of the target server and arranged in chronological order. By performing similar operations on all servers, the request timestamps of each server are formed into an independent access time series, and finally a complete set of server access time series is obtained.

[0038] The access frequency calculation submodule counts the number of requests per target server per unit time period based on the server access time series set, and constructs the target server access frequency sequence; By traversing the set of server access time series, the number of requests per unit time period for each target server is counted. The unit time period can be set to 1 minute, 5 minutes, or longer. The specific time period selected depends on the granularity requirements of anomaly detection. For example, for a target server's access time series, if there are 5 requests within 1 minute, then the server has been accessed 5 times within 1 minute. By traversing all time periods and calculating the number of requests within each time period, an access frequency sequence is ultimately constructed for each target server. This access frequency sequence reflects the request load intensity of the server in different time periods and can subsequently be used to detect time periods with abnormally high access frequencies.

[0039] The density region identification submodule calls the access frequency of each server in the target server access frequency sequence for the corresponding time period, uses the density peak clustering algorithm to cluster the access frequency change trend, identifies the concentration of access behavior in the time period, marks the area with sudden increase in access frequency, and generates abnormal access density area; Based on the target server's access frequency sequence, a density peak clustering algorithm is used to cluster the access frequency trends of each server. This algorithm identifies areas of high access frequency concentration within certain time periods and marks them as areas of abnormal access density. The clustering process involves the following steps: Access frequency data for each time period is extracted from the access frequency sequence. For example, suppose the request frequency for a server in different time periods is [1, 3, 5, 6, 2, 8, 10, 15]. These data represent the server's access frequency in each time period. The density peak clustering algorithm clusters data points based on their local density and relative distance. The density of each data point is represented by the number of points close to it, while the distance between that point and other points determines the cluster boundaries.

[0040] First, the access frequency density is calculated for each time period. For example, if the request frequency in a time period is 10 and there are multiple adjacent time periods with high frequencies, then the density in that time period is high; conversely, the density is low. Next, the difference in access frequency between each time period and other time periods is calculated. For example, if the access frequency in one time period is 10 and in another time period is 1, then the difference between them is large. Based on density and distance, the density peak clustering algorithm divides clusters based on the relative density and distance between data points. Suppose, after calculation, the access frequency is found to increase sharply during a certain period (such as the period where the frequency reaches 10 or above), this period is identified as a "high density" area and marked as an abnormal access density area. This method can identify time periods with sudden increases in access frequency, which often indicate abnormal access behavior, such as DDoS attacks or traffic bursts. Ultimately, the clustering algorithm marks time periods with significant frequency concentrations and sudden increases as abnormal access density areas. Marking these density areas facilitates subsequent anomaly detection and incident response.

[0041] For example, if the access frequencies of a server in the time periods [6, 7] and [7, 8] are 10 and 15 respectively, and the frequencies in the time periods around these time periods (such as [5, 6] and [8, 9]) are lower, the access frequency during this period will be considered a sudden increase and marked as an abnormal access density area.

[0042] See also Figure 6 ,The abnormal centralized identification module includes: The path feature extraction submodule calls the URL path recorded in the structure jump request sequence, extracts the path level jump position and calculates the path character similarity, aggregates the path structure according to the source client, and obtains the path aggregation feature data group; Get the URL path of each record from the structure-jumping request sequence and extract the path level jump position. The path level jump position refers to the place in the request sequence where the number of levels of the path changes significantly. For example, if a request path is " / api / user / login" with a level of 3, and the next request path is " / api / user / details" with a level of 4, the jump position occurs at the position where the path jumps from 3 to 4. For each pair of consecutive request paths, the module calculates the similarity of their path characters and uses methods such as Jaccard similarity to calculate the string similarity of adjacent paths.

[0043] Path character similarity calculation formula: ; in, and It is a set of two paths (the elements in the path string can be individual segments of the path, such as the segments of " / api / user / login" are "api", "user", "login"). Represents two path sets and The size of the intersection of , that is, the number of path segments they contain in common. Represents a path collection and The size of the union of , that is, the total number of path segments.

[0044] Suppose there are two paths: Path A: " / api / user / login", Path B: " / api / user / settings", segmentation means: the segmentation of path A is , the segments of path B are .

[0045] Compute intersection and union: Intersection: (Path A and Path B both contain these two path segments), union: , calculate the Jaccard similarity: Therefore, the character similarity between Path A and Path B is 0.5, indicating that their path structures are 50% similar. Next, we perform structural aggregation on the paths by source client, summarize all request paths issued by the same client, and generate a path aggregation feature data set based on the hierarchical jump position and similarity of the paths. The path aggregation feature data set includes each client's access behavior, path hierarchy changes, and path character similarity.

[0046] The resource pointing judgment submodule performs classification and comparison on the target server identifier based on the path aggregation feature data group to determine whether there are multiple client request paths tending to point to the resource location of the same server. Combined with the server list in the abnormal access density area, it screens the server behavior combinations whose path aggregation targets fall into the density area to generate the resource pointing focus matching result. Based on the path aggregation feature data group, the target server identification is classified and compared. First, the target server identification information in each path aggregation feature data group is extracted and classified. If the request paths of multiple clients tend to point to the same server resource location, it indicates that there is a certain concentration in the access behavior of these clients. For example, the path aggregation data groups of multiple different clients show that their request target servers all point to the same resource, which may be caused by a shared resource or public service. Then, combined with the list of servers in the abnormal access density area, the submodule will filter out the server behavior combinations whose path aggregation targets fall into the density area. Servers in the density area usually mean that the servers in the area have experienced high-frequency requests within a certain period of time and are therefore more susceptible to abnormal behavior. Finally, the resource pointing focus matching results are generated, reflecting which server and access behavior combinations are more closely related to the abnormal density area.

[0047] The centralized situation output submodule calls the resource pointing to focus on the matching results, all server behavior records with consistent pointing and clear aggregation trends, marking them as a set of target servers with similar access structures and concentrated frequencies, and generating abnormal access concentration trends; Receive data from the resource-pointing focused matching results, analyze and mark server behavior records with consistent pointing and clear aggregation trends. These records show that multiple clients frequently access the same server in the same time period, forming a pattern of convergent access structure. Through centralized analysis of these records, the submodule will mark these server behaviors as a set of target servers with concentrated frequencies, which means that these servers play a centralized role in abnormal access behaviors. These servers often face bursts of requests from multiple clients, which may indicate potential abnormal events such as DDoS attacks, crawler activities, or abnormal traffic. After generating the abnormal access concentration trend, it can provide a clear view showing which servers have the most concentrated access requests in abnormal access behaviors, thereby helping system administrators identify and take appropriate countermeasures.

[0048] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A network security situation assessment system based on data analysis, characterized by: The system comprises: The structural jump identification module monitors the server access log to obtain the corresponding client's request sequence. Based on each type of request URL path in the request sequence, it identifies the jump request behavior between URL paths and constructs the structural jump request sequence. An evolution trend analysis module analyzes the evolution trend of each jump request behavior in the structure jump request sequence over the entire request sequence, filters request behavior records with abnormal evolution trends, and obtains a trend anomaly set; A repeated access determination module obtains the source IP and request timestamp of the abnormal request behavior record in the trend anomaly set, determines the repeated abnormal access behavior, and obtains a repeated abnormal access event group; An access density statistics module, which counts the access frequency of the target server according to the repeated abnormal access event group, and identifies the abnormal access density area according to the access frequency; The abnormal concentration identification module obtains the abnormal access concentration situation according to whether the abnormal access density area and the URL path jump position in the structure jump request sequence point to the resource location of the same server.

2. The network security situation assessment system based on data analysis according to claim 1 is characterized in that: The structural jump request sequence includes the path jump request type, the jump level change range, and the corresponding response status attributes. The trend anomaly set includes the jump behavior change direction, the abnormal trend duration, and the trend deviation amplitude. The repeated abnormal access event group includes the number of repeated behaviors, the repeated request time distribution, and the concentrated source IP of the abnormal behavior. The abnormal access density area includes the access frequency abnormal segment, the server corresponding time interval, and the access behavior cumulative amplitude. The abnormal access concentration situation includes the target resource collection location, the path aggregation distribution, and the server pointing consistency.

3. The network security situation assessment system based on data analysis according to claim 1 is characterized in that: The structure jump identification module includes: The request aggregation submodule obtains the client IP, target server ID, URL path, request timestamp, response status code, and response duration in the access log, aggregates them into a continuous request record sequence based on the client and server combination, and arranges them in chronological order to generate a client continuous request sequence; The path change judgment submodule extracts the number of path levels and calculates the level change amplitude between adjacent paths based on the URL path of each record in the client's continuous request sequence. Combined with the corresponding response status code and response duration, it judges the path structure jump of the continuous records and obtains the structure change judgment result; The jump sequence construction submodule calls the access behavior identified in the structural change judgment result, extracts the corresponding URL path, request timestamp, response status code and response duration, and combines them into a structural jump request sequence while maintaining the original time sequence.

4. The network security situation assessment system based on data analysis according to claim 3 is characterized in that: The evolution trend analysis module includes: The response feature extraction submodule calls the response status code and response duration corresponding to each request record in the structure jump request sequence, extracts the response parameters in the order of record arrangement, and obtains a response change data group set; The time trajectory construction submodule constructs a time series trajectory of the jump request behavior according to the response change data set and the request timestamp, and establishes a response state evolution trajectory set by combining the recurrence frequency of the response status code and the continuous change direction of the response duration; The trend anomaly identification submodule calls the response status code aggregation degree and response duration change trend of each trajectory in the response state evolution trajectory set, establishes an inter-sequence change pattern based on the conditional random field algorithm, performs a joint judgment on the state aggregation intensity and duration increase rate in the evolution pattern, screens the jump request behavior records that meet the evolution trend anomaly conditions, and generates a trend anomaly set.

5. The network security situation assessment system based on data analysis according to claim 4 is characterized in that: The repeated access determination module includes: The source aggregation submodule obtains the source IP and request timestamp of each abnormal request behavior record in the trend anomaly set, classifies the request behavior records into source access sequences according to the source IP, arranges the corresponding abnormal access actions in chronological order, and generates a source access sequence set; The behavior interval analysis submodule calls the request timestamp data in the source access sequence set, calculates the time interval between adjacent request behaviors, and combines the response status codes of the corresponding records to determine whether there is a behavior combination with close consecutive request intervals and consistent response status codes in the source sequence, thereby obtaining the behavior repetition analysis results; The repeated behavior identification submodule, based on the behavior repetition analysis results, screens the continuous access record groups with time intervals less than the interval threshold and consistent response status codes, performs behavior aggregation judgment based on the source IP, marks the behavior combinations that meet the repeated access conditions, and generates repeated abnormal access event groups.

6. The network security situation assessment system based on data analysis according to claim 5 is characterized in that: The access density statistics module includes: The server access extraction submodule calls the target server identifier and request timestamp corresponding to each request record in the repeated abnormal access event group, classifies and sorts the request time data by target server, and establishes a server access time series set; The access frequency calculation submodule counts the number of requests to each target server within a unit time period according to the server access time series set, and constructs a target server access frequency sequence; The density area identification submodule calls the access frequency of each server in the target server access frequency sequence for the corresponding time period, uses the density peak clustering algorithm to cluster the access frequency change trend, identifies the concentration of access behavior in the time period, marks the area with sudden increase in access frequency, and generates abnormal access density area.

7. The network security situation assessment system based on data analysis according to claim 6 is characterized in that: The abnormal centralized identification module includes: A path feature extraction submodule calls the URL path recorded in the structure jump request sequence, extracts the path level jump position and calculates the path character similarity, aggregates the path structure according to the source client, and obtains a path aggregation feature data group; The resource pointing judgment submodule performs classification comparison on the target server identifier based on the path aggregation feature data group to determine whether there are multiple client request paths tending to point to the resource location of the same server. In combination with the server list in the abnormal access density area, the module selects the server behavior combination whose path aggregation target falls into the density area to generate a resource pointing focus matching result. The concentrated situation output submodule calls the server behavior records that all point to the same server and have a clear aggregation trend in the resource pointing focus matching results, marks them as a target server set with similar access structure and concentrated frequency, and generates an abnormal access concentration situation.

Images