Network security defense decision-making method

Abstract

The invention discloses a network security defense decision-making method, and particularly relates to the technical field of network security. The method comprises the following steps: acquiring user account access behavior data in a network environment and intrusion observation data output by an intrusion detector, and performing data cleaning and feature extraction to obtain historical trust evidence sequence data; performing time sequence feature analysis by using a recurrent neural network, and identifying an abnormal behavior mode of the account; trust evaluation is carried out based on an actor-commentator reinforcement learning algorithm, and account real-time trust score data is generated; performing association analysis by combining the behavior abnormal mode feature data and the real-time trust score data to obtain potential threat feature data of the user account; evaluating the effectiveness of the defense strategy by adopting a strategy evaluation network, and generating defense strategy evaluation data; and generating an optimal network defense decision according to the potential threat feature data of the user account and the defense strategy evaluation data. According to the method, the detection accuracy of potential network intrusion and the effectiveness of defense measures are improved.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and more specifically, to a network security defense decision-making method. Background Technology

[0002] With the increasing complexity of the network environment and the diversification of network attack methods, traditional static rule-based network defense methods are no longer effective in dealing with dynamic, complex and constantly changing network intrusion threats.

[0003] Traditional network security defense methods often struggle to deeply analyze the potential temporal correlation between user account access behavior and intrusion observation, resulting in the inability to timely identify and accurately assess potential security threats faced by user accounts. This leads to delayed response and insufficient effectiveness of defense strategies, posing serious risks to network security. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a network security defense decision-making method to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A network security defense decision-making method includes the following steps: S1: Collect user account access behavior data and intrusion observation data output by intrusion detectors in the network environment, and form standardized historical trust evidence sequence data through data cleaning and feature extraction; S2: Based on historical trust evidence sequence data, recurrent neural networks are used to analyze the temporal change characteristics of account access behavior, identify abnormal user behavior patterns, and output abnormal behavior pattern feature data. S3: Based on historical trust evidence sequence data, use the actor-critic reinforcement learning algorithm to perform trust assessment on intrusion observation data and generate real-time trust score data for accounts; S4: Perform correlation analysis on abnormal behavior pattern feature data and real-time account trust score data to evaluate the correlation between abnormal behavior patterns and trust scores, and generate potential threat feature data for user accounts. S5: Based on the potential threat characteristics data of user accounts, use the policy evaluation network to evaluate the effectiveness of the defense strategy and output the defense strategy evaluation data; S6: Adjust the defense strategy based on the user account potential threat characteristics data and defense strategy evaluation data, and generate the optimal network defense decision through the policy decision network.

[0006] In a preferred embodiment, S1 specifically refers to: Collect user account access behavior data and intrusion observation data output by intrusion detectors in the network environment; Data cleaning is performed on user account access behavior data and intrusion observation data; Extract access behavior feature data that characterizes the patterns of user account behavior and intrusion observation data that characterize the patterns of intrusion observation from user account access behavior data and intrusion observation data after data cleaning. Access behavior characteristic data and intrusion characteristic data are sorted and merged according to the time of occurrence to form a standardized historical trust evidence sequence data.

[0007] In a preferred embodiment, S2 specifically refers to: After normalizing the standardized historical trust evidence sequence data, a time series matrix is ​​constructed according to time steps; The time series matrix is ​​sequentially input into a recurrent neural network containing an input layer, a recurrent unit layer, and a fully connected output layer. The recurrent unit layer extracts the temporal variation feature vector of account access behavior through recursive calculation. The difference between the time-series change feature vector of account access behavior and the preset baseline feature vector of normal access behavior is calculated to obtain behavior difference data. The behavioral difference data is compared with the anomaly detection threshold to determine the abnormal behavior index; Generate behavioral anomaly pattern feature data based on the abnormal behavior index.

[0008] In a preferred embodiment, S3 specifically refers to: Historical trust evidence sequence data and intrusion observation data are synchronously matched and concatenated into a comprehensive state vector according to the occurrence time. The comprehensive state vector is input into the actor-critic reinforcement learning algorithm framework, which consists of an actor network and a critic network; the actor network outputs a defensive action vector, and the critic network calculates action value assessment data based on the comprehensive state vector and the defensive action vector; The action value assessment data is linearly normalized to obtain the account's real-time trust score data.

[0009] In a preferred embodiment, S4 specifically refers to: The abnormal behavior pattern feature data and the real-time trust score data of accounts are aligned by time step and concatenated into a joint analysis matrix; Calculate the correlation coefficient between the behavioral anomaly pattern vector at each time step in the joint analysis matrix and the real-time trust score of the corresponding account to obtain the correlation coefficient sequence data; The correlation coefficient sequence data is compared with the potential threat determination threshold, and the time step index where the correlation coefficient is greater than the potential threat determination threshold is marked to generate a potential threat index set. Based on the potential threat index set, extract the behavioral anomaly pattern vector and the real-time trust score of the account corresponding to each time step index from the joint analysis matrix to construct the potential threat feature data of user accounts.

[0010] In a preferred embodiment, S5 specifically refers to: The potential threat characteristics data of user accounts are input into the policy evaluation network; the policy evaluation network consists of an input layer, a hidden layer, and an output layer. The input layer receives potential threat feature data of user accounts, the hidden layer uses a fully connected structure to perform feature transformation and nonlinear mapping on the potential threat feature data of user accounts, and the output layer performs linear mapping on the nonlinear mapping result output by the hidden layer to generate the initial evaluation value of the defense strategy. The initial evaluation values ​​of the defense strategy are normalized to generate defense strategy evaluation data.

[0011] In a preferred embodiment, S6 specifically refers to: By concatenating user account potential threat characteristics data with defense strategy evaluation data, a strategy decision input vector is constructed. The policy decision network consists of a decision input layer, a policy hidden layer, and a decision output layer; The decision input layer receives the policy decision input vector, the policy hidden layer performs feature transformation on the policy decision input vector through a fully connected structure and activation function and outputs the policy hidden layer feature vector, and the decision output layer linearly maps the policy hidden layer feature vector and outputs a set of defense decision candidate policies. The candidate strategies in the set of defense decision candidate strategies are compared with the preset defense strategy selection criteria to select the optimal network defense decision; Adjust strategies based on optimal network defense decisions.

[0012] The technical effects and advantages of the network security defense decision-making method of the present invention are as follows: This system collects and cleans user account access behavior data and intrusion observation data, extracts high-quality historical trust evidence sequences, and provides an accurate and complete time-series data foundation. It introduces a recurrent neural network to deeply mine the temporal variation characteristics of access behavior, accurately identifying abnormal behavior patterns and improving the sensitivity and accuracy of anomaly detection. An actor-critic reinforcement learning algorithm is used to assess the trust level of intrusion observation data, effectively quantifying the real-time trust level of user accounts, enabling defense decisions to be adjusted based on dynamic trustworthiness indicators. An association system is used to analyze the correlation between abnormal patterns and trust scores, generating potential threat feature data and enhancing the comprehensiveness of threat assessment. A policy evaluation network is used to evaluate the effectiveness of current defense strategies, outputting defense strategy evaluation data to provide a basis for strategy optimization. Based on potential threat features and policy evaluation data, the defense strategy is dynamically adjusted, and the optimal defense decision is generated through a policy decision network, achieving real-time and accurate responses to different threat scenarios. This improves the accuracy of anomaly detection and risk assessment, enables adaptive optimization of defense strategies, and enhances the overall security protection capabilities of the network environment. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of a network security defense decision-making method according to the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0015] Example Figure 1 This invention provides a network security defense decision-making method, which includes the following steps: S1: Collect user account access behavior data and intrusion observation data output by intrusion detectors in the network environment, and form standardized historical trust evidence sequence data through data cleaning and feature extraction; S2: Based on historical trust evidence sequence data, recurrent neural networks are used to analyze the temporal change characteristics of account access behavior, identify abnormal user behavior patterns, and output abnormal behavior pattern feature data. S3: Based on historical trust evidence sequence data, use the actor-critic reinforcement learning algorithm to perform trust assessment on intrusion observation data and generate real-time trust score data for accounts; S4: Perform correlation analysis on abnormal behavior pattern feature data and real-time account trust score data to evaluate the correlation between abnormal behavior patterns and trust scores, and generate potential threat feature data for user accounts. S5: Based on the potential threat characteristics data of user accounts, use the policy evaluation network to evaluate the effectiveness of the defense strategy and output the defense strategy evaluation data; S6: Adjust the defense strategy based on the user account potential threat characteristics data and defense strategy evaluation data, and generate the optimal network defense decision through the policy decision network.

[0016] S1: Collect user account access behavior data and intrusion observation data output by intrusion detectors in the network environment, and form standardized historical trust evidence sequence data through data cleaning and feature extraction, including: Collect user account access behavior data and intrusion observation data output by intrusion detectors in the network environment; Data acquisition devices installed within the network environment monitor and collect information from all user accounts within that environment. The network environment can be a corporate office LAN or a wide area network (WAN) provided by an internet service provider. For example, if a user account is used by an employee to access resources on the corporate LAN, the data acquisition device continuously collects and stores access behavior data generated by this user account. This includes recording the time a user account logs into the network system, the number of requests to access the company's internal mail server, and the type and frequency of operations when accessing specific file servers. The data acquisition device also interacts with intrusion detectors deployed in the same network environment. These detectors monitor user account access behavior based on specific detection rules, such as preset abnormal behavior detection rules, like thresholds for abnormally high-frequency login failures, rules for multiple requests to access sensitive servers within a short period, or rules for abnormal changes in file access permissions. When an intrusion detector detects that a user account has repeatedly failed to log into a sensitive resource server within a certain period, triggering a predetermined threshold, the detector generates intrusion observation data recording this abnormal behavior and outputs it to the data acquisition device.

[0017] Data cleaning is performed on user account access behavior data and intrusion observation data; Data cleaning involves removing outliers, redundant records, and missing values ​​generated during the recording process of user account access behavior data and intrusion observation data. For example, in the process of recording user account login times, if there are timestamp data errors or missing data due to network latency, the data cleaning process will employ appropriate data correction strategies. For outlier timestamp data, the data cleaning process will replace the outlier data with the average of the data before and after the timestamp; for missing data, the data cleaning process will fill in the missing data by interpolating the behavioral feature data of the preceding and following records to ensure data integrity. Regarding potentially redundant records in intrusion observation data, such as duplicate observation records generated by intrusion detectors triggering the same detection rule multiple times within a short period, the data cleaning process will use deduplication strategies to consolidate duplicate intrusion observation records into a single record.

[0018] Extract access behavior feature data that characterizes the patterns of user account behavior and intrusion observation data that characterize the patterns of intrusion observation from user account access behavior data and intrusion observation data after data cleaning. Access behavior characteristic data includes user account access frequency characteristics, access time period distribution characteristics, single session duration characteristics, access resource type characteristics, login count, and login failure count. For example, access frequency characteristics refer to the total number of times a user account performs network resource access operations per unit time, such as the average number of times an hour accesses an enterprise's internal mail server; access time period distribution characteristics refer to the distribution of user account network resource access behavior at different times of the day, such as a distribution pattern where access frequency is higher in the morning and lower in the afternoon. Intrusion characteristic data includes specific abnormal behavior pattern characteristics, such as the number of multiple login failure events per unit time, the number of abnormal permission requests, the abnormal frequency of access to sensitive resources, and abnormal traffic surges. For example, intrusion characteristic data may include the characteristic that a user account's consecutive login failures exceed a threshold within 5 minutes or the characteristic of an abnormal increase in the number of accesses to sensitive file servers within a specific time period.

[0019] Access behavior characteristic data and intrusion characteristic data are sorted and merged according to the time of occurrence to form standardized historical trust evidence sequence data; Based on the actual occurrence time of each data feature, access behavior feature data and intrusion feature data are uniformly sorted by time. Assuming a user account accesses the mail server at 8:00 AM, generating access behavior feature data; at 8:05 AM, the intrusion detector generates intrusion observation data and extracts intrusion feature data; and at 8:10 AM, new access behavior feature data is generated again. In the standardized historical trust evidence sequence data, these three feature data are recorded in chronological order as the 8:00 AM access behavior feature data, the 8:05 AM intrusion feature data, and the 8:10 AM access behavior feature data, and so on, generating a complete historical trust evidence sequence data. Furthermore, the standardization process also includes converting the value range of each feature data into a uniform value interval, such as [0,1], through a specific numerical mapping method.

[0020] S2: Based on historical trust evidence sequence data, a recurrent neural network is used to analyze the temporal variation characteristics of account access behavior, identify abnormal user behavior patterns, and output abnormal behavior pattern feature data, including: After normalizing the standardized historical trust evidence sequence data, a time series matrix is ​​constructed according to time steps; Standardized historical trust evidence sequence data is composed of user account access behavior feature data and intrusion feature data arranged and merged in chronological order. Each data unit consists of feature data and includes a timestamp indicating its generation. The normalization process involves standardizing the numerical scale of the features contained in each data unit within the historical trust evidence sequence data, thus limiting the numerical values ​​of the feature data to a uniform range and eliminating orders-of-magnitude differences between different feature data. For example, the access frequency feature in user account access behavior feature data and the number of failed login attempts per unit time feature in intrusion feature data may have a range of tens of times, while the number of failed login attempts per unit time may only have a range of single digits. Therefore, normalization is required, for example, by using a minimum-maximum value mapping method to uniformly map the access frequency feature values ​​and the number of failed login attempts to a range between 0 and 1. The same processing is performed on each type of feature in all data units to obtain a data sequence with a uniform scale. Then, a time series matrix is ​​constructed using the timestamp of each data unit as the row index and the normalized feature data as the row elements. Each row in the time series matrix represents a data unit at a specific time point, while each column represents the numerical sequence of a certain feature over all time steps after normalization.

[0021] The time series matrix is ​​sequentially input into a recurrent neural network containing an input layer, a recurrent unit layer, and a fully connected output layer. The recurrent unit layer extracts the temporal variation feature vector of account access behavior through recursive calculation. A recurrent neural network (RNN) consists of an input layer, recurrent unit layers, and a fully connected output layer. The input layer receives and reads the data unit vector for each time step in the time series matrix, such as the data unit corresponding to a specific timestamp, and passes the data unit vector to the recurrent unit layer. The recurrent unit layer processes the data input recursively; that is, the input data unit vector of the current time step and the hidden state vector obtained from the previous time step are used together in the calculation to generate a new hidden state vector for the current time step. Taking the first time step of the matrix as an example, when the recurrent unit layer calculates for the first time, since there is no data from the previous time step, the initial hidden state vector is usually set to zero. Thereafter, the hidden state vector is continuously updated with each time step. For example, when the data unit of the second time step enters the recurrent unit layer, it uses both the hidden state vector generated in the first time step and the data unit vector of the second time step for calculation; when the data unit of the third time step enters, it uses the hidden state vector generated in the second time step and the data unit vector of the third time step for calculation; and so on, until the data units of all time steps have been calculated. Through the above recursive approach, the recurrent unit layer extracts the changing patterns of historical data sequences, capturing the user account access behavior change patterns contained in the historical data sequences and storing them in the form of feature vectors in the hidden state vector. Finally, the recurrent unit layer passes the generated hidden state vector to the fully connected output layer, which performs a linear mapping operation on the hidden state vector, ultimately outputting a temporal change feature vector describing the temporal change patterns of user account access behavior.

[0022] The difference between the time-series change feature vector of account access behavior and the preset baseline feature vector of normal access behavior is calculated to obtain behavior difference data. The baseline feature vector for normal access behavior is a feature vector representing the normal access behavior pattern of user accounts, formed during the initialization phase of the network security defense decision system through statistical analysis and calculation of normal access behavior feature vectors in historical data. For example, it can be calculated by statistically analyzing the access behavior data of user accounts not marked as abnormal over a long period, calculating the average or median of normal access behavior features in each dimension, and combining these to form the baseline feature vector. The difference calculation involves calculating the numerical difference between the time-series change feature vector and the baseline feature vector for normal access behavior. For example, the Euclidean distance method can be used, which calculates the squared difference between the time-series change feature vector and the baseline feature vector for normal access behavior in each dimension, then sums the squared differences in all dimensions and takes the square root to obtain the difference value, representing the degree of difference between the current account's access behavior and normal behavior.

[0023] The behavioral difference data is compared with the anomaly detection threshold to determine the abnormal behavior index; The anomaly detection threshold is a fixed value preset during the deployment of the network security defense system, representing the boundary used to distinguish between normal and abnormal behavior. When the behavior difference data exceeds the anomaly detection threshold, the current time step index is marked as an abnormal behavior index. For example, if the difference data value ranges from 0 to 1, the anomaly detection threshold is 0.6, and the calculated difference data is 0.75, then the current time step is marked as an abnormal behavior index; if the difference data is 0.3, then the time step does not exceed the threshold and is not marked as an abnormal behavior index.

[0024] Generate behavioral anomaly pattern feature data based on the abnormal behavior index; Based on all time steps marked as anomalous behavior indices, the corresponding temporal change feature vectors are extracted from the time series matrix, and the temporal change feature vectors are combined into an anomalous pattern feature data set, which is the final output behavioral anomalous pattern feature data.

[0025] S3: Based on historical trust evidence sequence data, the actor-critic reinforcement learning algorithm is used to perform trust assessment on intrusion observation data, generating real-time account trust score data, including: Historical trust evidence sequence data and intrusion observation data are synchronously matched and concatenated into a comprehensive state vector according to the occurrence time. Each data unit in the historical trust evidence sequence data is matched with each data unit in the intrusion observation data using the timestamp recorded within the data unit. For example, if a data unit in the historical trust evidence sequence data contains characteristic data of a user account accessing a specific database resource at 10:00 AM, and the intrusion observation data also records abnormal permission request observation data of the user account accessing the database resource at 10:00 AM, then these two data units are synchronized and matched to form synchronized data units with the same timestamp. If a data unit for a certain timestamp exists only in the historical trust evidence sequence data but not in the corresponding record in the intrusion observation data, or if a record exists only in the intrusion observation data but not in the corresponding record in the historical trust evidence sequence data, then the missing data unit is padded with a vector of zero value or a specific empty vector to maintain dimensional consistency among all synchronized data units. After synchronizing and completing all data units in the historical trust evidence sequence data and intrusion observation data, the user account access behavior feature data, intrusion feature data, and intrusion observation data contained in each pair of synchronized data units are horizontally concatenated. That is, they are concatenated in matrix form along the feature dimension of the data unit vector, ultimately forming a complete comprehensive state vector. Each element in the comprehensive state vector reflects the behavioral characteristics exhibited by the user account when accessing resources in the network environment, as well as the observation information of the intrusion detector when the behavior occurs. This allows the comprehensive state vector to simultaneously reflect the detailed correspondence between user account access behavior and corresponding intrusion observation data, revealing the user account's access status and potential intrusion risks at a specific moment.

[0026] The comprehensive state vector is input into the actor-critic reinforcement learning algorithm framework, which consists of an actor network and a critic network; the actor network outputs a defensive action vector, and the critic network calculates action value assessment data based on the comprehensive state vector and the defensive action vector; The actor-critic reinforcement learning algorithm framework comprises an actor network and a critic network. The actor network is responsible for generating defensive action vectors to execute network defense measures, while the critic network is responsible for evaluating the effectiveness of the defensive action vectors output by the actor network. The combined state vector is first input to the actor network. The actor network consists of an input layer, hidden layers, and an output layer. The input layer takes the combined state vector as input, and the hidden layers perform feature transformation and nonlinear mapping on the input combined state vector through a layer-by-layer fully connected structure, such as using a fully connected neural network structure with a nonlinear activation function. Finally, the output layer generates defensive action vectors corresponding to the network security defense measures. Each dimension of the defensive action vector represents the execution strength or probability of a network defense measure, such as password reset measures, temporary account locking measures, and access restriction measures for user accounts. Each defensive action is represented by a numerical value indicating the degree or likelihood of its implementation. After the defensive action vectors are generated, they are input into the critic network along with the combined state vector. The critic network also consists of an input layer, hidden layers, and an output layer. The input layer takes both the overall state vector and the defensive action vector as input. The hidden layer performs feature fusion and non-linear processing on the input data and learns the correlation between the input data and the action effect. The output layer uses a linear mapping method to finally output action value assessment data. The action value assessment data is numerical data, representing the defensive effectiveness and potential cost of the corresponding defensive action vector under the current overall state. For example, if the defensive action vector output by the actor network represents a high strength of the temporary account lockout measure, the critic network may output a high action value assessment value based on the information that the current user account risk level is high in the overall state vector, indicating a high defensive effectiveness.

[0027] The action value assessment data is linearly normalized to obtain the account's real-time trust score data; Action value assessment data, due to its potentially wide numerical range, requires linear normalization to facilitate unified decision analysis by network security defense systems. Linear normalization involves adjusting the numerical scale of the action value assessment data using a linear mapping method, mapping the data to a uniform numerical range, such as between 0 and 1. This ensures that the real-time trust score of an account represents its level of trustworthiness in a given real-time state; a higher score indicates a more secure and trustworthy account, while a lower score suggests the account may be under potential network intrusion risk. Once calculated and updated in real-time, the account trust score can be directly used for real-time adjustment and execution of network defense strategies.

[0028] S4: Perform correlation analysis on abnormal behavior pattern feature data and real-time account trust score data to assess the correlation between abnormal behavior patterns and trust scores, and generate potential threat feature data for user accounts, including: The abnormal behavior pattern feature data and the real-time trust score data of accounts are aligned by time step and concatenated into a joint analysis matrix; First, the abnormal behavior pattern feature data and the real-time trust score data of the account are matched according to their respective timestamps. That is, each feature vector or data point is aligned according to its timestamp. For each time step, the abnormal behavior pattern feature data vector and the real-time trust score data of the account are horizontally concatenated to form a data vector. Finally, the concatenated data vectors are arranged in the order of all time steps to form a joint analysis matrix with temporal order and consistent dimensions. For example, in a network environment, suppose that a certain time step of the abnormal behavior pattern feature data records the abnormal behavior feature vector of a user account when accessing a sensitive database at 9:00 AM, and the real-time trust score data of the account also contains the trust score data corresponding to the user account's access behavior at 9:00 AM, then the abnormal behavior pattern feature vector at 9:00 AM is horizontally concatenated with the real-time trust score of the account at the corresponding time step to form a unified data vector. This process is repeated for all time step data to generate a complete joint analysis matrix. Each row of the joint analysis matrix represents a specific time step, and each column represents the values ​​of each dimension of the abnormal behavior pattern feature data and the values ​​of the real-time trust score data of the account, ensuring the temporal correspondence between the abnormal behavior patterns and the real-time trust score data.

[0029] Calculate the correlation coefficient between the behavioral anomaly pattern vector at each time step in the joint analysis matrix and the real-time trust score of the corresponding account to obtain the correlation coefficient sequence data; Correlation coefficients typically range from -1 to 1. The closer the absolute value of the correlation coefficient is to 1, the stronger the linear correlation between the two data variables; a correlation coefficient close to 0 indicates no linear correlation between the two variables. Since the behavioral anomaly pattern vector is a multi-dimensional feature data vector, while the real-time account trust score data is single-dimensional numerical data, it is first necessary to perform dimensionality reduction or feature fusion calculations on each behavioral anomaly pattern feature vector to convert it into a single-dimensional numerical representation. For example, dimensionality reduction can be achieved by calculating the average value of the feature vector, the maximum value of the feature vector's dimensionality, or using principal component analysis. Then, the correlation coefficient is calculated with the corresponding real-time account trust score data using methods such as the Pearson correlation coefficient to assess the linear correlation. If a feature vector representing an abnormal behavior pattern at a certain time step, after feature fusion, yields a single-dimensional value of 0.8, and the corresponding real-time trust score for the account is 0.9, then a high positive correlation coefficient is calculated using the Pearson correlation coefficient formula, indicating a positive correlation. Conversely, if another time step represents an abnormal behavior pattern with a feature value of 0.8 and a real-time trust score for the account is 0.1, then a negative correlation coefficient is calculated, indicating a negative correlation. The correlation coefficients for all time steps in the joint analysis matrix are calculated and arranged sequentially to form a complete correlation coefficient sequence, reflecting the changing trend of the linear correlation between the abnormal behavior pattern feature data and the real-time trust score data.

[0030] The correlation coefficient sequence data is compared with the potential threat determination threshold, and the time step index where the correlation coefficient is greater than the potential threat determination threshold is marked to generate a potential threat index set. The potential threat assessment threshold is a fixed numerical boundary set during the initial deployment of a network security defense system to determine the potential threat status of user accounts. It is typically set between 0.7 and 0.9 to clearly define the severity of the relationship between strongly correlated abnormal behavioral patterns and real-time trust scores. When the correlation coefficient at a specific time step is greater than the threshold, it indicates a clear linear correlation between the abnormal behavioral pattern data and the account's real-time trust score data, reflecting a potential network threat risk to the user account at that time step. For example, when the threshold is set to 0.8, if the correlation coefficient at a certain time step is calculated to be 0.85, that time step is marked as a potential threat index and added to the potential threat index set; if the correlation coefficient at another time step is 0.6, it is not marked and is not added to the set. By sequentially comparing and marking all correlation coefficient sequences, an index set is formed that clearly identifies all time steps with potential threat risks.

[0031] Based on the potential threat index set, extract the behavioral anomaly pattern vector and the real-time trust score of the account corresponding to each time step index from the joint analysis matrix to construct the potential threat feature data of the user account. The potential threat index set records each time step where a potential threat risk is identified. Based on the index of each time step in the set, the behavioral anomaly pattern feature data and the corresponding real-time trust score data of the account for the corresponding index time step are extracted from the joint analysis matrix to form a separate data combination unit. Then, all data combination units are integrated into a complete data set, namely the user account potential threat feature data set.

[0032] S5: Based on user account potential threat characteristics data, utilize the policy evaluation network to assess the effectiveness of defense strategies and output defense strategy evaluation data, including: The potential threat characteristics data of user accounts are input into the policy evaluation network; the policy evaluation network consists of an input layer, a hidden layer, and an output layer. The input layer receives potential threat feature data of user accounts, the hidden layer uses a fully connected structure to perform feature transformation and nonlinear mapping on the potential threat feature data of user accounts, and the output layer performs linear mapping on the nonlinear mapping result output by the hidden layer to generate the initial evaluation value of the defense strategy. The input layer receives potential threat feature data of user accounts as network input data and transmits it to the hidden layer. The hidden layer performs multi-step feature transformation calculations on the potential threat feature data of user accounts transmitted from the input layer through a fully connected network structure. The transformation calculations typically consist of linear weighted summation and nonlinear activation function calculations. Each hidden layer node first performs a linear weighted summation operation, that is, multiplying the data value from each node in the input layer by the connection weight coefficient and accumulating them to obtain a preliminary linear combination value. Then, a nonlinear activation function operation is performed on the linear combination value, such as using common nonlinear activation functions like ReLU (Modified Linear Unit), tanh (hyperbolic tangent function), or sigmoid (logistic function) to effectively mine the nonlinear intrinsic relationships in the potential threat feature data of user accounts. Through the linear and nonlinear calculations of the hidden layer, each hidden layer node obtains a high-level feature representation of the input data features for use in defense strategy evaluation calculations. If there is a multi-layer structure between hidden layers, the data vector output by each hidden layer continues to be used as the input of the next hidden layer, repeating the above process, undergoing multiple linear weighted and nonlinear transformations to obtain a more expressive feature vector. Finally, the feature vector output from the hidden layer is directly input into the output layer of the policy evaluation network. The output layer is responsible for performing linear mapping calculations on the feature vectors obtained from the hidden layer to output the initial evaluation value of the defense policy. Each node in the output layer performs a linear combination calculation on the feature data values ​​output by each node in the hidden layer, that is, multiplying the data value of each hidden layer node by the corresponding connection weight of the output layer and then summing them. The calculated value is the initial evaluation value of the defense policy. The initial evaluation value of the defense policy is used to initially express the overall effectiveness of the current network security defense policy. For example, the range of the initial evaluation value of the defense policy may be large, ranging from negative infinity to positive infinity. A positive value indicates that the current defense policy has a positive overall defense effect, and the larger the value, the more obvious the effect; a negative value indicates that the defense policy may have a certain degree of failure or insufficient effect, and the smaller the value, the worse or less ideal the effect.

[0033] The initial evaluation values ​​of the defense strategy are normalized to generate defense strategy evaluation data; Since the initial evaluation values ​​of defense strategies have a wide range of values ​​and may be arbitrary real numbers, they need to undergo a standardization process—or normalization—to facilitate analysis and strategy adjustment by the network security defense decision-making system. Normalization involves using linear or nonlinear mathematical methods to limit the initial evaluation values ​​of the defense strategy to a specific, unified numerical range, such as 0 to 1. This normalization process creates unified defense strategy evaluation data, expressing the effectiveness level of the current network security defense strategy under different potential threat states of user accounts. This allows the network security defense decision-making system to optimize and adjust defense strategies in real time based on the evaluation data.

[0034] S6: Based on user account potential threat characteristics data and defense strategy evaluation data, adjust defense strategies and generate optimal network defense decisions through the policy decision network, including: By concatenating user account potential threat characteristics data with defense strategy evaluation data, a strategy decision input vector is constructed. The potential threat characteristics data of user accounts and the evaluation data of defense strategies are synchronized and matched according to their generation timestamps, and then the characteristic data is concatenated. For example, assuming that the potential threat characteristics data of a user account at a certain time step includes 0.7 abnormal login attempts and 0.6 abnormal access frequencies to sensitive resources, and the corresponding defense strategy evaluation data is 0.8, then after horizontal concatenation, the policy decision input vector for that time step is [0.7, 0.6, 0.8]. If the potential threat characteristics data of the user account at the next time step are 0.4 abnormal login attempts, 0.5 abnormal access frequencies to sensitive resources, and 0.3 defense strategy evaluation data, then the concatenated policy decision input vector is [0.4, 0.5, 0.3]. And so on, data concatenation is performed at each time step to form a sequence of policy decision input vectors. The sequence of policy decision input vectors can express a unified combination of features of the current potential threat status of user accounts in the network environment and the effectiveness of the current defense strategy, ensuring that network security defense decisions can simultaneously consider the overall information of threat status and strategy performance.

[0035] The policy decision network consists of a decision input layer, a policy hidden layer, and a decision output layer; The decision input layer receives the policy decision input vector, the policy hidden layer performs feature transformation on the policy decision input vector through a fully connected structure and activation function and outputs the policy hidden layer feature vector, and the decision output layer linearly maps the policy hidden layer feature vector and outputs a set of defense decision candidate policies. The policy decision network is a neural network structure designed for network security defense decision-making. It performs data analysis and policy generation based on the policy decision input vector. The decision input layer is responsible for receiving the policy decision input vector and transmitting the data to the policy hidden layer. The policy hidden layer consists of a fully connected structure, where each node in the layer has connection weights with nodes in adjacent layers. It uses activation functions to perform feature transformation to extract high-level feature data. The decision output layer performs a linear mapping on the feature data output by the policy hidden layer to output a set of candidate defense decision policies. Each node in the decision input layer receives the specific dimension value of the policy decision input vector. Assuming the input vector dimension is 10, the decision input layer will have 10 nodes. The policy hidden layer may contain multiple hidden nodes, connected to each node in the decision input layer through individual connection weights. The policy hidden layer nodes use non-linear activation functions, such as ReLU, tanh, or sigmoid, to achieve non-linear mapping and feature extraction of the feature data. After receiving data from the decision input layer, each node in the policy hidden layer calculates the weighted sum of the input data and connection weights, and then transforms it using a non-linear activation function to obtain the policy hidden layer feature vector. The policy hidden layer feature vector expresses the deep-level characteristics of the current network security defense environment. Finally, the policy hidden layer feature vector is transmitted to the decision output layer, where each node performs a linear mapping calculation again to form a set of defense decision candidate policies. The set of defense decision candidate policies consists of several policy schemes; for example, scheme 1 might be temporarily locking a user account; scheme 2 might be adjusting user account access permissions; and scheme 3 might be resetting the account password.

[0036] The candidate strategies in the set of defense decision candidate strategies are compared with the preset defense strategy selection criteria to select the optimal network defense decision; The preset criteria for selecting the best defense strategy are the evaluation criteria for defense actions explicitly set during the initial deployment phase of the network security defense decision-making system. These criteria include, but are not limited to, the criteria for lowest execution cost, strongest threat containment capability, and lowest risk in implementing the defense action. For example, if the candidate defense strategy set includes three options: temporarily locking user accounts, modifying account access permissions, and resetting passwords, the network security defense decision-making system evaluates the implementation cost, implementation risk, and threat containment capability of the three options. If, after evaluation, the option of temporarily locking user accounts has the lowest implementation cost and the strongest threat containment capability, then the option of temporarily locking user accounts is selected as the optimal network defense decision.

[0037] Adjust strategies based on optimal network defense decisions; The selected optimal network defense decision is transmitted to the network defense execution device. Based on this decision, the device automatically executes corresponding network defense actions. For example, if a user account is temporarily locked, the device immediately sets the account status to temporarily locked within the network environment to prevent potential network threats. If the user chooses to modify access permissions, the device immediately adjusts the access permission level to limit potential abnormal operations. If the user chooses to reset passwords, the device resets the password and requires re-authentication. Through these adjustments, the network security system achieves precise prevention and effective control of potential network threats to user accounts, ultimately ensuring the overall security of the network environment.

[0038] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0039] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0040] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0041] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0042] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0043] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0044] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0045] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0046] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0047] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A network security defense decision-making method, characterized in that, Includes the following steps: S1: Collect user account access behavior data and intrusion observation data output by intrusion detectors in the network environment, and form standardized historical trust evidence sequence data through data cleaning and feature extraction; S2: Based on historical trust evidence sequence data, recurrent neural networks are used to analyze the temporal change characteristics of account access behavior, identify abnormal user behavior patterns, and output abnormal behavior pattern feature data. S3: Based on historical trust evidence sequence data, use the actor-critic reinforcement learning algorithm to perform trust assessment on intrusion observation data and generate real-time trust score data for accounts; S4: Perform correlation analysis on abnormal behavior pattern feature data and real-time account trust score data to evaluate the correlation between abnormal behavior patterns and trust scores, and generate potential threat feature data for user accounts. S5: Based on the potential threat characteristics data of user accounts, use the policy evaluation network to evaluate the effectiveness of the defense strategy and output the defense strategy evaluation data; S6: Adjust the defense strategy based on the user account potential threat characteristics data and defense strategy evaluation data, and generate the optimal network defense decision through the policy decision network.

2. The network security defense decision-making method according to claim 1, characterized in that, S1, specifically: Collect user account access behavior data and intrusion observation data output by intrusion detectors in the network environment; Data cleaning is performed on user account access behavior data and intrusion observation data; Extract access behavior feature data that characterizes the patterns of user account behavior and intrusion observation data that characterize the patterns of intrusion observation from user account access behavior data and intrusion observation data after data cleaning. Access behavior characteristic data and intrusion characteristic data are sorted and merged according to the time of occurrence to form a standardized historical trust evidence sequence data.

3. The network security defense decision-making method according to claim 2, characterized in that, S2, specifically: After normalizing the standardized historical trust evidence sequence data, a time series matrix is ​​constructed according to time steps; The time series matrix is ​​sequentially input into a recurrent neural network containing an input layer, a recurrent unit layer, and a fully connected output layer. The recurrent unit layer extracts the temporal variation feature vector of account access behavior through recursive calculation. The difference between the temporal change feature vector of account access behavior and the preset baseline feature vector of normal access behavior is calculated to obtain behavior difference data. The behavioral difference data is compared with the anomaly detection threshold to determine the abnormal behavior index; Generate behavioral anomaly pattern feature data based on the abnormal behavior index.

4. The network security defense decision-making method according to claim 3, characterized in that, S3, specifically: Historical trust evidence sequence data and intrusion observation data are synchronously matched and concatenated into a comprehensive state vector according to the occurrence time. The comprehensive state vector is input into the actor-critic reinforcement learning algorithm framework, which consists of an actor network and a critic network; the actor network outputs a defensive action vector, and the critic network calculates action value assessment data based on the comprehensive state vector and the defensive action vector; The real-time trust score data of the account is obtained by linearly normalizing the action value assessment data.

5. The network security defense decision-making method according to claim 4, characterized in that, S4, specifically: The abnormal behavior pattern feature data and the real-time trust score data of accounts are aligned by time step and concatenated into a joint analysis matrix; Calculate the correlation coefficient between the behavioral anomaly pattern vector at each time step in the joint analysis matrix and the real-time trust score of the corresponding account to obtain the correlation coefficient sequence data; The correlation coefficient sequence data is compared with the potential threat determination threshold, and the time step index where the correlation coefficient is greater than the potential threat determination threshold is marked to generate a potential threat index set. Based on the potential threat index set, extract the behavioral anomaly pattern vector and the real-time trust score of the account corresponding to each time step index from the joint analysis matrix to construct the potential threat feature data of user accounts.

6. The network security defense decision-making method according to claim 5, characterized in that, S5, specifically: The potential threat characteristics data of user accounts are input into the policy evaluation network; the policy evaluation network consists of an input layer, a hidden layer, and an output layer. The input layer receives potential threat feature data of user accounts, the hidden layer uses a fully connected structure to perform feature transformation and nonlinear mapping on the potential threat feature data of user accounts, and the output layer performs linear mapping on the nonlinear mapping result output by the hidden layer to generate the initial evaluation value of the defense strategy. The initial evaluation values ​​of the defense strategy are normalized to generate defense strategy evaluation data.

7. A network security defense decision-making method according to claim 6, characterized in that, S6, specifically: By concatenating user account potential threat characteristics data with defense strategy evaluation data, a strategy decision input vector is constructed. The policy decision network consists of a decision input layer, a policy hidden layer, and a decision output layer; The decision input layer receives the policy decision input vector, the policy hidden layer performs feature transformation on the policy decision input vector through a fully connected structure and activation function and outputs the policy hidden layer feature vector, and the decision output layer linearly maps the policy hidden layer feature vector and outputs a set of defense decision candidate policies. The candidate strategies in the set of defense decision candidate strategies are compared with the preset defense strategy selection criteria to select the optimal network defense decision; Adjust strategies based on optimal network defense decisions.

Images