Big data-based risk management method, device, equipment, medium and product

Abstract

The application discloses a risk management method and device based on big data, equipment, medium and product, according to the communication log of the communication network, the key data information is extracted, the time sequence change and periodic characteristic analysis are carried out to the key data information, and the key variable set is generated, according to the dependent relationship between the key variables in the key variable set and the communication protocol of the communication network, the causal chain of communication data transmission is analyzed, and the causal relationship network is generated, wherein the causal relationship network includes key variables, dependent edges and causal chains, according to the causal relationship network, the key variables with abnormal behaviors in the communication network are detected as key risk nodes, the risk propagation probability and risk propagation path of the key risk nodes are evaluated, and the risk diffusion result is generated. By using the application, the timeliness and accuracy of identifying the communication network risk are improved through periodic characteristic capture and causal chain analysis.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and in particular to a risk management method, apparatus, device, medium, and product based on big data. Background Technology

[0002] The field of communication risk management technology focuses on identifying, assessing, monitoring, and mitigating various risks that may occur in communication systems, including cybersecurity risks, data breach risks, service interruptions, and network intrusions. With the development of communication technologies, such as the widespread application of 5G, cloud computing, and the Internet of Things, ensuring the security, stability, and reliability of communication systems has become particularly important.

[0003] Current big data risk management methods typically utilize advanced analytical tools, such as machine learning and artificial intelligence, for risk prediction and the implementation of defensive measures. However, existing methods often lack real-time data processing and precise data analysis capabilities, resulting in insufficient timeliness and accuracy. This is particularly true when dealing with rapidly changing communication technologies, such as 5G and the Internet of Things, where existing technologies often fail to reflect the dynamic changes in communication networks in a timely manner, increasing the challenges of risk management. Summary of the Invention

[0004] The purpose of this invention is to provide a risk management method, device, equipment, medium, and product based on big data, which improves the timeliness and accuracy of identifying communication network risks through periodic feature capture and causal chain analysis.

[0005] To achieve the above objectives, embodiments of the present invention provide a risk management method based on big data, comprising:

[0006] Based on the communication logs of the communication network, extract key data information; wherein, the key data information includes timestamp, source address, destination address, and data packet size;

[0007] The key data information is analyzed for time series changes and periodic characteristics to generate a set of key variables; wherein, the set of key variables includes several key variables;

[0008] Based on the dependencies between key variables in the set of key variables and the communication protocol of the communication network, the causal chain of communication data transmission is analyzed to generate a causal relationship network; wherein, the causal relationship network includes key variables, dependency edges, and causal chains;

[0009] Based on the causal relationship network, key variables exhibiting abnormal behavior in the communication network are detected as key risk nodes.

[0010] Assess the risk propagation probability and risk propagation path of the key risk nodes, and generate risk diffusion results.

[0011] As an improvement to the above solution, the step of performing time series variation and periodic feature analysis on the key data information to generate a set of key variables includes:

[0012] Calculate the frequency of occurrence of each of the timestamps;

[0013] Each frequency is sorted, and the difference between the timestamps corresponding to the maximum and minimum values ​​in the frequency is calculated as the peak-valley interval to analyze the time distribution pattern between peaks and valleys.

[0014] An address pair is formed based on the source address and the destination address, and an address pair-timestamp matrix is ​​constructed based on the correspondence between the address pair and the timestamp and the traffic of the address pair at the corresponding timestamp.

[0015] Based on the time distribution pattern, the address-timestamp matrix is ​​divided into peak period partitions and off-peak period partitions. Based on the traffic accumulation of the peak period partitions and the off-peak period partitions, the critical path of network traffic is mapped to generate a traffic path map.

[0016] Based on the traffic path map, extract the data packet size for peak and off-peak periods respectively;

[0017] The differences in data packet size between the peak and off-peak periods are calculated separately to analyze the periodicity of the data stream and construct a periodicity model of the data stream.

[0018] A set of key variables is generated based on the time distribution pattern, the traffic path map, and the periodic feature model.

[0019] As an improvement to the above scheme, the step of analyzing the causal chain of communication data transmission and generating a causal relationship network based on the dependencies between key variables in the key variable set and the communication protocol of the communication network includes:

[0020] Calculate the correlation coefficients among the key variables in the set of key variables to generate a dependency matrix;

[0021] Based on the dependency matrix, using the key variables as nodes and the correlation coefficients as the weights of the dependency edges, a preliminary network graph is generated.

[0022] Based on the communication protocol of the communication network, the transmission logic of the communication data is analyzed to obtain the causal chain of communication data transmission;

[0023] The initial network graph is adjusted based on the causal chain to generate a causal relationship network.

[0024] As an improvement to the above scheme, the step of generating a preliminary network graph based on the dependency matrix, using the key variables as nodes and the correlation coefficients as weights of the dependency edges, includes:

[0025] Based on the dependency matrix, the key variables are used as nodes, and the correlation coefficients are used as the weights of the dependency edges;

[0026] Based on the physical distance between nodes and the connection quality index, the weights of the dependent edges are adjusted using the minimum spanning tree algorithm to generate a preliminary network graph.

[0027] As an improvement to the above scheme, the step of detecting key variables exhibiting abnormal behavior in the communication network based on the causal relationship network, as key risk nodes, includes:

[0028] Based on the causal relationship network, the betweenness centrality of the nodes is calculated, and nodes with betweenness centrality higher than a first preset threshold are designated as high centrality nodes.

[0029] Based on the mean and standard deviation of the historical data of the nodes, calculate the abnormal behavior score of the highly central nodes, and identify nodes whose abnormal behavior scores are greater than a second preset threshold as abnormal nodes.

[0030] Based on the interaction pattern between the abnormal node and its neighboring nodes, the importance score of the abnormal node is calculated, and nodes with an importance score greater than a third preset threshold are designated as critical risk nodes.

[0031] As an improvement to the above scheme, the assessment of the risk propagation probability and risk propagation path of the key risk nodes, and the generation of risk diffusion results, includes:

[0032] Assess the risk weight of each of the aforementioned key risk nodes, and construct a linear regression model with the key risk nodes as independent variables and the quantitative indicators of network stability as dependent variables.

[0033] By fitting the linear regression model, an influence score is generated for each of the key risk nodes.

[0034] Based on the influence score, the interaction between the key risk nodes is simulated to test the risk propagation probability and generate the risk propagation path.

[0035] Based on the risk propagation probability and the risk propagation path, a risk diffusion result is generated.

[0036] This invention also provides a risk management device based on big data, comprising:

[0037] The data information acquisition module is used to extract key data information based on the communication logs of the communication network; wherein, the key data information includes timestamp, source address, destination address, and data packet size;

[0038] A key variable set generation module is used to perform time series change and periodic feature analysis on the key data information to generate a key variable set; wherein, the key variable set includes several key variables;

[0039] The causal relationship network generation module is used to analyze the causal chain of communication data transmission and generate a causal relationship network based on the dependency relationships between key variables in the key variable set and the communication protocol of the communication network; wherein, the causal relationship network includes key variables, dependency edges and causal chains;

[0040] The critical risk node detection module is used to detect key variables with abnormal behavior in the communication network based on the causal relationship network, and to identify them as critical risk nodes.

[0041] The risk diffusion result generation module is used to assess the risk propagation probability and risk propagation path of the key risk nodes and generate risk diffusion results.

[0042] This invention also provides a risk management device based on big data, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the risk management method based on big data as described in any of the above embodiments.

[0043] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the risk management method based on big data as described in any of the above embodiments.

[0044] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the risk management method based on big data as described above.

[0045] Compared with existing technologies, the risk management method, apparatus, equipment, medium, and product based on big data disclosed in this invention extracts communication logs from communication networks, performs time-series change and periodic feature analysis on key data information to extract key variables, and conducts causal chain analysis based on the dependencies between key variables and communication protocols to obtain a causal relationship network. Based on this causal relationship network, key risk nodes are identified, and the probability and path of risk propagation are assessed to generate risk diffusion results, achieving accurate identification and prediction of communication network risks. Compared with related technologies, the embodiments of this invention improve the timeliness and accuracy of risk identification by capturing periodic features and performing causal chain analysis on communication network data, which is beneficial to improving the security and stability of communication systems. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a risk management method based on big data provided in an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the structure of a risk management device based on big data provided in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of a risk management device based on big data provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0051] See Figure 1 This is a flowchart illustrating a risk management method based on big data provided in an embodiment of the present invention. The embodiment of the present invention provides a risk management method based on big data, including steps S11 to S15:

[0052] S11. Extract key data information from the communication logs of the communication network; wherein, the key data information includes timestamp, source address, destination address, and data packet size;

[0053] S12. Perform time series change and periodic feature analysis on the key data information to generate a set of key variables; wherein, the set of key variables includes several key variables;

[0054] S13. Based on the dependencies between key variables in the set of key variables and the communication protocol of the communication network, analyze the causal chain of communication data transmission and generate a causal relationship network; wherein, the causal relationship network includes key variables, dependency edges, and causal chains;

[0055] S14. Based on the causal relationship network, detect key variables in the communication network that exhibit abnormal behavior, and identify them as key risk nodes.

[0056] S15. Assess the risk propagation probability and risk propagation path of the key risk nodes, and generate risk diffusion results.

[0057] In this embodiment of the invention, communication logs of a communication network are obtained, and key data information related to big data risk assessment is extracted from the communication logs, including timestamps, addresses, and data packet sizes. For example, suppose that in a certain communication network, records for a continuous 24 hours are obtained from the communication logs, including: multiple time points between 08:00:00 on June 1, 2024 and 08:00:00 on June 2, 2024, such as 08:00:00 on June 1, 2024, 08:10:00 on June 1, 2024; source address, such as user terminal IP 192.168.1.101; destination address, such as server IP 203.0.113.5; and data packet size, such as 512KB.

[0058] By analyzing the time-series changes in the key data information and identifying the periodic characteristics of the data flow, multiple key variables are integrated to form a key variable set. These key variables refer to variables related to time, address, data packet size, and / or network traffic obtained after analyzing the time-series changes and periodic characteristics of the key data information. For example, these key variables include peak-hour data packet size, off-peak-hour data packet size, user terminal-to-server path traffic, and inter-server path traffic.

[0059] Furthermore, the dependencies between the key variables in the set of key variables are analyzed, and the variables are connected to form a network graph including nodes and dependent edges. Combined with the communication protocol analysis of the communication network, the causal chains in data transmission are determined, and a causal relationship network is established.

[0060] Finally, through the causal relationship network, key nodes and dependency edges are identified. Mutation analysis is used to detect anomalous behavior of key nodes in the network, identifying key risk nodes and generating a list of key risk nodes. Based on this list, the influence and risk propagation probability among multiple key risk nodes are calculated. By simulating risk propagation paths, the risk impact is assessed, thereby generating a risk diffusion prediction model and obtaining the risk diffusion results. These risk diffusion results characterize the big data risk situation of the communication network, facilitating timely risk assessment and management by users.

[0061] By employing the technical means of this invention, communication logs from a communication network are extracted, and time-series changes and periodic characteristics of key data information are analyzed to extract key variables. Based on the dependencies between key variables and in conjunction with communication protocols, causal chain analysis is performed to obtain a causal relationship network. Based on this causal relationship network, key risk nodes are identified, and the probability and path of risk propagation are assessed, generating risk diffusion results. This achieves accurate identification and prediction of communication network risks. Compared to related technologies, this invention, through periodic feature capture and causal chain analysis of communication network data, improves the timeliness and accuracy of risk identification, which is beneficial to improving the security and stability of the communication system.

[0062] As a preferred embodiment, the present invention further implements the above embodiments, wherein step S12, namely, performing time series change and periodic feature analysis on the key data information to generate a key variable set, includes steps S121 to S127:

[0063] S121. Calculate the frequency of occurrence of each timestamp;

[0064] S122. Sort each frequency and calculate the difference between the timestamps corresponding to the maximum and minimum values ​​in the frequency, as the peak-valley interval, so as to analyze the time distribution pattern between peaks and valleys.

[0065] In this embodiment of the invention, timestamps are extracted from the communication logs, the frequency of timestamp occurrences is counted, and the frequencies are sorted to identify peak and trough periods and analyze the time distribution pattern between peaks and troughs.

[0066] Preferably, calculating the frequency of each timestamp includes:

[0067] Count the frequency of each timestamp.

[0068] By introducing a time decay factor and a weighting coefficient, the frequency is adjusted to obtain the adjusted frequency for each timestamp.

[0069] Specifically, all timestamps of the records are extracted from the communication logs, for example, each 10 minutes is a record point, forming a timestamp array T = {t1, t2, ..., t...}. n The frequency of occurrence of the timestamps is counted to form a frequency array, where each element corresponds to the number of times a specific time point occurs. The frequency array is F = {f1, f2, ..., f...} n The higher the frequency, the more concentrated the traffic during that time period.

[0070] To dynamically reflect the time sensitivity of the data, a time decay factor and weighting coefficient are introduced to adjust the frequency, increasing the weight of recent data and improving the sensitivity and adaptability of the data analysis, resulting in the adjusted frequency:

[0071]

[0072] Where, α i For timestamp t i The weighting coefficient, λ is the time decay coefficient, t current At the current time, f i For timestamp t i The original frequency.

[0073] Next, the frequency array F ′ Sort the frequencies to determine a descending order, and determine the timestamps t corresponding to the maximum and minimum values ​​among the frequencies. peak and t valley The difference Δt between the two timestamps is calculated as the peak-valley interval, i.e., Δt = t peak -t valley This is used to analyze the temporal distribution patterns between peaks and valleys. For example, in the analysis of F... ′ After sorting, it was found that 8:00-10:00 and 18:00-20:00 were the frequency peaks (peak periods), and 0:00-6:00 was the frequency trough (low period). The peak-to-trough interval Δt = 8 hours (such as the time difference between peak and trough).

[0074] S123. Form an address pair based on the source address and the destination address, and construct an address pair-timestamp matrix based on the correspondence between the address pair and the timestamp and the traffic of the address pair at the corresponding timestamp.

[0075] S124. Based on the time distribution pattern, the address-timestamp matrix is ​​divided into peak period partitions and off-peak period partitions. Based on the traffic accumulation of the peak period partitions and the off-peak period partitions, the critical path of network traffic is mapped to generate a traffic path map.

[0076] Specifically, a time-distributed model is adopted to collect and analyze source and destination addresses, pairing them into address pairs, and constructing an address-time matrix A based on the correspondence between timestamps and addresses. tm In this matrix, rows represent address pairs, columns represent time, and element a ij This represents the traffic situation of the i-th address pair at the j-th time point.

[0077] Based on the peak-valley interval Δt, the timestamp is divided into peak periods and valley periods, and matrix A is... tm The network is partitioned into peak-hour and off-hour sections. By analyzing the cumulative traffic within each section, the critical paths of network traffic are mapped. For example, during peak hours, the main path is "user terminal → core server," such as 192.168.1.xxx → 203.0.113.5, accounting for 70% of the traffic. During off-hours, the main path is "inter-server synchronization," such as 203.0.113.5 → 198.51.100.8, accounting for 60% of the traffic. The critical paths and their corresponding traffic percentages are visualized in a traffic path map.

[0078] S125. Based on the traffic path map, extract the data packet size for peak and off-peak periods respectively;

[0079] S126. Calculate the difference in data packet size between the peak period and the off-peak period respectively, analyze the periodic characteristics of the data stream, and construct a periodic characteristic model of the data stream.

[0080] S127. Generate a set of key variables based on the time distribution pattern, the traffic path map, and the periodic feature model.

[0081] In this embodiment of the invention, by using a traffic path map, the distribution of data packets in different time periods is compared based on the size of data packets in multiple time periods to identify periodic changes. Based on these periodic changes, a periodic feature analysis model of the data flow is constructed, and a set of key variables is output.

[0082] Specifically, the data packet sizes during peak and off-peak hours are statistically analyzed to form a set S = {s1, s2, ..., s}. m The data packet sizes at different time intervals are compared, and the comparison function is set as C(x,y)=|s x -s y By calculating the differences in data packet size across different time periods, the periodic changes in the data stream can be identified.

[0083] For example, by comparing the differences in data packet size, it can be found that the difference C1 = 50KB within the same peak period, with relatively small fluctuations, while the difference C2 = 550KB between the peak and off-peak periods, with relatively large fluctuations, indicates that there is a periodicity.

[0084] Based on the identified periodicity, a periodic feature analysis model for the data stream is constructed, which uses the statistical characteristics of periodic changes:

[0085] P = {p1, p2, ..., p} k};

[0086] Where, p i This represents the i-th periodic feature.

[0087] As an example, based on difference analysis, the periodic characteristics of "peak hours from 8:00-10:00 and 18:00-20:00 daily, with an average data packet size of 850KB" and "low hours from 0:00-6:00, with an average data packet size of 280KB" are identified, forming a feature set P = {p1, p2}, where p1 = peak period and average value, and p2 = low period and average value.

[0088] Finally, the time distribution pattern, the traffic path map, and the periodic feature model are integrated to generate several key variables, forming a key variable set. For example, these key variables include peak-hour data packet size, off-peak-hour data packet size, user terminal-to-server path traffic, and inter-server path traffic, etc.

[0089] By employing the technical means of this invention, dynamic calculation of timestamp frequency enhances the sensitivity to recent data, solving the problem of lagging real-time data processing in existing technologies. By mapping addresses to timestamps, traffic peaks and troughs are accurately segmented, facilitating the generation of structured traffic path data. By statistically analyzing data packet sizes across multiple time periods through traffic path mapping, calculating time-period differences, and identifying periodicity, the deficiencies of existing technologies in capturing dynamic changes in data streams are overcome.

[0090] As a preferred embodiment, the present invention further implements the above embodiments. Step S13, namely, analyzing the causal chain of communication data transmission and generating a causal relationship network based on the dependency relationships between key variables in the key variable set and the communication protocol of the communication network, includes steps S131 to S134:

[0091] S131. Calculate the correlation coefficients among the key variables in the set of key variables and generate a dependency matrix;

[0092] In this embodiment of the invention, based on a set of key variables, the interactions and dependencies among multiple variables are analyzed. By calculating the correlation coefficients between variables, the direct influence relationships between variables are determined, and a dependency matrix is ​​generated.

[0093] Specifically, the first step is to analyze the interactions and dependencies among the key variables. Let the variable set be V = {v1, v2, ..., v...}. n} Calculate the correlation coefficients between key variables:

[0094]

[0095] in, and These are the key variables v i and v j The average value.

[0096] This formula allows us to determine the direct influence relationships between variables. Based on the correlation coefficients between the key variables, a dependency matrix M is generated, where each element m... ij Represents variable v i For variable v j The degree of influence can be directly represented by the correlation coefficient C.

[0097] S132. Based on the dependency matrix, using the key variables as nodes and the correlation coefficients as the weights of the dependency edges, generate a preliminary network graph.

[0098] Preferably, step S132 specifically includes:

[0099] Based on the dependency matrix, the key variables are used as nodes, and the correlation coefficients are used as the weights of the dependency edges;

[0100] Based on the physical distance between nodes and the connection quality index, the weights of the dependent edges are adjusted using the minimum spanning tree algorithm to generate a preliminary network graph.

[0101] Specifically, through a dependency matrix, using the key variables as nodes and the correlation coefficients as edge weights, the physical distance d between nodes is introduced. ij and connection quality index c ij The minimum spanning tree algorithm is used to optimize node connection logic and generate a preliminary network graph. The connection quality metrics include packet loss rate, among others.

[0102] The weights are adjusted using the minimum spanning tree algorithm, specifically as follows:

[0103]

[0104] Among them, W ′ w represents the total weight of the minimum spanning tree.ij Let λ be the original weight of edge (i,j), i.e., the correlation coefficient between nodes, λ be the edge weight adjustment coefficient, α be the distance penalty coefficient, and d be the distance between nodes. ij Let be the physical distance between nodes i and j, β be the connection quality reward coefficient, and c be the physical distance between nodes i and j. ij This is the connection quality metric between nodes i and j.

[0105] This invention introduces additional parameters such as λ, α, β, and d. ij c ij To adjust the edge weights, first calculate the physical distance d between each pair of nodes. ij and connection quality c ij Then according to the new weighting formula w ij ·λ+α·d ij -β·c ij Recalculate the weight of each edge, then sort all edges in ascending order of this new weight and add them to the spanning tree step by step until the tree contains all nodes, ensuring that no cycles are formed during the process.

[0106] The exact values ​​of the weight coefficients λ, α, and β can be determined using optimization algorithms, such as genetic algorithms or simulated annealing, to ensure that the total weight W of the spanning tree is maximized. ′ Minimization. This method generates a network graph that not only reflects the strongest correlations between nodes but also takes into account physical distance and connection quality, thereby optimizing the practical applicability and efficiency of the network design.

[0107] S133. Based on the communication protocol of the communication network, analyze the transmission logic of the communication data to obtain the causal chain of the communication data transmission;

[0108] S134. Adjust the preliminary network diagram according to the causal chain to generate a causal relationship network.

[0109] In this embodiment of the invention, based on a preliminary network graph and combined with data analysis of the communication protocol, the causal relationships of key variables and events in data transmission are identified through the strength of dependencies between variables and real-time analysis of communication data. Through logical reasoning, such as using Bayesian networks, probabilistic causal relationships are simulated and inferred to establish causal chains. Based on these causal chains, the preliminary network graph is adjusted to generate a causal relationship network.

[0110] As an example, taking the TCP / IP protocol as the communication protocol, based on the TCP / IP protocol, the data transmission logic is analyzed, and the causal relationship of "increased user terminal traffic (node ​​v3) → increased data packet size during peak hours (node ​​v1) → increased server load" is found. Through Bayesian network inference, the causal chain of "user terminal request → server response → change in data packet size" is established, forming a causal relationship network containing variable nodes, dependency edges, and protocol chains.

[0111] By employing the technical means of embodiments of the present invention, considering the uncertainty and conditional dependence of interactions between variables, a causal relationship network for communication risks is constructed. This not only demonstrates the causal relationships between events but also provides a visual representation of the interactions between key variables, thereby helping to more effectively manage potential risks in communication systems.

[0112] As a preferred embodiment, the present invention further implements the above embodiments. Step S14, namely, detecting key variables with abnormal behavior in the communication network based on the causal relationship network as key risk nodes, includes steps S141 to S143:

[0113] S141. Based on the causal relationship network, calculate the betweenness centrality of the nodes, and designate the nodes with betweenness centrality higher than a first preset threshold as high centrality nodes.

[0114] S142. Calculate the abnormal behavior score of the highly central node based on the mean and standard deviation of the historical data of the node, and designate the node whose abnormal behavior score is greater than the second preset threshold as an abnormal node.

[0115] S143. Based on the interaction pattern between the abnormal node and its neighboring nodes, calculate the importance score of the abnormal node, and designate nodes whose importance score is greater than a third preset threshold as key risk nodes.

[0116] In this embodiment of the invention, based on a causal relationship network, network topology analysis is performed to calculate the centrality index of each node in the network, particularly betweenness centrality:

[0117]

[0118] Where, σ st σ is the number of all shortest paths from node s to node t. st(v) represents the number of paths passing through node v. This metric reflects the bridging role of a node in the network, i.e., the degree to which information flow in the network can be controlled through that node. Nodes with betweenness centrality higher than a first preset threshold are classified as highly central nodes. By identifying highly central nodes, calculating the traffic and connection count of these nodes, a list of critical nodes is generated. These nodes, due to their key positions in the network, become the focus of monitoring and protection.

[0119] Furthermore, based on the aforementioned list of key nodes, time series analysis is performed to monitor the behavioral changes of these highly central nodes within different time windows. A time window ΔT′ is set, and behavioral data of highly central nodes is collected within this window and compared with historical behavioral data to determine the degree of deviation, thereby identifying nodes exhibiting abnormal behavior as anomalous nodes.

[0120] This invention employs statistical testing to calculate the anomalous behavior score of highly central nodes:

[0121]

[0122] Here, X represents the current data point, and μ and σ are the mean and standard deviation of the historical data, respectively, to identify the degree to which the behavior deviates from historical norms. This method can effectively identify nodes with abnormal behavior as anomalous nodes.

[0123] For example, monitoring the data packet size of highly central nodes during peak hours reveals that the data packet size suddenly increased to 1500KB at 8:30 on a certain day, while the historical average was 850KB. The abnormal behavior score Z = (1500-850) / 100 = 6.5 was calculated, which far exceeded the threshold of 3, and thus it was determined to be an abnormal node.

[0124] Furthermore, by analyzing the interaction patterns between abnormal nodes and other nodes through graph mining, the potential threats of these interactions to network security are assessed, nodes that trigger chain reactions are identified, and a list of critical risk nodes is generated.

[0125] The formula for the graph mining algorithm is used to characterize the importance score of nodes:

[0126]

[0127] Where v is a node in the graph, N(v) is the set of neighboring nodes of v, d(u) is the degree of node u, w(u,v) is the weight of the edge between nodes u and v, e(u,v) is the attribute strength of the edge between nodes u and v, c(u,v) is the connectivity score between nodes, and β is a new normalization constant. The value of β is determined by the ratio of the average connectivity of the node set to the average edge attribute strength.

[0128] Specifically, a specific node v is selected, and all its neighboring nodes N(v) are identified. Each edge connecting v and its neighbor u is assigned a weight w(u,v). The degree d(u) of each neighboring node u, the attribute strength e(u,v) of the edges between nodes, and the connectivity score c(u,v) are considered. These new parameters enhance the accuracy of node importance calculation by adding additional metric dimensions. Finally, a new normalization constant β is used to adjust the scale of all calculated score values, ensuring score standardization. Nodes with importance scores higher than a third preset threshold are designated as critical risk nodes, generating a list of critical risk nodes.

[0129] By employing the technical means of this invention, highly central nodes are identified through betweenness centrality analysis. Then, nodes with abnormal behavior are identified by monitoring the behavioral deviations of nodes in differentiated time windows. Finally, based on the interaction between abnormal nodes and their neighboring nodes, key risk nodes that may trigger chain reactions are accurately identified, providing precise targets for subsequent risk diffusion prediction and improving the pertinence of countermeasures.

[0130] As a preferred embodiment, the present invention further implements the above embodiments, wherein step S15, namely, assessing the risk propagation probability and risk propagation path of the key risk node and generating risk diffusion results, includes steps S151 to S154:

[0131] S151. Evaluate the risk weight of each of the key risk nodes, and construct a linear regression model with the key risk nodes as independent variables and the quantitative indicators of network stability as dependent variables.

[0132] S152. By fitting the linear regression model, generate an influence score for each of the key risk nodes;

[0133] In this embodiment of the invention, statistical analysis is performed based on a list of key risk nodes to assess the risk weight of each node, and a linear regression model is used to calculate the potential influence of each factor on network stability to generate an influence score.

[0134] Specifically, let the key risk nodes be R = {r1, r2, ..., r}. n}, each node r i Assigned a weight w i To calculate the potential impact of each node on network stability, a linear regression model is constructed, using the quantitative index of network stability as the dependent variable Y and the key risk nodes as the independent variable X:

[0135]

[0136] Where Y represents a quantitative indicator of network stability, and β iThese are regression coefficients, representing the node r i The influence is denoted by ∈, which represents the error term. By fitting this model, an influence score can be generated for each key risk node, where the influence score s = β. i / β, where β is the normalized value, β = 1. These scores reflect the importance of each node in affecting network stability.

[0137] S153. Based on the influence score, simulate the interaction between the key risk nodes, test the risk propagation probability, and generate the risk propagation path;

[0138] S154. Generate risk diffusion results based on the risk propagation probability and the risk propagation path.

[0139] In this embodiment of the invention, a network model is constructed based on the influence score, and the factor-based influence score S = {s1, s2, ..., s} n This study constructs a probabilistic model to simulate the interactions between multiple key risk nodes. The Monte Carlo method is employed, which generates a large number of random samples to simulate the probability of risk propagation and generate a risk propagation path analysis. This method generates statistical analysis of risk propagation paths through an iterative process. This process includes defining the probability P(r) of each node influencing other nodes. i →r j The system iterates through multiple iterations to assess the frequency of occurrence of various risk combinations, thereby generating a risk propagation path analysis.

[0140] Furthermore, through risk propagation path analysis, discrete event simulation techniques are used to simulate the diffusion process of risk events. Each risk event is treated as an independent event, and its occurrence and propagation are simulated based on the previous risk propagation path analysis. The impact of each risk event at different points in time and how these events propagate through the network are evaluated to predict the overall impact of the spread of these risk events on the communication network. Finally, a risk diffusion prediction model is generated as the risk diffusion result. This model can be used to predict the potential impact of future risk events on network stability, providing a scientific basis and decision support for network risk management.

[0141] By employing the technical means of this invention, the weight of each risk node on network stability is evaluated through a linear regression model, thereby improving the accuracy of influence score calculation. The Monte Carlo method is used to simulate the interaction of multiple risk factors, and propagation probabilities are predicted through numerous random experiments, which improves the accuracy of predicting risk propagation paths and thus the accuracy of predicting risk diffusion outcomes. This helps users formulate preventative strategies and ensure the security and stability of communication systems.

[0142] See Figure 2This is a schematic diagram of a risk management device based on big data provided in an embodiment of the present invention. The embodiment of the present invention provides a risk management device 10 based on big data, comprising:

[0143] The data information acquisition module 11 is used to extract key data information based on the communication logs of the communication network; wherein, the key data information includes timestamp, source address, destination address and data packet size;

[0144] The key variable set generation module 12 is used to perform time series change and periodic feature analysis on the key data information to generate a key variable set; wherein, the key variable set includes several key variables;

[0145] The causal relationship network generation module 13 is used to analyze the causal chain of communication data transmission and generate a causal relationship network based on the dependency relationship between key variables in the key variable set and the communication protocol of the communication network; wherein, the causal relationship network includes key variables, dependency edges and causal chains;

[0146] The critical risk node detection module 14 is used to detect key variables with abnormal behavior in the communication network based on the causal relationship network, and to identify them as critical risk nodes.

[0147] The risk diffusion result generation module 15 is used to assess the risk propagation probability and risk propagation path of the key risk nodes and generate risk diffusion results.

[0148] It should be noted that the risk management device based on big data provided in this embodiment of the invention is used to execute all the process steps of the risk management method based on big data in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0149] See Figure 3 This is a schematic diagram of the structure of a risk management device based on big data provided in an embodiment of the present invention. The present invention also provides a risk management device 20 based on big data, including a processor 21, a memory 22, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the risk management method based on big data as described in any of the above embodiments.

[0150] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the big data-based risk management method as described in any of the above embodiments.

[0151] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the big data-based risk management method as described in any of the above embodiments.

[0152] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0153] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A risk management method based on big data, characterized in that, include: Based on the communication logs of the communication network, extract key data information; wherein, the key data information includes timestamp, source address, destination address, and data packet size; The key data information is analyzed for time series changes and periodic characteristics to generate a set of key variables; wherein, the set of key variables includes several key variables; Based on the dependencies between key variables in the set of key variables and the communication protocol of the communication network, the causal chain of communication data transmission is analyzed to generate a causal relationship network; wherein, the causal relationship network includes key variables, dependency edges, and causal chains; Based on the causal relationship network, key variables exhibiting abnormal behavior in the communication network are detected as key risk nodes. Assess the risk propagation probability and risk propagation path of the key risk nodes, and generate risk diffusion results.

2. The risk management method based on big data as described in claim 1, characterized in that, The step of performing time series variation and periodic feature analysis on the key data information to generate a set of key variables includes: Calculate the frequency of occurrence of each of the timestamps; Each frequency is sorted, and the difference between the timestamps corresponding to the maximum and minimum values ​​in the frequency is calculated as the peak-valley interval to analyze the time distribution pattern between peaks and valleys. An address pair is formed based on the source address and the destination address, and an address pair-timestamp matrix is ​​constructed based on the correspondence between the address pair and the timestamp and the traffic of the address pair at the corresponding timestamp. Based on the time distribution pattern, the address-timestamp matrix is ​​divided into peak period partitions and off-peak period partitions. Based on the traffic accumulation of the peak period partitions and the off-peak period partitions, the critical path of network traffic is mapped to generate a traffic path map. Based on the traffic path map, extract the data packet size for peak and off-peak periods respectively; The differences in data packet size between the peak and off-peak periods are calculated separately to analyze the periodicity of the data stream and construct a periodicity model of the data stream. A set of key variables is generated based on the time distribution pattern, the traffic path map, and the periodic feature model.

3. The risk management method based on big data as described in claim 1, characterized in that, The step of analyzing the causal chain of communication data transmission and generating a causal relationship network based on the dependencies between key variables in the key variable set and the communication protocol of the communication network includes: Calculate the correlation coefficients among the key variables in the set of key variables to generate a dependency matrix; Based on the dependency matrix, using the key variables as nodes and the correlation coefficients as the weights of the dependency edges, a preliminary network graph is generated. Based on the communication protocol of the communication network, the transmission logic of the communication data is analyzed to obtain the causal chain of communication data transmission; The initial network graph is adjusted based on the causal chain to generate a causal relationship network.

4. The risk management method based on big data as described in claim 3, characterized in that, The step of generating a preliminary network graph based on the dependency matrix, using the key variables as nodes and the correlation coefficients as weights of the dependency edges, includes: Based on the dependency matrix, the key variables are used as nodes, and the correlation coefficients are used as the weights of the dependency edges; Based on the physical distance between nodes and the connection quality index, the weights of the dependent edges are adjusted using the minimum spanning tree algorithm to generate a preliminary network graph.

5. The risk management method based on big data as described in claim 1, characterized in that, The step of detecting key variables exhibiting abnormal behavior in the communication network based on the causal relationship network, as key risk nodes, includes: Based on the causal relationship network, the betweenness centrality of the nodes is calculated, and nodes with betweenness centrality higher than a first preset threshold are designated as high centrality nodes. Based on the mean and standard deviation of the historical data of the nodes, calculate the abnormal behavior score of the highly central nodes, and identify nodes whose abnormal behavior scores are greater than a second preset threshold as abnormal nodes. Based on the interaction pattern between the abnormal node and its neighboring nodes, the importance score of the abnormal node is calculated, and nodes with an importance score greater than a third preset threshold are designated as critical risk nodes.

6. The risk management method based on big data as described in claim 1, characterized in that, The assessment of the risk propagation probability and risk propagation path of the key risk nodes, generating risk diffusion results, includes: Assess the risk weight of each of the aforementioned key risk nodes, and construct a linear regression model with the key risk nodes as independent variables and the quantitative indicators of network stability as dependent variables. By fitting the linear regression model, an influence score is generated for each of the key risk nodes. Based on the influence score, the interaction between the key risk nodes is simulated to test the risk propagation probability and generate the risk propagation path. Based on the risk propagation probability and the risk propagation path, a risk diffusion result is generated.

7. A risk management device based on big data, characterized in that, include: The data information acquisition module is used to extract key data information based on the communication logs of the communication network; wherein, the key data information includes timestamp, source address, destination address, and data packet size; A key variable set generation module is used to perform time series change and periodic feature analysis on the key data information to generate a key variable set; wherein, the key variable set includes several key variables; The causal relationship network generation module is used to analyze the causal chain of communication data transmission and generate a causal relationship network based on the dependency relationships between key variables in the key variable set and the communication protocol of the communication network; wherein, the causal relationship network includes key variables, dependency edges and causal chains; The critical risk node detection module is used to detect key variables with abnormal behavior in the communication network based on the causal relationship network, and to identify them as critical risk nodes. The risk diffusion result generation module is used to assess the risk propagation probability and risk propagation path of the key risk nodes and generate risk diffusion results.

8. A risk management device based on big data, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the big data-based risk management method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the risk management method based on big data as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, implement the risk management method based on big data as described in any one of claims 1 to 6.

Images