Network information security comprehensive analysis and monitoring system and method

Abstract

The present invention relates to the field of network security technology, and more specifically, to a network information security comprehensive analysis and monitoring system, comprising: a data slicing and encryption module, a dynamic mapping and distributed storage module, a consistency check and data reconstruction module, a real-time threat monitoring and response module, and a reporting and auditing module; the data slicing and encryption module is used to slice and encrypt original data; the dynamic mapping and distributed storage module is used to dynamically adjust the storage strategy of data slices according to the network status; the consistency check and data reconstruction module is used to verify the integrity of the distributed storage slices and perform data recovery when necessary; the real-time threat monitoring and response module is used to detect security threats in the network and trigger corresponding defense mechanisms; the reporting and auditing module is used to generate system operation reports and record key events for administrators to audit and analyze.

Description

Technical Field

[0001] The present invention relates to the technical field of network security, and more specifically, to a network information security comprehensive analysis and monitoring system and method. Background Art

[0002] In recent years, with the rapid development of information technology and the increasing complexity of cyberspace, network information security issues have become increasingly prominent, becoming a major bottleneck restricting the development of the digital economy and national security. According to statistics, there were 1 billion cyber attacks worldwide in 2022, causing economic losses of more than 6 trillion US dollars. Among them, targeted attacks on critical information infrastructure have continued to escalate, and the national power grid, financial system, government affairs platform, etc. have been repeatedly infiltrated and destroyed by hacker organizations, seriously endangering the national economy and people's livelihood.

[0003] In the face of complex network security threats, traditional security protection methods are unsustainable. On the one hand, most traditional security systems use simple feature code matching or rule engines to detect attacks, such as virus databases, blacklists and whitelists, which are difficult to cope with new attacks that are constantly changing. Taking encrypted traffic as an example, it is estimated that 60% of Web traffic has been encrypted, and malware is also using encrypted communications in large quantities. The false negative rate of static feature detection is as high as more than 90%. On the other hand, traditional security systems lack an effective mechanism for gathering and sharing threat intelligence, making it difficult to form a coordinated linkage on a global scale. According to statistics, the average resolution time of current security equipment is 67 days, which is far behind the average life cycle of attack variants of 19 days, allowing a large number of threats to lurk and spread for a long time.

[0004] In a dynamic and ever-changing network environment, there is an urgent need for an advanced security protection system that can intelligently perceive, analyze in real time, and defend adaptively. However, most existing security management platforms lack the ability to process big data and analyze artificial intelligence, and are unable to effectively aggregate massive amounts of heterogeneous security data, let alone discover potential threat patterns. Taking the SIEM system as an example, it uses a simple statistical threshold model. Although it is flexible to deploy, it has a coarse analysis granularity and a high false alarm rate, making it difficult to fully characterize the security situation, let alone deal with complex APT attacks. In addition, the data storage and computing capabilities of existing platforms are limited, and they cannot support the real-time storage and correlation analysis of massive security logs, making it impossible to restore and trace threat events.

[0005] Efficient storage and processing of data is the primary challenge of a secure big data platform. Traditional centralized storage is difficult to carry the massive heterogeneous data required for security management, and there are problems such as capacity bottlenecks and performance hotspots. Although simple data sharding solutions can alleviate the pressure to a certain extent, they lack dynamic scaling and intelligent scheduling mechanisms, cannot adapt to dynamic changes in load, and decentralized data storage faces serious security risks. Existing metadata management solutions such as RAID and HDFS mostly adopt static allocation strategies with fixed node roles, and lack adaptive capabilities in heterogeneous environments. In addition, existing data redundancy backup technologies such as multiple copies and erasure codes are difficult to balance data reliability, storage efficiency, and access performance. Taking three copies as an example, the storage cost is as high as 200%, which greatly increases the system overhead.

[0006] In summary, the existing network information security system is difficult to adapt to the increasingly complex network environment and security threats, and mainly faces the following technical challenges: first, it lacks big data aggregation and intelligent analysis capabilities, and cannot accurately perceive complex threats; second, it lacks an adaptive collaborative defense mechanism, and the ability to detect and deal with new attacks is insufficient; third, it lacks a scalable, secure, and efficient big data storage and processing platform, and it is difficult to support the real-time computing and analysis of massive heterogeneous security data. Therefore, it is urgent to develop a new set of network information security integrated management platforms supported by big data and artificial intelligence to achieve full-domain perception, intelligent analysis, dynamic defense, and visualization of cyberspace, and provide solid guarantees for maintaining national cyberspace sovereignty and security. Summary of the invention

[0007] The purpose of the present invention is to provide a network information security comprehensive analysis and monitoring system and method to solve the problems existing in the prior art.

[0008] The present invention provides a network information security comprehensive analysis and monitoring system, including a data sharding and encryption module, a dynamic mapping and distributed storage module, a consistency check and data reconstruction module, a real-time threat monitoring and response module, and a reporting and auditing module;

[0009] The data slicing and encryption module is used to slice and encrypt the original data;

[0010] The dynamic mapping and distributed storage module is used to dynamically adjust the storage strategy of data slices according to the network status;

[0011] The consistency check and data reconstruction module is used to verify the integrity of the distributed storage slices and perform data recovery when necessary;

[0012] The real-time threat monitoring and response module is used to detect security threats in the network and trigger corresponding defense mechanisms;

[0013] The report and audit module is used to generate system operation reports and record key events for administrators to audit and analyze;

[0014] Wherein, the dynamic mapping and distributed storage module includes:

[0015] Adaptive mapping unit, used to realize dynamic mapping of data slices to storage nodes and adjust the mapping strategy in real time according to the network status;

[0016] Frequent remapping unit, used to mine data access patterns, aggregate and store frequently accessed slices, and periodically randomly remap the remaining slices;

[0017] Distributed hash index unit, used to build slice DHT index based on XOR distance, to achieve fast and accurate slice location and query;

[0018] The multi-path redundant storage unit uses threshold secret sharing and random walk algorithm to generate multiple independent storage paths for each slice.

[0019] Specifically, the working method of the adaptive mapping unit includes: system state fuzzy set The system state fuzzy set contains fuzzy descriptions of different network states, such as "high load" and "medium threat level"; the mapping behavior fuzzy set The mapping behavior fuzzy set represents the fuzzy description of different slice storage mapping strategies, such as "balanced distributed storage" and "centralized redundant storage". W is a fuzzy incidence matrix, where the element w ij ∈[-1,1] represents the state Behavior The intensity and direction of the impact; real-time monitoring of network status indicators, such as load, delay, packet loss rate, and mapping them into a state membership vector U(t) = [μ1(t),μ2(t),...,μ n (t)], according to U(t) and FCM, the behavior incentive vector V(t) = [v1(t), v2(t), ..., v m (t)], the calculation formula is V(t) = f(U(t)⊙W), where f is the fuzzy inference operator, such as max-min synthesis, ⊙ represents fuzzy matrix multiplication, and the mapping behavior with the largest excitation degree is selected according to V(t). Map the data slices to the corresponding storage nodes, and finally generate the reinforcement signal r(t)∈[-1, 1] based on the system feedback, such as QoS and security events, and use it to update the FCM association matrix. The update formula is: Where η is the learning rate, Represents the outer product operation.

[0020] Specifically, the working method of the frequent remapping unit includes: defining a data slice item set D = {d1, d2, ..., d k}, each d i Represents the unique identifier of a data slice; analyze the user request log within a certain time window, mine the frequently accessed slice set, and generate the frequent slice item set F = {F1, F2, ..., F l},in Then use the pigeonhole principle to optimize if the frequent item set F i The size of exceeds the number of storage nodes m, that is, |F i |>m, then F i Divide into m subsets equally {f i1 , f i2 , ..., f im}, so that the intersection of any two subsets is minimized, which can be achieved by minimizing the following objective function: min∑ i，j |f ii ∩f ij |, satisfying the condition ∪ i f ij =F i And|f ij |≈[|F i | / m]; for each subset f ij , mapping the data slices to the jth storage node to achieve clustered storage of frequently accessed slices; for slices that do not appear in F, a random remapping strategy is adopted, that is, a part of the slices are randomly selected and remapped to different nodes at regular intervals.

[0021] Specifically, the working method of the distributed hash index unit includes: mapping the SHA-256 fingerprint of the data slice to 2 256 Kademlia keyspace, get the DHT key for slice k i ; Each storage node is also mapped to the key space, and the node identifier is the hash value of its public key; using the improved XOR distance metric, the XOR distance between nodes is defined as Among them C x , C y is the storage capacity of nodes x and y, is an XOR operation. When storing slices, query the k i The nearest, that is, the K nodes with the smallest XOR distance, and store the slice in the node with the largest capacity; when querying a slice, first locate the nearest node, and then search in the direction of increasing XOR distance until the target slice is found or the preset number of hops is reached.

[0022] Specifically, the working method of the multi-path redundant storage unit includes: i Generate n slice shares {s i1 ,s i2 ,…,s in}, where any k shares can restore the original slice; for each slice d i , based on the Kademlia protocol, generate m independent storage paths P i ={p i1 , p i2 , ..., p im}, where p ij is d i The node sequence of the j-th path;

[0023] A path construction algorithm based on random walk is used, in which the initial node is randomly selected uniformly and the transfer probability of each step is Where N(x) is the set of neighbor nodes of x, d(·) is the improved XOR distance, and α is the path diversity parameter. ij On the top, randomly select k nodes and store d i One of the k slice shares is collected to ensure that the storage nodes of each path do not overlap. When restoring data, m paths are queried separately, and each path restores a slice share. Once k shares are collected, the original slice can be reconstructed.

[0024] Specifically, the data slicing and encryption module includes: using the SHA-256 algorithm to slice the original data, and the size of each data slice is 256KB; the slicing process adopts a sliding window mechanism, the window size is 128KB, and the step size is 64KB, so that there is a 50% overlap between the slices; then any of the three-level encryption algorithms of AES-256, SM4 and Glemi is used to encrypt the slices, and the original data is encrypted in combination with the ECDSA digital signature.

[0025] Specifically, the consistency check and data reconstruction module includes:

[0026] The integrity verification unit uses the Merkle tree and BLS aggregate signature scheme to verify the integrity and consistency of the distributed storage slices;

[0027] The data recovery unit uses the Berlekamp-Welch algorithm to decode the erasure code and recover the original data from the local complete slices, tolerating the damage of up to 1 / 3 of the slices.

[0028] Specifically, the real-time threat monitoring and response module includes:

[0029] Multi-layer threat detection unit, combined with host-based and network-based intrusion detection systems, uses machine learning algorithms to analyze network traffic and system logs in real time to promptly detect DDoS attacks, port scans, and vulnerability exploit threats;

[0030] The adaptive defense unit dynamically adjusts the system's security policy through software-defined networking, and mirrors, redirects, limits the speed, and isolates suspicious threat traffic.

[0031] Specifically, the reporting and auditing module uses blockchain to form tamper-proof audit logs for key system events, and comprehensively uses big data analysis and visualization to generate multi-dimensional statistical reports to conduct real-time monitoring and early warning of system operation status.

[0032] The network information security comprehensive analysis and monitoring method comprises the following steps:

[0033] Step 1: Data segmentation and encryption: Use SHA-256 and sliding window mechanism to segment the data, and use adaptive multi-level encryption scheme to encrypt it;

[0034] Step 2: Adaptive dynamic mapping storage: The mapping strategy of data slices to storage nodes is inferred based on the fuzzy cognitive graph and adjusted in real time according to the network status;

[0035] Step 3: Frequent access mining and cluster storage: Regularly mine data access logs, extract frequent slice item sets, cluster and store them using the pigeonhole principle, and randomly remap the remaining slices;

[0036] Step 4: Construct XOR distance hash index: Calculate the XOR distance between data slices and storage nodes, map the slices to Kademlia distributed hash index, and achieve efficient slice positioning and load balancing;

[0037] Step 5, slice sharing and multi-path redundant storage: Use threshold secret sharing to generate shares for slices, use random walk algorithm to build multiple independent storage paths, and randomly select nodes on the path for redundant storage sharing;

[0038] Step 6: Integrity verification and erasure code recovery: Use Merkle tree and BLS signature to verify slice integrity and consistency, and restore damaged slice data through Berlekamp-Welch algorithm;

[0039] Step 7: Real-time detection and adaptive defense of multi-layer threats: Comprehensive host and network multi-layer threat detection, and dynamically adjust security policies through software-defined networks to implement targeted protection against suspicious threats;

[0040] Step 8. Blockchain cannot be tampered with log storage: key event logs and operation records are stored on the chain, and big data visualization technology is used to intuitively display the system operation status and abnormal events.

[0041] The present invention has the following beneficial effects:

[0042] First, in terms of system architecture, the present invention adopts a modular and layered design approach, dividing key functions such as data security, dynamic storage, consistency verification, and threat defense into relatively independent and closely coordinated sub-modules, achieving high cohesion and low coupling. This loosely coupled architecture has extremely strong flexibility and scalability, allowing the system to flexibly adjust storage strategies, security strategies, and defense mechanisms as demand changes and technology develops, achieving smooth upgrades and elastic scaling of functional modules.

[0043] At the same time, each module is standardized and seamlessly connected in terms of logic and interface. For example, the frequent slice set generated by the frequent remapping unit can be directly used as the input of the distributed hash index unit, and the multi-path redundant storage unit needs to call the integrity check result of the consistency verification unit to trigger data recovery. This loosely coupled and strongly connected design not only simplifies the difficulty of system development and integration, but also enables synergy between modules to improve system performance.

[0044] In addition, the architecture also adopts both dedicated hardware acceleration and software definition, which fully utilizes the high-speed advantages of hardware on the critical path and enjoys the customization benefits of software in flexible orchestration, which can be said to take into account both performance and flexibility.

[0045] Secondly, in terms of algorithm mechanism, the four core algorithms of the system complement each other and complement each other's advantages. Among them, the adaptive mapping algorithm is equivalent to the "central nervous system" of the system, which perceives the global state of the network in a fuzzy cognitive way and makes the best storage strategy decision based on it. This is like a real-time load balancer, ensuring the continuous availability and elastic scalability of storage services.

[0046] The frequent remapping algorithm is like an offline recommendation system. Through frequent item set mining and pigeonhole mapping, it transforms the user's historical access behavior into optimized data locality, greatly improving the cache hit rate and read and write performance of hot data. At the same time, the clustered storage of hot data alleviates the pressure of adaptive mapping to a certain extent. The two form a benign interaction of combining dynamic and static, and dividing hot and cold.

[0047] From the perspective of data organization, the XOR distance index cleverly utilizes the high-dimensionality reduction characteristics of the XOR algorithm, achieving efficient information compression while retaining data similarity. The Kademlia topology built based on XOR distance naturally has the small-world characteristics and the load balancing effect of consistent hashing, allowing data slices to be stored in the nearest neighbor and routed quickly, which provides an efficient infrastructure for adaptive mapping. At the same time, the good decentralized characteristics of the Kademlia topology have also become a prerequisite for frequent remapping and multi-path redundant storage.

[0048] Finally, in response to the needs of reliability and security, multi-path redundant storage adopts an innovative combination of threshold sharing and random walks, which optimizes storage redundancy while achieving balanced distribution and parallel access of slices, which not only greatly reduces the risk of data loss, but also significantly improves the experience quality of data-intensive applications. In addition, multi-copy storage of shared keys, coupled with erasure codes based on the Berlekamp-Welch algorithm, also enables data to withstand corruption or malicious behavior of a certain proportion of nodes, and has active immunity in a zero-trust environment.

[0049] It can be seen that the above four algorithms have formed cross-collaboration and complementary gains in different dimensions such as time and space, global and local, static and dynamic, and jointly protect the security, efficiency and reliability of the entire life cycle of data, and finally converge into the overall efficiency of the system. This synergy and cross-integration between algorithms is a unique advantage of the system.

[0050] In addition, the system also consciously balances conflicts and seeks benefits while avoiding harms in the selection of technical solutions. For example, in the face of the CAP contradiction between data consistency and availability in a distributed environment, multi-path redundant storage is adopted in the architecture, and Merkle tree verification and Berlekamp-Welch erasure code are integrated into the algorithm, which ultimately achieves the final consistency and fault self-recovery of cross-node storage, which largely alleviates the design dilemma brought by the CAP theorem.

[0051] For example, when high-dimensional data is mapped to Kademlia topology, the semantic features of node IDs and data objects naturally have the problem of dimensionality curse that cannot be preserved. To address this problem, the node storage capacity factor is introduced into the mapping algorithm, the calculation method of XOR distance is improved, and high-capacity nodes are preferred as neighbors when constructing Kademlia indexes. This cleverly transforms node heterogeneity into the system's load balancing capability, achieving a win-win situation between semantic mapping and computational balance.

[0052] In summary, the reason why the system can produce performance improvements that are superior to conventional solutions is due to its modular and layered design in architecture, as well as its innovative combination and mechanism coordination in algorithms. It is the organic integration of multiple technical links and system optimization that enables the system to form a set of global optimal solutions for data security and storage efficiency issues in massive heterogeneous environments, and ultimately delivers a highly reliable, high-performance, scalable, and controllable data management solution to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a framework diagram of the network information security comprehensive analysis and monitoring system of the present invention;

[0054] Figure 2 The flowchart of the network information security comprehensive analysis and monitoring system of the present invention. DETAILED DESCRIPTION

[0055] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the network information security comprehensive analysis and monitoring system proposed by the present invention, its specific implementation method, structure, features and effects are described in detail below in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0056] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0057] The specific scheme of the network information security comprehensive analysis and monitoring system provided by the present invention is described in detail below with reference to the accompanying drawings.

[0058] Example 1

[0059] See also Figure 1 , Network information security comprehensive analysis and monitoring system, Network information security comprehensive analysis and monitoring system, including data sharding and encryption module 1, dynamic mapping and distributed storage module 2, consistency check and data reconstruction module 3, real-time threat monitoring and response module 4 and reporting and auditing module 5;

[0060] A key module of the present invention is a data slicing and encryption module 1, which is used to perform secure slicing and encryption processing on the original data. The module uses the SHA-256 algorithm to slice the data, and the size of each data slice is set to 256KB. A sliding window mechanism is used in the slicing process, with a window size of 128KB and a sliding step of 64KB. This ensures that there is a 50% overlap between two adjacent slices, which is conducive to subsequent data recovery.

[0061] In terms of data encryption, this module adopts an adaptive multi-level encryption scheme. According to the sensitivity of the data, you can dynamically select AES-256, SM4 national encryption algorithm or Grameen three-level encryption algorithm to ensure data security while taking into account encryption and decryption efficiency. In addition, ECDSA digital signature technology is introduced to authenticate the integrity and source of data slices to ensure the non-repudiation of data.

[0062] For example, for a 1GB original data file, the above sharding scheme can obtain about 4000 slices. If the security level of the data is marked as "high", each slice will be encrypted with AES-256 and signed with ECDSA before subsequent storage and distribution. In this way, even if one or two slices are intercepted, it is difficult to damage the confidentiality of the original data.

[0063] In the embodiment of the present invention, the dynamic mapping and distributed storage module 2 is one of the core components of the system, responsible for intelligently mapping the encrypted data slices to the storage nodes in the distributed network, and dynamically optimizing the storage strategy according to the network status and data characteristics to achieve the goals of load balancing, high availability and security protection. The module mainly includes four innovative functional units: adaptive mapping unit, frequent remapping unit, distributed hash index unit and multi-path redundant storage unit.

[0064] The adaptive mapping unit adopts an intelligent algorithm that integrates learning automata and fuzzy cognitive maps (FCM) to automatically adjust the mapping strategy of data slices to storage nodes according to the real-time status of the system, enabling the system to dynamically adapt to load changes and security events and provide continuous high-quality services.

[0065] The algorithm of the adaptive mapping unit first defines a fuzzy set of system states and a fuzzy set of storage behaviors Respectively represent different network states and data storage strategies. For example, It can mean "the system load is high". Indicates "medium security threat level", It can mean "balanced distributed storage". represents "centralized redundant storage", etc. Next, the algorithm builds a fuzzy cognitive graph Where W is an n×m matrix, which represents the influence of the state concept on the behavior concept, and its value range is [-1, 1]. ij >0 means that state i is conducive to taking action j, W ij <0 means that state i is not conducive to behavior j, W ij The larger the absolute value of is, the stronger the impact is. FCM links the system state to the optimal decision through a causal chain, providing an explainable reasoning framework for the adaptive process.

[0066] During the operation of the system, the adaptive mapping unit monitors various performance indicators of the network in real time, including node load, response delay, transmission bandwidth, packet loss rate, available storage, etc., and calculates their membership to each fuzzy state through a predefined membership function to form a state membership vector U(t) = [μ1(t), μ2(t), ..., μ n (t)]. For example, if the current system load is 80%, the corresponding membership vector may be [0.8, 0.4, 0.1], indicating that the probability that the system is in a "high load" state is 80%, "medium load" is 40%, and "low load" is 10%.

[0067] After obtaining the state membership vector, the adaptive mapping unit inputs it into the FCM and calculates the corresponding behavior incentive vector V(t) = [v1(t), v2(t), ..., v m (t)]:

[0068] V(t)=f(U(t)⊙W)

[0069] Where f is the fuzzy reasoning operator, commonly used ones include max-min synthesis method, max-prod synthesis method, etc., and ⊙ represents fuzzy matrix multiplication.

[0070] Continuing with the above example, assuming U(t) = [0.8, 0.4, 0.1], and the first column of W is [0.7, -0.4, 0.5], then the first element v1(t) in V(t) = max{min{0.8, 0.7}, min{0.4, -0.4}, min{0.1, 0.5}} = 0.5, indicating that under the current state, the incentive for adopting a balanced distributed storage strategy is 0.5.

[0071] The system will select the storage strategy with the highest incentive As the optimal mapping behavior, the mapping relationship between data slices and storage nodes is dynamically adjusted. It is worth noting that the weight matrix W in FCM is usually initialized by expert knowledge, but it will be continuously optimized as the system learns. After a decision is made, the adaptive mapping unit will generate a reinforcement signal r(t)∈[-1, 1] based on network feedback (such as QoS improvement, security incident mitigation, etc.), and use it to correct the weight of FCM to maximize the positive feedback of future decisions:

[0072]

[0073] Where η∈(0,1) is the learning rate, is an outer product operation. In this way, FCM can continue to evolve according to the real-time feedback of the system, and continuously improve the accuracy and timeliness of mapping decisions. For example, if balanced distributed storage is adopted under high load, the system throughput drops by 20%, and a negative feedback r(t) = -0.2 is generated. After adjustment, FCM will reduce the causal strength of the "high load" state to the "balanced distributed" behavior, and will tend to choose other strategies such as centralized storage next time.

[0074] The frequent remapping unit aims to discover frequently accessed data slices by mining user access logs and aggregate them and store them on the same node to improve the access efficiency and security of the system. The algorithm first defines the data slice item set D = {d1, d2, ..., d k} n where d i is the unique identifier of the slice. Then, the slice access pattern mining is performed on the user request log within a certain time window (such as 1 hour) to generate a frequent slice item set F = {F1, F2, ..., F l},in is a set of frequently accessed slices. Next, the algorithm uses the pigeonhole principle to partition frequent item sets. Specifically, if |F i |>m (where m is the number of storage nodes), then F i Divide into m subsets equally {f i1 , f i2 , ..., f im}, so that the intersection of any two subsets is minimal. That is:

[0075]

[0076] This ensures that frequently accessed slices are distributed on the same node as much as possible. After the partition is completed, for each subset f ijThe algorithm maps the data slices to the jth storage node to achieve clustered storage of frequently accessed slices and improve access efficiency. For slices that do not appear in the frequent itemset F, the algorithm adopts a random remapping strategy, that is, randomly selects a portion of slices (such as 10%) at regular intervals (such as 1 day) and remaps them to different nodes to improve the security and robustness of the system.

[0077] For example: Assume that the system has 1000 slices {d1, d2, ..., d 1000}, after mining the log, three frequent item sets are generated:

[0078] F1={d2, d5, d7}, F2={d3, d6, d9}, F3={d1, d4, d8}.

[0079] F1 can be divided into {d2, d5} and {d7}, F2 can be divided into {d3, d6} and {d9}, and F3 can be divided into {d1, d4} and {d8}. Then {d2, d5} is stored in node 1, {d7} is stored in node 2, {d3, d6} is stored in node 3, {d9} is stored in node 4, {d1, d4} is stored in node 5, and {d8} is stored in node 1. For the remaining 990 infrequent slices, 99 slices are randomly selected for remapping every day, such as {d 10 , d 20 , ..., d 990} is mapped from the original node to {2, 4, 1, 3, 5, ...}.

[0080] Based on the Kademlia distributed hash table (DHT) protocol, the distributed hash index unit is optimized for the characteristics of data slices to achieve efficient slice storage and query. First, the SHA-256 fingerprint of the data slice is mapped to 2 256 Kademlia keyspace, get the DHT key for slice k i ; Each storage node is also mapped to the key space, and the node identifier is the hash value of its public key; using the improved XOR distance metric, the XOR distance between nodes is defined as Among them C x , C y is the storage capacity of nodes x and y, This formula introduces the node capacity factor while maintaining the XOR distance property, and tends to select high-capacity nodes to store data to achieve load balancing. When storing slices, query the nodes with k nodes in turn. i The nearest, that is, the K nodes with the smallest XOR distance, and the slice is stored in the node with the largest capacity; when querying a slice, first locate the nearest node, and then search in the direction of increasing XOR distance until the target slice is found or the preset number of hops is reached

[0081] When storing a slice, the algorithm sequentially queries the DHT key k associated with the slice. i XOR the K nodes with the closest distance (usually K = 20), and store the slice on the node with the largest storage capacity. When querying a slice, the algorithm first locates the node with the k closest distance. i The nearest node is searched in the direction of increasing XOR distance until the target slice is found or the preset maximum number of hops (such as 10) is reached.

[0082] For example: Assume that the DHT key of a slice is 0011, there are 8 nodes in the system, and the node identification and storage capacity are:

[0083] A(0000, 500G), B(1001, 200G), C(0100, 900G), D(1011, 600G), E(0010, 300G), F(1100, 700G), G(0111, 400G), H(1110, 100G). When storing the slice, query the nodes E(0001), A(0011), G(0100), and C(0111) in order of XOR distance from near to far. Among these four nodes, C has the largest storage capacity, so the slice is stored in C. When querying the slice, first visit the nearest E node (0001), then visit the next nearest A node (0011), find the target slice at A node, and the query ends. The path length is 2 hops.

[0084] In the process of slice mapping, the multi-path redundant storage unit generates multiple independent storage paths at the same time, and introduces error correction code technology to realize the distributed redundant storage of slices, so as to improve the reliability and fault tolerance of the system. Specifically, the algorithm first adopts the (n, k) threshold Shamir secret sharing scheme to store each data slice d i Generate n slice shares {s i1 ,s i2 ,…,s in}, so that any k shares can restore the original slice.

[0085] Then, the algorithm performs a i Generate m independent storage paths P based on Kademlia protocol i ={p i1 , p i2 , ..., p im}, where ρ ij is d i The node sequence of the jth path. When generating the path, the algorithm adopts a random walk-based approach, the initial node is discretely uniformly randomly selected, and the probability of transferring to the next node x at each step is:

[0086]

[0087] Where y is the neighbor node of x, N(x) is the set of neighbor nodes of x, d(-) is the improved XOR distance metric, and α is the parameter that controls the diversity of the path. The larger α is, the more it tends to choose nodes that are farther away. After generating the path, the algorithm selects each path p. ij Randomly select k nodes and store d i One of the k slice shares is collected to ensure that the storage nodes of each path do not overlap. When data needs to be restored, m paths are queried separately, and each path restores a slice share. Once any k shares are collected, the original slice can be reconstructed.

[0088] For example: suppose a slice adopts the (6, 3) Shamir scheme to generate 6 slice shares {s1, s2, s3, s4, s5, s6}, and constructs 3 independent storage paths in the DHT network: p1 = {A, B, C, D}, p2 = {E, F, G, H}, p3 = {I, J, K, L}, then 3 nodes can be randomly selected on p1 to store {s1, s2, s3}, {s4, s5} can be stored on p2, and s6 can be stored on p3. If any one node of p1 and any two nodes of p2 fail, one share s6 can still be obtained from p3, and the original slice can be restored together with the remaining one share of p2 and the remaining two shares of p1. Due to the randomness and independence of the path, the risk of an attacker destroying all redundant shares of a slice at the same time is greatly reduced.

[0089] In an embodiment of the present invention, the consistency check and data reconstruction module 3 is mainly composed of an integrity verification unit and a data recovery unit, which aims to ensure the consistency and recoverability of data slices in a distributed storage environment.

[0090] The integrity verification unit uses Merkle tree and BLS aggregate signature scheme to perform integrity verification on the slice data on each storage node. Merkle tree is a hash binary tree. Leaf nodes store the hash value of the data slice, and non-leaf nodes store the hash value of their child node hash value, recursively to the root node. Merkle tree can be used to efficiently verify the integrity of the data set: if a slice is tampered with, its hash value changes, which causes the root hash of the Merkle branch where it is located to be inconsistent with the original Merkle root, and the integrity is damaged.

[0091] However, frequent verification of the Merkle tree will bring a lot of computational and communication overhead. To this end, the integrity verification unit introduces the BLS aggregate signature scheme on top of the Merkle tree, which can aggregate the Merkle root signatures of all nodes into a unified signature, greatly reducing the signature overhead. The BLS signature uses bilinear mapping to aggregate multiple signatures and supports batch verification. Its security is based on the computational Diffie-Hellman (CDH) assumption.

[0092] The specific workflow is as follows: Each storage node regularly calculates the Merkle tree of the slices it stores, and uses its own private key to perform BLS signature on the Merkle root hash, and submits it to the integrity verification unit. The verification unit first verifies the validity of the signatures of each node with the BLS public key, and then aggregates all valid signatures into a signature σ. Finally, the verification unit randomly selects some slices, recalculates their Merkle root hashes, and performs batch verification with the aggregated signature σ. If the verification passes, all slices are considered complete and consistent; otherwise, the tampered slices and nodes can be located by binary search.

[0093] For example: suppose the system has 1,000 storage nodes, and each node stores 10,000 slices. The traditional Merkle tree solution requires each node to submit 10,000 hash values ​​and 1 signature, totaling 10 million hash values ​​and 1,000 signatures. After adopting the BLS aggregate signature, although 10 million hash values ​​still need to be transmitted, all signatures are aggregated and compressed into 1, and the communication volume is reduced by nearly 1,000 times. At the same time, the verification complexity of the aggregate signature is only O(1), which is much better than the O(n) complexity of a single verification and then summation. Therefore, the Merkle tree + BLS solution significantly improves the verification efficiency while ensuring the security of integrity verification.

[0094] In the embodiment of the present invention, the data recovery unit is responsible for automatically repairing data when data corruption is detected to ensure the availability of the storage system. Compared with traditional solutions such as three copies and K+M erasure codes, this unit uses the Berlekamp-Welch algorithm to achieve a better balance between coding redundancy and recovery efficiency.

[0095] The Berlekamp-Welch algorithm is a highly robust algebraic erasure coding scheme. Its core idea is to use the polynomial data value principle and robustness inference to accurately reconstruct the original data under the premise that some data is known to be damaged. Specifically, the algorithm regards the original data as an n-degree polynomial f(x) over a finite field, and converts the n+1 points (x) on the polynomial into i ,y i) is stored as data fragments on n+1 nodes. When some fragments are missing or damaged, the Welch-Berlekamp algorithm can be used to restore the surviving m fragments. Perform interpolation to obtain a value no more than The “error location polynomial” E(x) and the original data polynomial f(x) are used to recover the complete data. As long as the damaged fragments do not exceed The algorithm is always able to reconstruct exactly.

[0096] In actual applications, the data recovery unit first collects data slices and their locations from each storage node and removes the slices that fail the integrity verification. Then, the Berlekamp-Welch algorithm is used to interpolate the remaining slices to obtain the original data polynomial and the location of the missing slices. Finally, the reconstructed data polynomial is substituted into the missing position to fill in all the damaged slices and complete the data recovery.

[0097] As an example, the system uses the (6, 3) Shamir threshold scheme to store data slices in 7 nodes. If the slices of nodes 1, 4, and 5 are lost or damaged, 4 valid slices (x2, y2), (x3, y3), (x6, y6), and (x7, y7) can be collected from nodes 2, 3, 6, and 7. The Berlekamp-Welch algorithm can be used to solve the quadratic polynomial f(x) and the linear polynomial E(x), so that for any i,

[0098] E(x i )f(x i )=y i E(x i ). Substituting x1, x4, x5 into f(x), we can get the missing slice values ​​y1, y4, y5 and successfully restore the original data. Compared with the traditional RS erasure code, the decoding complexity of the Berlekamp-Welch algorithm is increased from O(n 3 ) is reduced to O(n 2 ), and can tolerate up to half of the data corruption, making it more robust.

[0099] In an embodiment of the present invention, the real-time threat monitoring and response module 4 is composed of a multi-layer threat detection unit and an adaptive defense unit working together to achieve fast and accurate detection and response to various security threats through multi-level threat perception of the network and host, as well as software-defined flexible scheduling.

[0100] The multi-layer threat detection unit deploys the network-based intrusion detection system NIDS and the host-based intrusion detection system HIDS in the system to monitor the system's network communications and host activities in an all-round manner. Among them, NIDS obtains network data packets through mirror splitting and bypass sniffing, and constructs a multi-dimensional traffic description vector through operations such as protocol analysis, traffic statistics, and feature extraction; while HIDS collects data such as the host's application process, resource usage, and user behavior through kernel hooks, system logs, and file integrity, forming a series of behavioral event sequences.

[0101] Based on data collection, the threat detection unit uses machine learning algorithms to analyze network and host data in real time to detect various intrusion threats in a timely manner. For malicious behaviors such as DDoS attacks, port scanning, and vulnerability exploits at the network layer, the unit focuses on abnormal feature patterns in traffic, such as sudden massive small packets, suspicious port combination sequences, and deformed protocol structures. By training an unsupervised learning model (such as single-classification SVM and isolation forest) based on normal traffic, the unit can sensitively capture abnormal deviations in traffic and achieve early warning of DDoS attacks. In addition, for known vulnerability attacks, the unit uses supervised learning models (such as XGBoost and LSTM) for accurate classification, which can identify malicious traffic as soon as it appears.

[0102] In response to threats such as system penetration, privilege escalation, and backdoor implantation at the host layer, the multi-layer detection unit extracts a series of behavioral pattern features from the creation, change, and interaction of processes, files, users, and other objects, and establishes a behavioral baseline through time series analysis, graph embedding, and other technologies. When an abnormal deviation occurs, the unit describes the evolution of the threat through methods such as finite state machines (FSMs), forming a causal reasoning diagram for hazard judgment and intuitively explaining the root cause of the risk. In short, with the cooperation of NIDS and HIDS, the multi-layer threat detection unit implements in-depth defense-style security monitoring of the network and host, greatly improving the system's ability to resist complex threats.

[0103] On the basis of threat detection, the adaptive defense unit further utilizes the advantages of software-defined network SDN to dynamically adjust security policies and accurately combat intrusion behaviors according to factors such as the degree of threat damage and attack stage. The unit is deeply integrated with the SDN controller, and through programmable Open Flow switches and flow table mechanisms, it flexibly arranges security functions and implements mirroring, redirection, speed limiting, isolation and other disposal measures for suspicious traffic.

[0104] For example, when NIDS finds that a source IP continuously launches SYN Flood attacks on the system, it will immediately trigger the adaptive defense unit. The defense unit dynamically sends filtering rules in the TCAM flow table of the switch through the Open Flow southbound interface, imports the malicious SYN request of the IP into the "black hole" (NULL device), and other normal traffic is not affected, so as to achieve accurate filtering. When the attack intensifies, the defense unit further dispatches the software-defined cleaning device to filter the attack traffic in real time, and links the cloud protection platform to elastically expand the capacity to minimize the impact of the attack. In addition, with the service chain SFC orchestration capability of the NFV platform, the defense unit can also redirect suspicious traffic and divert it to special analysis devices such as WAF, honeypot, sandbox, etc., to achieve in-depth analysis and trapping of threats.

[0105] It can be seen that the adaptive defense unit combines the flexible programmability of SDN with professional security protection capabilities, enabling the system to quickly and accurately detect and respond to known and unknown network threats. Compared with traditional isolated protection and post-event response, this mechanism has taken a big step forward in "trading attack time for defense space."

[0106] In an embodiment of the present invention, the reporting and auditing module 5 makes full use of blockchain and big data technology to carry out "trusted recording, visual presentation, and accountable management and control" around the operation process of the system.

[0107] Each module of the system generates a large amount of log data while processing business, including security logs such as network communication logs and host behavior logs, as well as business logs such as service call logs and resource usage logs. These logs contain a complete record of system behavior and are an important basis for post-audit and accountability for violations. However, the original logs stored in a decentralized manner can easily be maliciously tampered with, and the massive heterogeneous logs pose great challenges to analysis and management.

[0108] Therefore, the report and audit module 5 innovatively introduces the alliance blockchain, which stores the key behaviors and major events in the system operation in the form of transactions in the blockchain ledger after hash calculation. The blockchain network consists of multiple trusted nodes. The generation of new blocks requires the consensus of the majority of nodes. No single node can tamper with the stored logs privately. In addition, the block header is associated with all transactions through the Merkle tree hash, and a slight data change will also cause the hash value of the entire blockchain to become invalid. Using this chain storage and consensus verification mechanism, the blockchain can provide strong anti-tampering and traceability protection for audit logs.

[0109] For example, an employee attempted to use the database administrator's privileges to steal data and then destroy the logs. However, under the blockchain mechanism, the illegal READ operation of accessing the database and the dangerous command to clear the logs have been synchronously recorded in the ledger block, witnessed by all nodes in the network, and cannot be denied. The administrator later traced back each transaction through the blockchain, not only discovering the data theft, but also finding the complete chain of evidence such as the time, path, and system involved in the intrusion, providing decisive evidence support for subsequent responsibility tracing and loss assessment. Therefore, compared with the traditional centralized log system, the blockchain can better avoid various security risks and improve the credibility of system audits.

[0110] While storing data in a trusted manner, the reporting and auditing modules also make full use of big data and visualization technology to depict the overall operating status of the system from multiple levels and perspectives, and achieve real-time warning of abnormalities, situation awareness, and correlation analysis. This module aggregates massive log data scattered at different nodes and time points to the big data platform, and after ETL processing such as structured analysis and field extraction, it is stored in the distributed file system HDFS, and indexed and organized through tools such as Hive to build a structured and secure big data warehouse.

[0111] On this basis, the module designs a series of multi-dimensional OLAP analysis models for security management, which aggregate and calculate system events from the perspectives of time, space, and logic. For example, in the time dimension, the traffic change trend can be counted at different granularities (days, hours, and minutes) to discover the "oscillation" characteristics of DDoS attacks; in the spatial dimension, the traffic distribution of each node, subnet, and region can be analyzed to locate the "convergence" area of ​​abnormal traffic; in the logical dimension, the association rules between multiple alarm events can be mined to identify the "kill chain" of APT attacks. These calculation results are rendered and organized by visualization engines such as Kibana and Graphite to form a set of management-oriented security situation screens, covering the key performance indicators KPI of system operation, the statistical distribution of various security events and threats, and early warning prompts of high-risk vulnerabilities and major events, etc., providing managers with a comprehensive and intuitive understanding of the system security status and a powerful tool for managing global security risks.

[0112] For example, after the security administrator enters the situation screen, he will first see a security score dashboard, which quantitatively displays the overall security level of the system from multiple dimensions such as network security, host security, data security, and compliance. Below it, a 7×24-hour full-time security event statistical matrix chart uses heat colors to mark the event frequency distribution in different time periods, which can easily find abnormal peaks during nighttime and weekends. Click on any cell with a significantly increased frequency, and the report will automatically drill down to a TOP5 security event portrait, which intuitively lists high-risk events such as Windows remote desktop brute force cracking and Web SQL injection. Click on the brute force cracking to jump to a source IP and destination IP association diagram. The two largest nodes in the diagram correspond to two overseas IPs on the public network, which are connected to multiple nodes on the intranet, showing a typical botnet distribution. The administrator exported the IP involved and sent it to the firewall and WAF devices for continuous blocking. After that, the traffic column of the report quickly fell back, and the security score of the dashboard also rebounded, and the risk was controlled.

[0113] In summary, the real-time threat monitoring and response module 4 builds a deep, collaborative, and adaptive detection and defense system to ensure the system's ability to resist various known and unknown threats, while the reporting and auditing module 5 uses emerging technologies such as blockchain and big data to provide reliable behavior records and visual situational awareness for system operation, ultimately forming a comprehensive set of dynamic security protection and control closed loops. These two modules, together with traditional security components such as identity management and key management, jointly build a solid and reliable security line of defense for the system.

[0114] Example 2

[0115] See also Figure 2 , the network information security comprehensive analysis and monitoring method includes the following steps:

[0116] Step 1: Data segmentation and encryption: Use SHA-256 and sliding window mechanism to segment the data, and use adaptive multi-level encryption scheme to encrypt it;

[0117] Step 2: Adaptive dynamic mapping storage: The mapping strategy of data slices to storage nodes is inferred based on the fuzzy cognitive graph and adjusted in real time according to the network status;

[0118] Step 3: Frequent access mining and cluster storage: Regularly mine data access logs, extract frequent slice item sets, cluster and store them using the pigeonhole principle, and randomly remap the remaining slices;

[0119] Step 4: Construct XOR distance hash index: Calculate the XOR distance between data slices and storage nodes, map the slices to Kademlia distributed hash index, and achieve efficient slice positioning and load balancing;

[0120] Step 5, slice sharing and multi-path redundant storage: Use threshold secret sharing to generate shares for slices, use random walk algorithm to build multiple independent storage paths, and randomly select nodes on the path for redundant storage sharing;

[0121] Step 6: Integrity verification and erasure code recovery: Use Merkle tree and BLS signature to verify slice integrity and consistency, and restore damaged slice data through Berlekamp-Welch algorithm;

[0122] Step 7: Real-time detection and adaptive defense of multi-layer threats: Comprehensive host and network multi-layer threat detection, and dynamically adjust security policies through software-defined networks to implement targeted protection against suspicious threats;

[0123] Step 8. Blockchain cannot be tampered with log storage: key event logs and operation records are stored on the chain, and big data visualization technology is used to intuitively display the system operation status and abnormal events.

[0124] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Network information security comprehensive analysis and monitoring system, characterized by , including: data sharding and encryption module, dynamic mapping and distributed storage module, consistency check and data reconstruction module, real-time threat monitoring and response module and reporting and auditing module; The data slicing and encryption module is used to slice and encrypt the original data; The dynamic mapping and distributed storage module is used to dynamically adjust the storage strategy of data slices according to the network status; The consistency check and data reconstruction module is used to verify the integrity of the distributed storage slices and perform data recovery; The real-time threat monitoring and response module is used to detect security threats in the network and trigger corresponding defense mechanisms; The report and audit module is used to generate system operation reports and record key events for administrators to audit and analyze; Wherein, the dynamic mapping and distributed storage module includes: Adaptive mapping unit, used to realize dynamic mapping of data slices to storage nodes and adjust the mapping strategy in real time according to the network status; Frequent remapping unit, used to mine data access patterns, aggregate and store frequently accessed slices, and periodically randomly remap the remaining slices; Distributed hash index unit, used to build slice DHT index based on XOR distance, to achieve fast and accurate slice location and query; Multi-path redundant storage unit, which uses threshold secret sharing and random walk algorithm to generate multiple independent storage paths for each slice; The working method of the adaptive mapping unit comprises: Fuzzy set of system states The fuzzy set of system states contains fuzzy descriptions of different network states; Fuzzy set of mapping behaviors The fuzzy set of mapping behaviors represents fuzzy descriptions of different slice storage mapping strategies; Fuzzy Cognitive Map FCM W is a fuzzy association matrix, and the element w ij ∈[-1,1] represents the influence intensity and direction of the state on the behavior ; Real-time monitor network state indicators, and map them into a state membership degree vector U(t)=[μ1(t), μ2(t),..., μ n (t)] through fuzzy processing. According to U(t) and FCM, calculate the behavior excitation degree vector V(t)=[v1(t), v2(t),..., v m (t)] through fuzzy inference. The calculation formula is V(t)=f(U(t)⊙W), where f is a fuzzy inference operator, and ⊙ represents fuzzy matrix multiplication. Select the mapping behavior with the largest excitation degree according to V(t) to map the data slice to the corresponding storage node. Finally, according to the system feedback, generate a reinforcement signal r(t)∈[-1,1], and use it to update the association matrix of FCM. The update formula is where η is the learning rate represents the outer product operation 2. The system according to claim 1, characterized in that The working method of the frequent remapping unit includes: defining a data slice item set D = {d1, d2, ..., d k }, each d i Represents the unique identifier of a data slice; analyze the user request log within a certain time window, mine the frequently accessed slice set, and generate the frequent slice item set F = {F1, F2, ..., F l },in Then use the pigeonhole principle to optimize if the frequent item set F i The size of exceeds the number of storage nodes m, that is, |F i |>m, then F i Divide into m subsets equally {f i1 , f i2 , ..., f im }, so that the intersection of any two subsets is minimized, which is achieved by minimizing the following objective function: min∑ i，j |f ii ∩f ij |, satisfying the condition ∪ i f ij =F i And|f ij |≈[|F i | / m]; for each subset f ij , mapping the data slices to the jth storage node to achieve clustered storage of frequently accessed slices; for slices that do not appear in F, a random remapping strategy is adopted to randomly select a part of the slices and remap them to different nodes at regular intervals.

3. The system according to claim 2, characterized in that The working method of the distributed hash index unit includes: mapping the SHA-256 fingerprint of the data slice to 2 256 Kademlia keyspace, get the DHT key for slice k i ; Each storage node is also mapped to the key space, and the node identifier is the hash value of its public key; using the improved XOR distance metric, the XOR distance between nodes is defined as Among them C x , C y is the storage capacity of nodes x and y, is an XOR operation. When storing slices, query the k i Find the K nodes with the smallest XOR distance and store the slice in the node with the largest capacity. When querying a slice, first locate the nearest node and then search in the direction of increasing XOR distance until the target slice is found or the preset number of hops is reached.

4. The system according to claim 3, characterized in that The working method of the multi-path redundant storage unit comprises: i Generate n slice shares {s i1 ,s i2 , ..., s in }, where any k shares can restore the original slice; for each slice d i , based on the Kademlia protocol, generate m independent storage paths P i ={p i1 , p i2 , ..., p im }, where p ij is d i The node sequence of the j-th path; A path construction algorithm based on random walk is used, in which the initial node is randomly selected uniformly and the transfer probability of each step is Where N(x) is the set of neighbor nodes of x, d(·) is the improved XOR distance, and α is the path diversity parameter. ij On the top, randomly select k nodes and store d i One of the k slice shares is collected to ensure that the storage nodes of each path do not overlap. When restoring data, m paths are queried separately, and each path restores a slice share. Once k shares are collected, the original slice can be reconstructed.

5. The system according to claim 1, characterized in that The data slicing and encryption module includes: using the SHA-256 algorithm to slice the original data, and the size of each data slice is 256KB ； The sharding process uses a sliding window mechanism with a window size of 128KB and a step size of 64KB, so that there is a 50% overlap between slices; then any pair of slices is encrypted using the AES-256, SM4 and Grameen three-level encryption algorithms, and the original data is encrypted in combination with the ECDSA digital signature.

6. The system according to claim 1, characterized in that The consistency check and data reconstruction module includes: Integrity verification unit, using Merkl e Tree and BLS aggregate signature scheme to verify the integrity and consistency of distributed storage slices; The data recovery unit uses the Berlekamp-Welch algorithm to decode the erasure code and recover the original data from the local complete slices, tolerating up to 1 / 3 of the slice damage.

7. The system according to claim 1, characterized in that , the real-time threat monitoring and response module includes: Multi-layer threat detection unit, combined with host-based and network-based intrusion detection systems, uses machine learning algorithms to analyze network traffic and system logs in real time to promptly detect DDoS attacks, port scans, and vulnerability exploit threats; The adaptive defense unit dynamically adjusts the system's security policy through software-defined networking, and mirrors, redirects, limits the speed, and isolates suspicious threat traffic.

8. The system according to claim 1, characterized in that ,The report and audit module uses blockchain to form tamper-proof audit logs for key ,system events, and comprehensively uses big data analysis and visualization to ,generate multi-dimensional statistical reports for real-time monitoring and ,early warning of the system operation status.

9. A method according to any one of claims 1 to 8, characterized in that , including the following steps: Step 1: Data segmentation and encryption: Use SHA-256 and sliding window mechanism to segment the data, and use adaptive multi-level encryption scheme to encrypt it; Step 2: Adaptive dynamic mapping storage: The mapping strategy of data slices to storage nodes is inferred based on the fuzzy cognitive graph and adjusted in real time according to the network status; Step 3: Frequent access mining and cluster storage: Regularly mine data access logs, extract frequent slice item sets, cluster and store them using the pigeonhole principle, and randomly remap the remaining slices; Step 4: Construct XOR distance hash index: Calculate the XOR distance between data slices and storage nodes, map the slices to Kademlia distributed hash index, and achieve efficient slice positioning and load balancing; Step 5, slice sharing and multi-path redundant storage: Use threshold secret sharing to generate shares for slices, use random walk algorithm to build multiple independent storage paths, and randomly select nodes on the path for redundant storage sharing; Step 6: Integrity verification and erasure code recovery: Use Merkle tree and BLS signature to verify slice integrity and consistency, and restore damaged slice data through Berlekamp-Welch algorithm; Step 7: Real-time detection and adaptive defense of multi-layer threats: Comprehensive host and network multi-layer threat detection, and dynamically adjust security policies through software-defined networks to implement targeted protection against suspicious threats; Step 8. Blockchain cannot be tampered with log storage: key event logs and operation records are stored on the chain, and big data visualization technology is used to intuitively display the system operation status and abnormal events.

Images