A data security processing method and system based on distributed storage

Abstract

This invention discloses a data security processing method and system based on distributed storage, belonging to the field of distributed storage technology. The method includes: responding to storage requests, performing multi-dimensional feature analysis on the target data to generate data structure feature vectors and access behavior feature vectors. A dynamic block partitioning strategy is adopted to divide the data into multiple blocks based on the two types of feature vectors and generate block attribute labels. Then, combining the block labels with the real-time load of storage nodes, the optimal target node is matched, and a routing table for inter-node association is constructed. A security engine is invoked to encrypt and encapsulate the routing table to ensure transmission security, while simultaneously writing the blocks to the corresponding nodes in parallel. After writing is complete, a global data index is generated based on the block attribute labels and the routing table, and stored independently on the index management node, achieving separation of index and data. This invention improves the scalability, load balancing capability, and retrieval efficiency of the storage system, and is suitable for large-scale data management scenarios.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, specifically to a data security processing method and system based on distributed storage. Background Technology

[0002] Distributed storage systems provide scalable storage capabilities by distributing data across multiple nodes. During data writing, the data needs to be segmented into blocks, and the storage location of each block needs to be determined. Simultaneously, index information for subsequent data retrieval needs to be generated. Traditional data segmentation methods typically use fixed sizes or pre-defined granularity based solely on file type, without considering the structural characteristics of the data itself and the dynamic relationship between access behavior. When data access patterns exhibit high frequency or concentrated periods, fixed segmentation struggles to balance access efficiency and storage resource utilization, easily leading to latency in accessing hot data or storage fragmentation.

[0003] At the data security level, the routing information of the nodes where each block resides in distributed storage is crucial for reconstructing complete data. Existing solutions typically store routing information directly on the nodes participating in data storage, or attach the routing table in plaintext to the data blocks. This approach exposes the routing table to the same security domain as the data. If a single storage node is attacked or data is leaked, attackers can obtain the complete routing mapping relationship, thereby reconstructing all data blocks and compromising data confidentiality. Furthermore, the coupling of index information with storage nodes makes index updates complex during node load changes or fault migration, and the lack of independent security isolation mechanisms for the index nodes further expands the security risk exposure.

[0004] To address the aforementioned issues, a processing method is needed that can dynamically adjust the granularity of data chunks based on the data's structural characteristics and access behavior. Simultaneously, a processing mechanism is required to securely isolate routing information from data chunks and encrypt the routing itself, thereby eliminating the risk of global data exposure due to routing information leakage. Summary of the Invention

[0005] This invention provides a data security processing method and system based on distributed storage. It dynamically adjusts the granularity of data segmentation based on the data structure feature vector and access behavior feature vector of the target data to generate data blocks that adapt to data characteristics and access patterns. It also encrypts and encapsulates the inter-node association routing table to generate a secure routing encapsulation, which is then stored separately from the data blocks on an independent index management node. This achieves secure isolation and encryption protection between routing information and storage nodes, thereby improving the access efficiency and anti-data leakage capabilities of distributed storage.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a data security processing method based on distributed storage, the method comprising: responding to a data storage request, performing multi-dimensional feature parsing processing on the target data to generate a data structure feature vector and an access behavior feature vector of the target data; based on the data structure feature vector and the access behavior feature vector, performing data block segmentation processing on the target data through a dynamic segmentation strategy to generate multiple data blocks and a segmentation attribute label corresponding to each data block; based on the segmentation attribute label and the load status information of the current storage node, performing storage node matching processing on the multiple data blocks to determine the target storage node and the inter-node association routing table corresponding to each data block; calling a pre-built distributed storage security engine to encrypt and encapsulate the inter-node association routing table to generate a route security encapsulation body, and performing distributed write operations on the multiple data blocks according to the target storage node; after the data writing is completed, generating a global data index of the target data based on the segmentation attribute label and the inter-node association routing table, and storing the global data index in an index management node independent of the target storage node. By combining data structure characteristics with access behavior characteristics, the data partitioning strategy is dynamically adjusted to make data segmentation more in line with actual use cases. At the same time, it is matched with the node load status to improve storage balance. The associated routing table is securely encapsulated and the global index is stored independently, which effectively isolates the risk of direct exposure of data storage paths and ensures the confidentiality, integrity and availability of data in a distributed environment.

[0007] As a technical solution of this invention, multi-dimensional feature parsing processing is performed on the target data to generate a data structure feature vector and an access behavior feature vector. Specifically, this includes: performing binary stream parsing processing on the target data to extract file header identification information, data block length distribution information, and data type identification information; combining these three information into a data structure feature vector; extracting the identity information and historical access sequence records of the accessing entity initiating the data storage request; determining the access level parameter of the accessing entity based on the identity information; calculating the access frequency feature value and access time period distribution feature value based on the historical access sequence records; and combining these three information into an access behavior feature vector. Constructing feature vectors from both the data's own structure and the visitor's behavior provides a precise basis for the subsequent dynamic adjustment of the block granularity, improving the rationality and targeting of the block processing.

[0008] Preferably, the method calculates access frequency characteristic values ​​and access time period distribution characteristic values ​​based on historical access time sequence records. Specifically, this includes: extracting the total number of accesses within a preset time window and the timestamp corresponding to each access from the historical access time sequence records; using the ratio of the total number of accesses to the duration of the preset time window as the access frequency characteristic value; and statistically distributing the access counts of all timestamps by hour, using the hourly time period with the highest number of accesses as the access time period distribution characteristic value. This method can accurately quantify the density and peak periods of access, enabling the segmentation strategy to effectively adapt to dynamic changes in access patterns.

[0009] Furthermore, based on the data structure feature vector and access behavior feature vector, the target data is segmented into data blocks using a dynamic segmentation strategy, generating multiple data blocks and corresponding block attribute labels for each data block. This includes: extracting data block length distribution information from the data structure feature vector and determining an initial segmentation granularity benchmark value based on the data block length distribution information; extracting access frequency feature values ​​and access time period distribution feature values ​​from the access behavior feature vector, calculating a dynamic adjustment coefficient based on the access frequency feature values ​​and access time period distribution feature values, and multiplying the initial segmentation granularity benchmark value by the dynamic adjustment coefficient to obtain the actual segmentation granularity value; performing sequential segmentation of the target data according to the actual segmentation granularity value to generate an initial data block sequence; performing cyclic redundancy check calculation on each data block in the initial data block sequence, and appending the check result to the end of the corresponding data block to form multiple data blocks; extracting the block number, block length value, and starting offset for each data block, and combining the block number, block length value, and starting offset to form the block attribute label for the corresponding data block. By adjusting the initial granularity based on the access frequency and time period distribution, high-frequency access data can be distributed and stored with finer granularity, which helps to improve the efficiency of concurrent reading; the additional verification information attached to each block provides support for integrity verification in the subsequent data reading process.

[0010] Preferably, a dynamic adjustment coefficient is calculated based on the access frequency characteristic value and the access time period distribution characteristic value. The actual segmentation granularity value is obtained by multiplying the initial segmentation granularity benchmark value by the dynamic adjustment coefficient. This includes comparing the access frequency characteristic value with a preset high-frequency threshold. When the access frequency characteristic value is greater than the preset high-frequency threshold, the dynamic adjustment coefficient is set to a value greater than one. When the access frequency characteristic value is less than or equal to the preset high-frequency threshold, the dynamic adjustment coefficient is set to a value less than one. Furthermore, the dynamic adjustment coefficient is positively correlated with the peak access concentration in the access time period distribution characteristic value. This process allows high-frequency data to be divided into smaller blocks to distribute read and write pressure, while low-frequency data is stored in larger blocks to reduce metadata overhead, achieving an adaptive balance between storage resources and access efficiency.

[0011] Furthermore, the target storage node and inter-node routing table for each data block are determined, including: sending a load status query request to each storage node in the distributed storage system, receiving the current load status information returned by each storage node, which includes the node's CPU utilization, remaining memory capacity, and remaining network bandwidth; calculating the comprehensive load weight value of each storage node based on the node's CPU utilization, remaining memory capacity, and remaining network bandwidth; determining the required storage capacity for each data block based on its block number and block length, and performing storage capacity matching processing starting from the storage node with the highest comprehensive load weight value, identifying the successfully matched storage node as the target storage node for that data block; after completing the storage node matching for all data blocks, connecting the node addresses of the target storage nodes corresponding to adjacent data blocks in ascending order of block number to generate an inter-node routing table. By introducing multi-dimensional load information such as CPU utilization, memory, and network bandwidth to calculate the comprehensive weight, data blocks are preferentially allocated to nodes with lighter loads, significantly optimizing the overall throughput and response speed of the storage cluster.

[0012] As a preferred embodiment of the present invention, the comprehensive load weight value is calculated as follows: the node's CPU utilization, remaining memory capacity, and remaining network bandwidth are each multiplied by their respective preset weight coefficients and then summed. The preset weight coefficient for remaining network bandwidth is greater than that for remaining memory capacity, and the preset weight coefficient for remaining memory capacity is greater than that for node CPU utilization. This weighting method reflects the decisive impact of network transmission capability on access performance in distributed storage scenarios, ensuring the accuracy of node load assessment.

[0013] Furthermore, the pre-built distributed storage security engine is invoked to encrypt and encapsulate the inter-node association routing table, generating a secure routing encapsulation. Multiple data blocks are then written in a distributed manner to the target storage nodes. This process includes: activating the key derivation module in the distributed storage security engine to generate a session root key based on the access subject's permission level parameters and the global timestamp of the target data; using the first encryption component in the distributed storage security engine, performing symmetric encryption on the inter-node association routing table using the session root key to generate encrypted routing data blocks; using the second encryption component in the distributed storage security engine, performing asymmetric encryption on the session root key using the access subject's public key to generate key-encapsulated data blocks; combining the encrypted routing data blocks and the key-encapsulated data blocks into a secure routing encapsulation, storing the secure routing encapsulation in the secure storage area of ​​the index management node, and writing multiple data blocks to the physical storage media of their respective target storage nodes. This two-layer encryption mechanism—symmetric encryption protecting the routing table itself and asymmetric encryption protecting the session key—ensures the confidentiality of routing information while achieving efficient key distribution and management, preventing side-channel leakage of data storage paths.

[0014] Furthermore, after the data writing is completed, a global data index for the target data is generated based on the block attribute tags and the inter-node association routing table. The global data index is stored in an index management node independent of the target storage node. This includes: extracting the block number and starting offset from the block attribute tags of each data block, and generating a block sequence mapping table by arranging the block numbers in ascending order; extracting the node address of each target storage node from the inter-node association routing table, associating and binding the node address with the corresponding block number, and generating a node address mapping table; merging the block sequence mapping table and the node address mapping table into a global data index, which contains complete routing information required for sequential access from the first block to the last block of the target data; sending an index write request to the index management node, allocating independent index storage space for the target data in the index management node, writing the global data index into the independent index storage space, and recording the correspondence between the access subject's identity information and the independent index storage space in the index management node. By constructing a two-layer index structure that includes sequence mapping and address mapping, and physically isolating the index from the data itself, even if some storage nodes fail, the complete data distribution map can still be quickly located through an independent index management node, ensuring data recoverability and read reliability.

[0015] As a technical solution of this invention, after generating the global data index of the target data, the method further includes: receiving a data read request initiated for the target data; querying the corresponding relationship in the index management node based on the access subject identity information carried in the data read request to obtain the storage location of the global data index; reading the global data index from the index management node according to the storage location; parsing out the block sequence mapping table and the node address mapping table from the global data index; sequentially reading the corresponding data blocks from each target storage node according to the block sequence number indicated by the block sequence mapping table and the node address of the target storage node corresponding to each block sequence number in the node address mapping table; and concatenating and reassembling all the read data blocks according to the block sequence number to restore the complete original data stream of the target data. This process strictly follows the routing order of the index planning for data reassembly, ensuring that the read data is completely consistent with the original written data in logical order and content, and can be efficiently restored without reordering the entire data.

[0016] Preferably, according to the block sequence number indicated by the block sequence mapping table, and combined with the node address of the target storage node corresponding to each block sequence number in the node address mapping table, the corresponding data blocks are read sequentially from each target storage node, including: obtaining the target block sequence number to be read from the block sequence mapping table, and querying the target node address corresponding to the target block sequence number from the node address mapping table; sending a block read instruction to the corresponding target storage node according to the target node address, the block read instruction containing the target block sequence number and the expected value of cyclic redundancy check; receiving the original data block returned by the target storage node and the actual value of cyclic redundancy check appended to the end of the original data block, and comparing the expected value of cyclic redundancy check with the actual value of cyclic redundancy check; if the comparison is consistent, marking the original data block as a valid block and temporarily storing it in the read cache area; if the comparison is inconsistent, resending the block read instruction to the target storage node, and marking the target block as an abnormal block and triggering the data repair process when the number of retries exceeds a preset threshold. By performing cyclic redundancy checks in real time during the reading phase, damaged blocks can be accurately detected and isolated before the data is spliced. Combined with automatic retry and repair mechanisms, the fault tolerance and self-healing ability of data reading are significantly improved.

[0017] As a technical solution of this invention, an index write request is sent to the index management node, an independent index storage space is allocated for the target data in the index management node, and the global data index is written to the independent index storage space. This includes: sending an index write request to the index management node, the index write request containing the data length information of the global data index and the identity information of the accessing subject; receiving the space address and space access token of the independent index storage space returned by the index management node, the validity period of the space access token being associated with the accessing subject's permission level parameter; dividing the global data index into an index header area and an index data area, recording the unique identifier of the target data, the total number of data blocks, and the version number of the global data index in the index header area, and recording a block sequence mapping table and a node address mapping table in the index data area; writing the index header area and the index data area sequentially to the independent index storage space, and sending a write confirmation message to the index management node after the write is completed, the write confirmation message carrying the space address and space access token. By binding the space access token to the permission level, fine-grained permission control of the index write process is achieved, preventing unauthorized entities from tampering with the index structure and ensuring the security of the data location chain.

[0018] As a preferred embodiment of the present invention, after performing distributed write operations on multiple data blocks according to target storage nodes, the method further includes: monitoring the real-time load change of each target storage node; when the real-time load change exceeds a preset load fluctuation threshold, initiating a data block migration evaluation process; calculating the load difference between the source storage node and the target migration node based on the real-time load change; determining the block number and migration data volume of the data block to be migrated based on the load difference; reading the data block to be migrated from the source storage node and writing it to the target migration node; updating the node address information corresponding to the data block to be migrated in the inter-node association routing table after the data writing is completed; sending a route update request to the index management node to update the node address of the corresponding data block to be migrated in the global data index to the node address of the target migration node, and updating the version number of the global data index. This dynamic load balancing and route synchronization update mechanism enables the storage system to automatically adapt to load changes during operation, complete data redistribution without interrupting service, and always maintain balanced pressure on each node and overall high performance of the system.

[0019] This invention also provides a data security processing system based on distributed storage, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements each step of the aforementioned distributed storage data security processing method. This system can integrate and automate the entire process from data feature parsing, dynamic partitioning, secure encapsulation and storage to global index management and dynamic load balancing, constructing an adaptive, highly reliable data security storage architecture with security depth.

[0020] The technical effects and advantages provided by the present invention in the above technical solution are as follows: The initial granularity benchmark value is determined by extracting data block length distribution information from the data structure feature vector. A dynamic adjustment coefficient is calculated by extracting access frequency and access time distribution feature values ​​from the access behavior feature vector. The initial granularity benchmark value is multiplied by the dynamic adjustment coefficient to obtain the actual granularity value, and the target data is then segmented according to this granularity. When the access frequency feature value is greater than a preset high-frequency threshold, the dynamic adjustment coefficient is set to a value greater than 1. Furthermore, the adjustment coefficient is positively correlated with the peak access concentration in the access time distribution feature value. The resulting data blocks have a larger granularity during periods of intensive access and in high-frequency access scenarios, reducing the number of block addressing operations and network round-trip overhead during data reading, and improving sequential read throughput. In low-frequency access scenarios, a smaller granularity segmentation is used, improving the load balancing scheduling accuracy between storage nodes and the storage space utilization rate, and preventing large data blocks from forming resource fragments on low-load nodes.

[0021] The distributed storage security engine's key derivation module generates a session root key based on the access subject's permission level parameters and a global timestamp. The first encryption component uses this session root key to perform symmetric encryption on the inter-node association routing table, generating encrypted routing data blocks. The second encryption component uses the access subject's public key to perform asymmetric encryption on the session root key, generating key-encapsulated data blocks. These are combined into a secure routing encapsulation and stored in a secure storage area of ​​the index management node, independent of the target storage node. The inter-node association routing table itself is not written to each storage node along with the data blocks. Even if an attacker obtains data blocks from one or more storage nodes, they cannot obtain the encrypted routing information stored in the independent index management node. After obtaining the encrypted routing data blocks, the corresponding access subject's private key is still required to decrypt the key-encapsulated data blocks to obtain the session root key. The session root key is bound to permission level parameters and a timestamp, limiting the key's reuse scope and time window. This severs the direct association between routing information and data blocks, constructing a multi-layered encrypted routing isolation system and blocking the technical path of restoring complete data through the routing table. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0023] Figure 1 This is a flowchart of a data security processing method based on distributed storage; Figure 2 This is a flowchart of the data structure feature vector and access behavior feature vector generation process; Figure 3 This is a flowchart of data block processing and storage node matching based on a dynamic block strategy; Figure 4 This is a flowchart of data block distributed writing and global index generation; Figure 5 This is a flowchart of load monitoring and data block migration for a distributed storage system; Figure 6 It is a scatter plot showing the relationship between access frequency characteristic values ​​and dynamic adjustment coefficients. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] See Figure 1 This invention provides a data security processing method based on distributed storage. This method achieves secure data storage and efficient access under a distributed storage architecture through a series of processing steps, including multi-dimensional feature parsing, dynamic block division, storage node matching, secure encapsulation, and index management.

[0026] In response to data storage requests, multi-dimensional feature parsing is performed on the target data to generate data structure feature vectors and access behavior feature vectors. Based on these vectors, the target data is partitioned into data blocks using a dynamic partitioning strategy, generating multiple data blocks and corresponding block attribute labels for each block. Based on the block attribute labels and the current storage node load status information, storage node matching is performed on the multiple data blocks to determine the target storage node and the inter-node routing table for each block. A pre-built distributed storage security engine is invoked to encrypt and encapsulate the inter-node routing table, generating a secure routing encapsulation. Multiple data blocks are then written in a distributed manner according to the target storage nodes. After the data writing is complete, a global data index for the target data is generated based on the block attribute labels and the inter-node routing table. This global data index is stored in an index management node independent of the target storage nodes.

[0027] Example 1

[0028] In specific implementation, please refer to Figure 2 In response to a data storage request, it acquires the data stream of the target data and the identity information of the accessing entity that initiated the data storage request. It then performs multi-dimensional feature parsing on the target data, generating data structure feature vectors and access behavior feature vectors.

[0029] In some embodiments, when performing multi-dimensional feature parsing processing on target data, binary stream parsing is performed first. The binary stream of the target data is read, and a fixed-length file header identifier is extracted from the beginning of the binary stream. The file header identifier is a byte sequence truncated to a preset byte length, which reflects the standard header signature of the file format to which the target data belongs. Further parsing of the data block length distribution information of the target data from the binary stream is performed. In the specific processing, according to the file format specification corresponding to the target data, the delimiter markers used to separate data blocks or the length field recording the data block length in the binary stream are identified. By traversing the binary stream, the start and end positions of each data block are recorded sequentially, and the length value of each data block is calculated. The set of all data block length values ​​is used as the data block length distribution information. If the target data does not have a clear data block delimiter structure, the target data is considered a single data block, and the total length of the target data is used as the unique element of the data block length distribution information. Simultaneously, based on the format magic number contained in the file header identifier of the binary stream or based on the extension information of the target data, the data type identifier information of the target data is determined. The data type identifier information is a predefined enumeration value used to distinguish between structured data, unstructured data, or semi-structured data. The file header identifier, data block length distribution information, and data type identifier are combined into a data structure feature vector. The combination method is to encode the file header identifier into a fixed-length binary sequence, the data block length distribution information into an array of length values, and the data type identifier into an integer enumeration value. The three are then concatenated in order to form the data structure feature vector.

[0030] During the generation of access behavior feature vectors, the identity information of the accessing entity is extracted from the data storage request. Based on this identity information, the corresponding permission level parameter is queried from the permission management database. The permission level parameter is a pre-defined multi-level value representing the accessing entity's operational permission level in the distributed storage system. Simultaneously, the historical access time sequence records of the accessing entity are extracted and stored in the access log system, containing the timestamps corresponding to each access operation by the accessing entity within a preset time window. The preset time window is set to a fixed duration tracing back from the current time, such as 720 hours.

[0031] The access frequency characteristic value is calculated based on historical access time sequence records. The calculation process is as follows: The total number of access records within a preset time window is counted from the historical access time sequence records. This total number of accesses is denoted as... The preset time window duration is Then the access frequency characteristic value Calculate using the following formula:

[0032] in, This represents the cumulative number of all access operations initiated by the accessing entity within a preset time window. This indicates the duration of the preset time window, in hours. Access frequency characteristic value. The unit of measurement is times / hour, reflecting the average frequency of visits by the user.

[0033] When calculating the access time period distribution characteristic value, the timestamp corresponding to each access is extracted from the historical access time series records. These timestamps are then grouped by hour, and the number of accesses within each hourly time period is counted. Specifically, a day is divided into 24-hour time periods, and each hourly time period is represented as... , The value of is an integer ranging from 0 to 23, and it is used to count the historical access time sequence records that fall within each Number of visits within a time period Compare all The value determines the hourly time period with the highest number of visits. ,Will As a characteristic value of the access time period distribution.

[0034] Access level parameters and access frequency characteristics and access time distribution characteristics The combination forms an access behavior feature vector. The combination method is to encode the permission level parameter as an integer value, the access frequency feature value as a floating-point value, and the access time period distribution feature value as an integer value, and arrange the three in order to form the access behavior feature vector.

[0035] Example 2

[0036] In specific implementation, please refer to Figure 3 Based on the data structure feature vector and access behavior feature vector, the target data is divided into data blocks using a dynamic block partitioning strategy, generating multiple data blocks and corresponding block attribute labels for each data block.

[0037] Data block length distribution information is extracted from the data structure feature vector. The data structure feature vector contains three parts: file header identification information, data block length distribution information, and data type identification information. The data block length distribution information is an array of length values, where each element corresponds to the length of a data block within the target data. When determining the initial segmentation granularity benchmark based on the data block length distribution information, the arithmetic mean of all length values ​​in the length value array is calculated, and this arithmetic mean is used as the initial segmentation granularity benchmark. If the length value array is empty or the target data does not have a clearly defined data block separation structure, then the total length of the target data is used as the initial segmentation granularity benchmark.

[0038] The access behavior feature vector extracts access frequency and access time period distribution features. The access behavior feature vector comprises three parts: permission level parameters, access frequency features, and access time period distribution features. The access frequency feature reflects the average access intensity of the accessing entity, while the access time period distribution feature reflects the hourly time period during which the accessing entity historically accessed the most frequently.

[0039] The dynamic adjustment coefficient is calculated based on the access frequency characteristic value and the access time period distribution characteristic value. The actual segmentation granularity value is obtained by multiplying the initial segmentation granularity benchmark value by the dynamic adjustment coefficient.

[0040] In some embodiments, the dynamic adjustment coefficient is calculated as follows: The access frequency characteristic value is compared with a preset high-frequency threshold. When the access frequency characteristic value is greater than the preset high-frequency threshold, the dynamic adjustment coefficient is set to a value greater than 1; when the access frequency characteristic value is less than or equal to the preset high-frequency threshold, the dynamic adjustment coefficient is set to a value less than 1. Furthermore, the dynamic adjustment coefficient is positively correlated with the peak access concentration in the access time period distribution characteristic value. Peak access concentration represents the proportion of visits by the accessing subject within the peak hour period indicated by the access time period distribution characteristic value to the total number of visits within a preset time window. Dynamic adjustment coefficient Calculate using the following formula:

[0041] in, This represents the access frequency feature value extracted from the access behavior feature vector. The access frequency feature value is the average number of times the accessing subject accesses the device per hour within a preset time window. This indicates a preset high-frequency threshold, which is a fixed reference value obtained based on the historical access load statistics of the distributed storage system. The basis for setting it is to ensure that the access frequency characteristic value of the top 30% of access subjects in the system falls within the range greater than this threshold. This indicates the peak access concentration. The peak access concentration is calculated as the ratio of the number of visits within the peak hour period indicated by the access time distribution characteristic value to the total number of visits within the preset time window. This represents the concentration influence factor, which ranges from 0 to 1, and is set to 0.5. When... hour, A value greater than 1 makes the dynamic adjustment coefficient... Greater than 1; when When the ratio is less than or equal to 1, the dynamic adjustment coefficient is adjusted. The corresponding value is less than or equal to 1; meanwhile, the peak access concentration is... Increasing will make Increase, dynamic adjustment coefficient The value increases accordingly, demonstrating a positive correlation.

[0042] The initial granularity benchmark value and the calculated dynamic adjustment coefficient are used. Multiplying these values ​​yields the actual granularity of the data segmentation. The unit of the actual granularity is bytes, representing the baseline length of each data block when segmenting the target data.

[0043] The target data is sequentially segmented according to the actual segmentation granularity value to generate an initial data block sequence. Starting from the beginning of the binary stream of the target data, a data segment of length equal to the actual segmentation granularity value is read each time, and this is sequentially segmented into multiple consecutive data blocks. The length of the last data block may be smaller than the actual segmentation granularity value; in this case, the actual length of the remaining data is used as the length of that data block. After segmentation, the initial data block sequence is obtained, which contains several data blocks arranged in the original data order.

[0044] Cyclic redundancy check (CRC) calculations are performed on each data block in the initial data block sequence. The CRC result is appended to the end of the corresponding data block to form multiple data blocks. The CRC calculation uses a preset CRC polynomial to calculate all bytes of the data block, generating a fixed-length check value. This check value is appended to the end of the data block as the CRC result, increasing the length of each data block. This process generates a set of data blocks comprising multiple data blocks.

[0045] For each data block, extract the block number, block length, and starting offset. The block number is the number of the data block in the splitting order, starting from 1 and incrementing; the block length is the original data length after splitting the data block, excluding the length of the additional cyclic redundancy check result; the starting offset is the offset of the starting byte position of the data block in the target data relative to the starting position of the target data. Combine the block number, block length, and starting offset to form the block attribute label for the corresponding data block. The combination method is to concatenate the three values ​​in sequence to form structured label data.

[0046] Based on the block attribute tags and the current storage node load status information, storage node matching is performed on multiple data blocks to determine the target storage node and the inter-node routing table corresponding to each data block. A load status query request is sent to each storage node in the distributed storage system via network messages to the status listening port of each storage node. The system receives the current load status information returned by each storage node, which includes the node's CPU utilization, remaining memory capacity, and remaining network bandwidth. The CPU utilization is expressed as a percentage, the remaining memory capacity is in megabytes, and the remaining network bandwidth is in megabits per second.

[0047] The overall load weight value for each storage node is calculated based on the node's CPU utilization, remaining memory capacity, and remaining network bandwidth. The overall load weight value is calculated by multiplying each of these factors by their respective preset weight coefficients and then summing the results. The preset weight coefficient for remaining network bandwidth is greater than that for remaining memory capacity, and the preset weight coefficient for remaining memory capacity is greater than that for CPU utilization. These preset weight coefficients are parameters initially configured in the distributed storage system, set to 0.5 for remaining network bandwidth, 0.3 for remaining memory capacity, and 0.2 for CPU utilization.

[0048] The required storage capacity for each data block is determined based on its block sequence number and block length value in the block attribute label. The required storage capacity equals the block length value plus the fixed length of the cyclic redundancy check result. Storage capacity matching is performed starting with the storage node with the highest overall load weight value. Candidate storage nodes are sorted from highest to lowest according to their overall load weight value. The remaining memory capacity of each candidate storage node is checked sequentially to see if it is greater than or equal to the required storage capacity of the data block. The first storage node that meets the condition is determined as the target storage node for that data block. If the remaining memory capacity of all storage nodes is insufficient, the data storage request is rejected and a "storage space insufficient" response is returned.

[0049] After matching storage nodes for all data blocks, the node addresses of the target storage nodes corresponding to adjacent data blocks are connected in ascending order of block sequence number to generate an inter-node association routing table. The node address is the network address identifier of the storage node in the distributed storage system. For each block sequence number, the node address of its target storage node is obtained; starting from block sequence number 1, the node address corresponding to the current block sequence number is recorded in an entry of the inter-node association routing table. The entry contains the correspondence between block sequence number and node address, and adjacent entries are arranged sequentially according to block sequence number. The inter-node association routing table records the sequence of node addresses corresponding to all data blocks in list form, reflecting the storage location and access order of each data block in the distributed storage system.

[0050] See Figure 6 The horizontal axis in the graph represents the access frequency characteristic value. The unit is times per hour, and the vertical axis represents the dynamic adjustment coefficient Ω. The data points in the graph are based on peak access concentration. Divided into three categories, each using hollow circles ( ∈[0.00,0.33)), solid cube ( ∈[0.33,0.66)) and solid rhombus ( The graph (∈[0.66,1.00]) clearly distinguishes three peak access concentration intervals.

[0051] Overall, with the characteristic value of access frequency As the value increases, the dynamic adjustment coefficient Ω shows a clear increasing trend. In the lower range of 0 to 10 times / hour, the Ω value is mainly concentrated between 0.2 and 0.8, and varies. The data points in the intervals overlap considerably, indicating that the dynamic adjustment coefficient has a relatively small impact on users with low-frequency access.

[0052] when Exceeding the preset high-frequency threshold (Approximately 10 times / hour in the figure) After this, the dynamic adjustment coefficient Ω increases significantly, with some data points showing Ω values ​​reaching or even exceeding 1.6, indicating a substantial amplification effect. At this point, the peak access concentration... The effect on Ω is even more significant: Higher (solid rhombus) data points generally correspond to higher Ω values, reflecting the relationship between Ω and... They show a positive correlation, and the adjustment effect of the concentration influence factor η being set to 0.5 is significant.

[0053] In addition, In the higher range (10 to 30 times / hour), the fluctuation of the dynamic adjustment coefficient Ω increased. Some data points with low peak access concentration (hollow circles) but high access frequency also showed higher Ω values, but were generally lower than those with high concentration, further verifying the calculation logic of the dynamic adjustment coefficient.

[0054] Example 3

[0055] In specific implementation, please refer to Figure 4 The overall load weight value is calculated based on pre-set weighting coefficients and collected load status information. The current load status information includes node CPU utilization, remaining memory capacity, and remaining network bandwidth, each reflecting the storage node's load status from different resource dimensions. The node CPU utilization is denoted as... The remaining memory capacity is expressed as a percentage; The remaining network bandwidth is expressed in megabytes; , expressed in megabits per second.

[0056] Overall load weight value The calculation formula is:

[0057] in, This represents the CPU utilization rate of a node, and its value ranges from 0 to 100. A higher value indicates a higher CPU usage level. Convert processor utilization rate to processor idle rate for calculation; This indicates the remaining memory capacity, measured in megabytes. A larger value indicates that the storage node currently has more memory space available for allocation. This indicates the remaining network bandwidth, measured in megabits per second. A larger value indicates a stronger network transmission capacity currently available to the storage node. This represents the weighting coefficient of the central processing unit. This represents the memory weighting coefficient. This represents the network bandwidth weighting coefficients. The three weighting coefficients satisfy... The size relationship, and Central Processing Unit Weighting Coefficient Set to 0.2, memory weight coefficient Set to 0.3, network bandwidth weighting coefficient The value is set to 0.5, based on the fact that in distributed storage scenarios, network transmission latency has the greatest impact on overall performance for data read / write operations, followed by memory capacity affecting data cache hit rate, while CPU utilization has a relatively smaller impact on storage node response speed. (Comprehensive load weight value) A higher value indicates a lighter overall load on the storage node and more available resources.

[0058] In some embodiments, a pre-built distributed storage security engine is invoked to encrypt and encapsulate the inter-node association routing table, generating a secure routing encapsulation. Multiple data blocks are then written in a distributed manner according to the target storage nodes. The key derivation module in the distributed storage security engine is activated. This module receives two input parameters: the access subject's permission level parameter and the global timestamp of the target data. The permission level parameter is obtained from the access behavior feature vector, which has been generated and stored during the multi-dimensional feature parsing process of the target data. The global timestamp of the target data is the system time value when the data storage request arrives at the distributed storage system, accurate to milliseconds. The key derivation module uses a hash-based message authentication code algorithm, with the permission level parameter as the message input and the global timestamp as the key seed, to generate a fixed-length session root key through iterative hash operations. The session root key is 256 bits long.

[0059] The first encryption component in the distributed storage security engine performs symmetric encryption on the inter-node association routing table using the session root key, generating encrypted routing data blocks. The first encryption component employs an Advanced Encryption Standard (AES) algorithm, operating in Galois counter mode. The session root key serves as the encryption key for symmetric encryption, and the complete binary sequence of the inter-node association routing table is used as the plaintext to be encrypted. The encryption process generates ciphertext data and an authentication tag, which is appended to the end of the ciphertext data to form the encrypted routing data block.

[0060] The second encryption component in the distributed storage security engine performs asymmetric encryption on the session root key using the access subject's public key, generating a key-encapsulated data block. This second encryption component employs an elliptic curve integration (ECI) encryption scheme, using the access subject's public key as the encryption key and the session root key as the plaintext to be encrypted, performing elliptic curve encryption operations and outputting the key-encapsulated data block. The access subject's public key is pre-stored in the distributed storage system's key management service and can be retrieved by querying the access subject's identity information.

[0061] The route encryption data block and the key-encapsulated data block are combined into a route security package. The combination involves writing a fixed-length separator and a length field at the header of the route security package. The separator distinguishes the boundary between the route encryption data block and the key-encapsulated data block, and the length field records the lengths of both the route encryption data block and the key-encapsulated data block. Then, all bytes of the route encryption data block and the key-encapsulated data block are concatenated sequentially. The route security package is stored in the secure storage area of ​​the index management node. This secure storage area is a storage partition on the index management node with independent access control policies, accessible only to the distributed storage security engine and authorized index management processes. Multiple data blocks are then written separately to the physical storage media of their respective target storage nodes. The physical storage media can be solid-state drives (SSDs) or hard disk drives (HDDs). The writing method is to write the data blocks sequentially according to their binary stream order, using a pre-allocated data storage directory on the target storage node.

[0062] After data writing is complete, a global data index for the target data is generated based on the block attribute tags and the inter-node routing table. The block number and starting offset are extracted from the block attribute tags of each data block. The block attribute tags are structured tag data containing three fields: block number, block length, and starting offset. The block number and starting offset fields are parsed from the block attribute tags of each data block. All extracted block numbers and starting offsets are sorted in ascending order of block number to generate a block sequence mapping table. Each entry in the block sequence mapping table records the block number and starting offset of a data block, with entries arranged sequentially in ascending order of block number.

[0063] The node address of each target storage node is extracted from the inter-node association routing table. The inter-node association routing table contains multiple sets of entries, each recording the correspondence between block sequence numbers and node addresses. All block sequence numbers and node addresses in the inter-node association routing table are read, and the node addresses are associated and bound with their corresponding block sequence numbers to generate a node address mapping table. Each entry in the node address mapping table records a block sequence number and its corresponding node address.

[0064] The block sequence mapping table and the node address mapping table are merged into a global data index. The merging method involves setting an index header area at the beginning of the global data index, followed by an index data area. The index header area records the metadata information of the global data index, while the index data area sequentially records the complete contents of both the block sequence mapping table and the node address mapping table. The global data index contains complete routing information required to access the target data from the first block to the last. This complete routing information is represented by a one-to-one mapping between the starting offset in the block sequence mapping table and the node address in the node address mapping table, using the block sequence number.

[0065] A write request is sent to the index management node. The index management node allocates independent index storage space for the target data and writes the global data index to this independent index storage space. The write request includes the data length information of the global data index and the identity information of the accessing subject. Upon receiving the write request, the index management node partitions a contiguous storage area with matching capacity in its own storage medium based on the data length information, generating a space address and space access token for this contiguous storage area. The system then receives the space address and space access token for the independent index storage space returned by the index management node. The validity period of the space access token is related to the accessing subject's permission level parameter; the higher the permission level parameter, the longer the validity period of the space access token.

[0066] The global data index is divided into an index header area and an index data area. The index header area records the unique identifier of the target data, which is a globally unique identifier assigned to the target data during the data storage request processing. The index header area also records the total number of data blocks, which is the total number of data blocks generated after the splitting process. The index header area also records the version number of the global data index, initially set to 1. The index data area records the block sequence mapping table and the node address mapping table. The index header area and index data area are written sequentially to an independent index storage space, with the write starting at the storage offset indicated by the space address. After writing is complete, a write confirmation message is sent to the index management node, carrying the space address and space access token.

[0067] The index management node records the correspondence between the access subject's identity information and the independent index storage space. The correspondence is recorded by inserting a record into the metadata management table of the index management node. This record contains the access subject's identity information, the space address of the independent index storage space, the space access token, and the creation timestamp.

[0068] Example 4

[0069] In practice, after generating a global data index for the target data, the distributed storage system receives data read requests initiated by the accessing entity. These requests carry the accessing entity's identity information and a unique identifier for the target data. Based on the accessing entity's identity information carried in the data read request, the system queries the mapping between the accessing entity's identity information and the independent index storage space recorded in the index management node to obtain the storage location of the global data index. The index management node maintains a metadata management table. Each record in the metadata management table contains the accessing entity's identity information, the space address of the independent index storage space, the space access token, and the creation timestamp. During a query, the accessing entity's identity information is used as the search keyword to match the corresponding record in the metadata management table. The space address of the independent index storage space in that record is then extracted; this space address is the storage location of the global data index.

[0070] The global data index is read from the index management node based on its storage location. An index read request is sent to the index management node, containing the space address and space access token of the global data index's storage location. The index management node verifies the validity of the space access token. If the verification is successful, it reads all the binary data of the global data index from the independent index storage space corresponding to the space address and returns the global data index.

[0071] The block sequence map table and node address map table are parsed from the read global data index. The global data index consists of an index header area and an index data area. During parsing, the index header area is read first to obtain the unique identifier of the target data, the total number of data blocks, and the version number of the global data index. Based on the format information recorded in the index header area, the starting offset and length of the block sequence map table in the index data area are determined. The complete content of the block sequence map table is read. The block sequence map table contains multiple entries, each recording the block number and the starting offset of a data block. The complete content of the node address map table is then read. The node address map table contains multiple entries, each recording a block number and the node address of the target storage node corresponding to that block number.

[0072] According to the block sequence number indicated by the block sequence mapping table, and combined with the node address of the target storage node corresponding to each block number in the node address mapping table, the corresponding data blocks are read from each target storage node in sequence.

[0073] The target block number to be read is obtained from the block sequence mapping table. The target block numbers are selected sequentially in ascending order according to the entries in the block sequence mapping table, starting from block number 1 until the block number equals the total number of data blocks. The target node address corresponding to the target block number is retrieved from the node address mapping table. The corresponding entry in the node address mapping table is searched using the block number as the index key, and the node address of the target storage node recorded in that entry is extracted.

[0074] A block read command is sent to the corresponding target storage node based on the target node address. The block read command includes the target block sequence number and the expected cyclic redundancy check (CRC) value. The target block sequence number identifies a specific data block stored on the target storage node. The expected CRC value is the verification result generated when performing CRC calculations on this data block during the data write phase. The expected CRC value is obtained from the block attribute tag or the data block metadata. Specifically, during the data read preparation phase, the block attribute tag corresponding to the target data's unique identifier and the target block sequence number is queried from the distributed storage system's metadata storage, and the recorded CRC result is extracted as the expected CRC value.

[0075] The system receives the original data block and the actual cyclic redundancy check (CRC) value appended to the end of the original data block from the target storage node. Upon receiving the block read instruction, the target storage node locates the corresponding data block in its physical storage medium based on the target block sequence number, reads all bytes of the original data block and the CRC value appended to the end, and returns the original data block and the CRC value as response data.

[0076] The expected value of the Cyclic Redundancy Check (CRC) is compared with the actual value. The comparison method is to compare the binary content of the two check values ​​byte by byte.

[0077] If the comparison is successful, the original data block is marked as a valid block and temporarily stored in the read cache. The read cache is a memory buffer allocated during the data read request processing to temporarily store valid blocks read from each target storage node. The total capacity of the read cache is greater than or equal to the total length of all data blocks.

[0078] In case of discrepancies, the block read command is resent to the target storage node. The resend is sent after a fixed interval of 100 milliseconds. If the number of retries exceeds a preset threshold, the target block is marked as an abnormal block, triggering a data repair process. The preset threshold is set to 3 retries, based on the premise that network jitter or storage node load spikes in a distributed storage system typically recover within 3 retries; exceeding 3 retries indicates a persistent fault. The data repair process involves sending an abnormal block notification to the erasure coding repair module of the distributed storage system. The erasure coding repair module then reconstructs the content of the abnormal block based on redundancy check data and rewrites it to the target storage node or migrates it to another storage node.

[0079] After all data blocks have been read and marked as valid blocks according to the block sequence mapping table, the read data blocks are concatenated and reassembled according to their block numbers. The concatenation and reassembly process retrieves each valid block from the read buffer in ascending order of block number, removes the cyclic redundancy check value appended to the end of each valid block, and then concatenates the original data blocks (after removing the check value) byte-by-byte according to their block numbers to obtain a continuous and complete binary data stream. This complete binary data stream is the complete original data stream of the target data to be restored.

[0080] Example 5

[0081] See Figure 5 After dividing multiple data blocks and performing distributed write operations on target storage nodes, the distributed storage system starts a load monitoring process to monitor the real-time load changes of each target storage node. The load monitoring process sends load status query requests to each target storage node at a fixed collection period of 30 seconds. It receives real-time load status information returned by each target storage node, including the node's CPU utilization, remaining memory capacity, and remaining network bandwidth. The CPU utilization obtained in the current collection period is recorded as... The remaining memory capacity is denoted as The remaining network bandwidth is denoted as .according to , , Calculate the real-time comprehensive load index value of the target storage node. The calculation method is to... Converted to processor idle rate and remaining memory capacity Remaining network bandwidth Multiply each value by a preset resource weighting coefficient and sum them. The resource weighting coefficients are set in the same way as those used in the calculation of the comprehensive load weight value mentioned earlier. Then, calculate the real-time comprehensive load index value obtained in the current data collection period. The real-time comprehensive load index value calculated with the previous collection cycle Compare and calculate real-time load changes. The real-time load change is When the real-time load changes When the load fluctuation threshold is exceeded, the data block migration assessment process is initiated. The preset load fluctuation threshold is set to the real-time comprehensive load index value. The 20% threshold is set based on the premise that the performance of distributed storage nodes can remain stable within the normal load fluctuation range. A load change exceeding 20% ​​indicates a significant change in the resource status of the storage nodes, requiring load adjustment through data migration.

[0082] After the data block migration assessment process is initiated, the load difference between the source storage node and the target migration node is calculated based on the real-time load change. The source storage node is the target storage node whose real-time load change triggers the migration assessment process and exceeds a preset load fluctuation threshold. A set of candidate migration nodes is selected from the remaining storage nodes in the distributed storage system (excluding the source storage node). The selection criterion is that the remaining memory capacity of the candidate migration node is greater than the minimum data block length stored on the source storage node. For each candidate migration node in the candidate migration node set, its real-time comprehensive load index value for the current collection period is obtained. Calculate the load difference between the source storage node and each candidate migration node. The calculation formula is:

[0083] in, This represents the real-time comprehensive load index value of the source storage node in the current collection period. This represents the real-time comprehensive load index value of the candidate migration node in the current data collection period. Load Difference A positive value indicates that the load on the source storage node is higher than the load on the candidate migration node. A larger value indicates a more significant improvement in load balancing achieved by migrating data blocks from the source storage node to the candidate migration node. (Select...) The candidate migration node corresponding to the maximum value is used as the target migration node. If there are multiple candidate migration nodes... If both are the maximum values, then select the remaining network bandwidth. The largest candidate migration node is selected as the target migration node.

[0084] The block number and migration volume of the data to be migrated are determined based on the load difference. The migration volume is determined by calculating the base value of the migration volume. ,in This represents the load difference between the source storage node and the selected target migration node. This represents the migration conversion factor, set at 50 megabytes per unit load difference. This factor is based on the principle that the amount of data in a single migration operation should be sufficient to offset the load differences between nodes, while avoiding excessive data migration that could cause an overload on the target migration node. On the source storage node, all data blocks that use the source storage node as the target storage node are located, and the block length value from the block attribute tag of each data block is obtained. The data blocks on the source storage node are arranged in descending order of their block length values, forming a candidate block list. Data blocks are selected sequentially from the candidate block list, and the block length values ​​of the selected data blocks are accumulated. When the accumulated value is greater than or equal to the base amount of the migrated data, the migration is considered complete. When the selection stops, the set of block numbers corresponding to the selected data blocks is used as the block number of the data blocks to be migrated, and the accumulated value is determined as the actual amount of data to be migrated.

[0085] The system reads data blocks to be migrated from the source storage node and writes them to the target migration node. Based on the block sequence number of the data blocks to be migrated, a block read command is sent to the source storage node. This command includes the block sequence number and the corresponding expected cyclic redundancy check (CRC) value. The system receives the original data blocks and the actual CRC value returned by the source storage node, performs a CRC check comparison, and if they match, temporarily stores the original data blocks in the migration buffer. The system writes the original data blocks in the migration buffer to the physical storage medium of the target migration node in their original byte order, using the pre-allocated data storage directory on the target migration node as the write path. After writing is complete, the system receives a write success confirmation from the target migration node.

[0086] After the data is successfully written to the target migration node, the node address information corresponding to the data block to be migrated in the inter-node association routing table is updated. All entries in the inter-node association routing table are traversed. For entries where the block sequence number matches the block sequence number of the data block to be migrated, the node address field in that entry is replaced with the node address of the target migration node, thus completing the update of the inter-node association routing table.

[0087] A route update request is sent to the index management node. This request includes the unique identifier of the target data, the set of block sequence numbers for the data blocks to be migrated, and the node address of the target migration node. The index management node locates the independent index storage space based on the unique identifier of the target data, reads the global data index from the independent index storage space, and parses the node address mapping table. In the node address mapping table, the node address field in the entry corresponding to the block sequence number of the data blocks to be migrated is updated to the node address of the target migration node. The updated node address mapping table is merged back with the block sequence mapping table to generate an updated global data index. The version number of the global data index is incremented by 1, and the new version number is recorded in the index header area. The updated global data index is rewritten to the independent index storage space, overwriting the original global data index. After writing is complete, a migration completion confirmation is sent to the source storage node, and the source storage node releases the storage space occupied by the migrated data blocks.

[0088] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A data security processing method based on distributed storage, characterized in that, The method includes: In response to a data storage request, multi-dimensional feature parsing processing is performed on the target data to generate a data structure feature vector and an access behavior feature vector for the target data. Based on the data structure feature vector and the access behavior feature vector, the target data is divided into data blocks using a dynamic block partitioning strategy to generate multiple data blocks and block attribute labels corresponding to each data block. Based on the block attribute tags and the load status information of the current storage node, storage node matching processing is performed on the multiple data blocks to determine the target storage node and the inter-node routing table corresponding to each data block. The pre-built distributed storage security engine is invoked to encrypt and encapsulate the routing table associated with the nodes, generating a route security encapsulation body, and the multiple data blocks are divided and written in a distributed manner according to the target storage node; After the data is written, a global data index for the target data is generated based on the block attribute label and the inter-node association routing table, and the global data index is stored in an index management node that is independent of the target storage node.

2. The data security processing method based on distributed storage according to claim 1, characterized in that, The step of performing multi-dimensional feature parsing processing on the target data to generate data structure feature vectors and access behavior feature vectors for the target data includes: Perform binary stream parsing processing on the target data to extract the file header identification information, data block length distribution information and data type identification information of the target data, and combine the file header identification information, the data block length distribution information and the data type identification information into the data structure feature vector; Extract the identity information and historical access sequence records of the access subject that initiated the data storage request. Determine the access level parameter of the access subject based on the identity information. Calculate the access frequency feature value and access time period distribution feature value based on the historical access sequence records. Combine the access level parameter, the access frequency feature value, and the access time period distribution feature value into the access behavior feature vector.

3. The data security processing method based on distributed storage according to claim 2, characterized in that, Based on the historical access time sequence records, the access frequency feature value and access time period distribution feature value are calculated, including: Extract the total number of visits within a preset time window and the timestamp corresponding to each visit from the historical access time sequence records. Use the ratio of the total number of visits to the duration of the preset time window as the access frequency feature value. Statistically distribute the number of visits by hour for all timestamps and use the hour period with the highest number of visits as the access time period distribution feature value.

4. The data security processing method based on distributed storage according to claim 2, characterized in that, Based on the data structure feature vector and the access behavior feature vector, the target data is segmented into data blocks using a dynamic segmentation strategy, generating multiple data blocks and corresponding block attribute labels for each data block, including: Extract the data block length distribution information from the data structure feature vector, and determine the initial segmentation granularity benchmark value based on the data block length distribution information; The access frequency feature value and the access time period distribution feature value are extracted from the access behavior feature vector. A dynamic adjustment coefficient is calculated based on the access frequency feature value and the access time period distribution feature value. The actual segmentation granularity value is obtained by multiplying the initial segmentation granularity benchmark value with the dynamic adjustment coefficient. The target data is sequentially segmented according to the actual segmentation granularity value to generate an initial data block sequence. Cyclic redundancy check calculation is performed on each data block in the initial data block sequence, and the check result is appended to the end of the corresponding data block to form the multiple data blocks. For each data block, extract the block number, block length value, and starting offset, and combine the block number, block length value, and starting offset to form the block attribute label of the corresponding data block.

5. A data security processing method based on distributed storage according to claim 4, characterized in that, A dynamic adjustment coefficient is calculated based on the access frequency characteristic value and the access time period distribution characteristic value. The actual segmentation granularity value is obtained by multiplying the initial segmentation granularity benchmark value by the dynamic adjustment coefficient, including: The access frequency feature value is compared with a preset high-frequency threshold. When the access frequency feature value is greater than the preset high-frequency threshold, the dynamic adjustment coefficient is set to a value greater than 1. When the access frequency feature value is less than or equal to the preset high-frequency threshold, the dynamic adjustment coefficient is set to a value less than 1. The dynamic adjustment coefficient is positively correlated with the peak access concentration in the access time period distribution feature value.

6. A data security processing method based on distributed storage according to claim 4, characterized in that, Determine the target storage node and the routing table between nodes corresponding to each data block, including: Send a load status query request to each storage node in the distributed storage system, and receive the current load status information returned by each storage node. The current load status information includes the node's central processing unit utilization, remaining memory capacity, and remaining network bandwidth. The comprehensive load weight value of each storage node is calculated based on the node's central processing unit utilization, remaining memory capacity, and remaining network bandwidth. The required storage capacity for each data block is determined based on the block number and the block length value. Storage capacity matching is performed starting from the storage node with the highest comprehensive load weight value, and the successfully matched storage node is determined as the target storage node for the data block. After matching the storage nodes for all data blocks, the node addresses of the target storage nodes corresponding to adjacent data blocks are connected in ascending order of the block number to generate the inter-node association routing table.

7. A data security processing method based on distributed storage as described in claim 6, characterized in that, The comprehensive load weight value is calculated by multiplying the node CPU utilization rate, the remaining memory capacity, and the remaining network bandwidth by their respective preset weight coefficients and then summing them up. The preset weight coefficient corresponding to the remaining network bandwidth is greater than the preset weight coefficient corresponding to the remaining memory capacity, and the preset weight coefficient corresponding to the remaining memory capacity is greater than the preset weight coefficient corresponding to the node CPU utilization rate.

8. A data security processing method based on distributed storage according to claim 6, characterized in that, The process of invoking a pre-built distributed storage security engine to encrypt and encapsulate the inter-node association routing table, generating a secure routing encapsulation, and then performing distributed write operations on the multiple data blocks according to the target storage nodes includes: The key derivation module in the distributed storage security engine is activated to generate a session root key based on the access subject's permission level parameter and the global timestamp of the target data. The first encryption component in the distributed storage security engine uses the session root key to perform symmetric encryption on the inter-node associated routing table to generate a route encrypted data block. The second encryption component in the distributed storage security engine uses the public key of the access subject to perform asymmetric encryption on the session root key to generate a key-encapsulated data block. The route encryption data block and the key encapsulation data block are combined into the route security encapsulation body, the route security encapsulation body is stored in the secure storage area of ​​the index management node, and the multiple data blocks are written into the physical storage medium of their respective target storage nodes.

9. A data security processing method based on distributed storage according to claim 8, characterized in that, After the data writing is completed, a global data index for the target data is generated based on the block attribute tags and the inter-node association routing table, and the global data index is stored in an index management node independent of the target storage node, including: Extract the block number and the starting offset from the block attribute label of each data block, and generate a block sequence mapping table according to the ascending order of the block number; Extract the node address of each target storage node from the inter-node association routing table, associate and bind the node address with the corresponding block sequence number, and generate a node address mapping table; The block sequence mapping table and the node address mapping table are merged into the global data index, which contains complete routing information required to access the target data sequentially from the first block to the last block. Send an index write request to the index management node, allocate an independent index storage space for the target data in the index management node, write the global data index into the independent index storage space, and record the correspondence between the identity information of the access subject and the independent index storage space in the index management node.

10. A data security processing system based on distributed storage, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the data security processing method based on distributed storage as described in any one of claims 1 to 9.

Images