A data security storage and privacy protection method for high concurrency scenarios

Abstract

The application relates to the technical field of data management, and discloses a data security storage and privacy protection method for a high-concurrency scene, which is used for solving the problem that the running state of a storage node may suddenly change when data is stored, leading to unreasonable selection of the storage node for data storage. The method comprises the following steps: selecting a storage node according to the load state of a current storage system; obtaining the running parameters of the selected storage node; calculating an instantaneous distortion index according to the running parameters of the storage node; determining whether the selected storage node has instantaneous fluctuation according to the instantaneous distortion index; if it is determined that the selected storage node has instantaneous fluctuation, the storage node is reselected; and if it is determined that the selected storage node does not have instantaneous fluctuation, the selected storage node is taken as a target storage node, so that the reliability and practicability of the data security storage system in a high-concurrency scene are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and more specifically to a method for secure data storage and privacy protection in high-concurrency scenarios. Background Technology

[0002] With the widespread application of cloud computing, big data, and distributed system technologies in internet services, industrial informatization, and digital business, data storage and management for high-concurrency scenarios have gradually become a crucial foundation for the stable operation of information systems. Among these, the secure storage and controlled access of data under high-concurrency conditions, as an important component of data governance, directly impacts the continuous operation of business systems and the effectiveness of data privacy protection through storage reliability and access consistency. To meet the demands of rapid writing and high-frequency access to massive amounts of data, distributed storage architectures are widely used in high-concurrency data processing scenarios. By having multiple storage nodes work collaboratively to improve the overall throughput and scalability of the system, this has become a common technical approach in current data infrastructure construction.

[0003] Existing secure data storage systems typically employ a multi-node distributed architecture, uniformly accessing data storage requests from different data sources and distributing the data to different storage nodes for write operations. During this process, storage nodes are generally selected based on their operational status information to achieve load balancing and optimized storage performance. Simultaneously, appropriate access control policies and privacy protection rules are configured for the stored data. During the data access phase, identity verification and authorization checks restrict the scope of data access for different entities, thus balancing data storage efficiency and data security requirements under high concurrency conditions.

[0004] However, the above-mentioned technologies have at least the following technical problems:

[0005] However, in real-world high-concurrency environments, the operating status of storage nodes often exhibits significant dynamic changes, and there is an inevitable time lag between acquiring operating status information and making storage scheduling decisions. When the system receives a large number of concurrent data storage requests in a short period, some storage nodes may be continuously selected for data writing before their operating status has changed significantly or been reflected in a timely manner, causing a sudden change in the actual operating status of the nodes within a very short time. This instantaneous change in operating status makes it difficult for storage node selection results based on existing status information to accurately reflect the true load situation of the nodes during the actual writing phase, easily leading to problems such as data writing delays, local node congestion, or instability in the writing process. When there is a deviation between the storage node selection result and the actual operating status of the nodes, it not only affects the stability of the high-concurrency data writing process but may also adversely affect the response consistency of subsequent data access processes and the effective implementation of privacy protection strategies, thereby reducing the reliability and practicality of the entire data security storage system in high-concurrency scenarios. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art, the present invention provides a data security storage and privacy protection method for high-concurrency scenarios, so as to solve the problems existing in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for secure data storage and privacy protection in high-concurrency scenarios includes the following steps: Step 1: Receive data storage requests, schedule and process the data storage requests to form a queue of data storage requests to be processed; Step 2: Parse the data to be stored in the queue of data storage requests to obtain data field types, data uses, and data rules, classify the data according to the data field types, data uses, and data rules to obtain classified data to be stored, and each classified data to be stored corresponds to a data type, perform sensitivity identification on the classified data to be stored to determine whether sensitive data exists; Step 3: If sensitive data is determined to exist, perform security processing operations on the classified data to be stored for the identified sensitive data to obtain secure data to be stored; if no sensitive data is determined to exist, directly use the classified data to be stored as the secure data to be stored; Step 4: According to The system selects a storage node based on the current storage system load status and obtains the operating parameters of the selected storage node, including the write request queuing depth and sampling timestamp. The transient distortion index is calculated based on these parameters, and the system determines whether the currently selected storage node exhibits transient fluctuations. Step 5: If the currently selected storage node exhibits transient fluctuations, a new storage node is selected. If the currently selected storage node does not exhibit transient fluctuations, it is designated as the target storage node. Step 6: The securely processed data to be stored is written to the target storage node, resulting in the stored data. Access control policies are also configured for the stored data. Step 7: Upon receiving a data access request for the stored data, the system verifies the request's identity and permissions. If the verification is successful, data content within the access scope is provided to the accessing entity. If the verification fails, the data access request is rejected.

[0009] Preferably, the step of performing sensitivity identification on the categorized data to be stored and determining whether sensitive data exists comprises: for the categorized data to be stored, obtaining sensitivity identification rules corresponding to the data type, the sensitivity identification rules including several sensitive data feature conditions; according to the sensitivity identification rules, comparing the content of each data field in the categorized data to be stored item by item to determine whether each data field meets the sensitive data feature conditions defined by the sensitivity identification rules; when at least one data field meets the sensitive data feature conditions corresponding to the sensitivity identification rules, it is determined that sensitive data exists in the currently categorized data to be stored; when none of the data fields in the categorized data to be stored meet the sensitive data feature conditions corresponding to the sensitivity identification rules, it is determined that sensitive data does not exist in the currently categorized data to be stored.

[0010] Preferably, the steps for obtaining the instantaneous distortion index are as follows: obtaining the write request queuing depth sequence and the sampling timestamp sequence within the current scheduling period; calculating the write queue pressure coefficient and the write queue growth disturbance coefficient based on the write request queuing depth sequence and the sampling timestamp sequence; obtaining the sampling time interval sequence within the current scheduling period; calculating the load sampling stability coefficient based on the sampling time interval sequence; normalizing the write queue pressure coefficient, the write queue growth disturbance coefficient, and the load sampling stability coefficient; and calculating the instantaneous distortion index based on the normalized write queue pressure coefficient, the write queue growth disturbance coefficient, and the load sampling stability coefficient.

[0011] Preferably, the step of obtaining the write queue pressure coefficient is as follows: For the currently selected storage node, obtain the write request queuing depth sequence and the corresponding sampling timestamp sequence of several consecutive sampling points within the current scheduling period; calculate the write queuing depth change and sampling time interval for each adjacent sampling point to obtain the write queuing depth change sequence and sampling time interval sequence; construct the queue growth rate sequence and queue decay rate sequence based on the write queuing depth change sequence and sampling time interval sequence; take the maximum value from the queue growth rate sequence and queue decay rate sequence to obtain the maximum growth rate and maximum decay rate, and calculate the short-term growth dominance based on the maximum growth rate and maximum decay rate; obtain the last value and the maximum value in the write request queuing depth sequence, and record them as the last queuing depth and the maximum queuing depth, respectively, and divide the last queuing depth by the maximum queuing depth to obtain the queue occupancy; multiply the short-term growth dominance by the queue occupancy and take the square root to obtain the write queue pressure coefficient.

[0012] Preferably, the step of obtaining the write queue growth perturbation coefficient is as follows: for the currently selected storage node, obtain the write request queuing depth sequence and the corresponding sampling timestamp sequence for several consecutive sampling times within the current scheduling period; calculate the write queue depth change and sampling time interval for each adjacent sampling point to obtain the write queue depth change sequence and sampling time interval sequence; for each time interval, divide the corresponding write queue depth change by the sampling time interval to obtain the write queue change rate; extract the growth rate subsequence from the change rate sequence; take the median of the growth rate subsequence to obtain the robust center value of the growth rate; obtain the maximum and minimum values ​​of the growth rate subsequence and calculate the difference to obtain the growth rate dispersion amplitude; calculate the write queue growth perturbation coefficient based on the growth rate dispersion amplitude and the robust center value of the growth rate.

[0013] Preferably, the step of obtaining the load sampling stability coefficient is as follows: For the currently selected storage node, obtain a sequence of sampling timestamps corresponding to several consecutive load samples within the current scheduling period; calculate the difference between adjacent sampling timestamps to obtain a sampling time interval sequence, and further calculate the difference between adjacent sampling time intervals in the sampling time interval sequence to obtain an interval increment sequence; obtain the maximum and minimum sampling intervals in the sampling time interval sequence, and calculate the difference to obtain the sampling interval fluctuation amplitude; determine the direction of change of each adjacent interval increment in the interval increment sequence; compare the direction of change of adjacent interval increments in the interval increment sequence segment by segment, and when the direction of change of two adjacent interval increments is inconsistent, it is recorded as a symbol flip, and the number of symbol flips occurring within the current scheduling period is counted. The symbol flip frequency is obtained by dividing the number of symbol flips by the total number of comparable adjacent interval increment segments. The absolute value of all interval increments in the interval increment sequence is taken to obtain the absolute change amplitude. Each absolute change amplitude is then normalized to the sampling interval fluctuation amplitude within the current scheduling period to obtain a normalized change amplitude sequence. The median of the normalized change amplitude sequence is obtained as the typical level of the normalized change amplitude sequence. Using the typical level as a reference, each normalized change amplitude is judged segment by segment, and the number of change segments whose change amplitude reaches or exceeds the typical level is counted as the jump number. The jump number is divided by the total number of comparable change segments to obtain the jump consistency ratio. The load sampling stability coefficient is calculated based on the symbol flip frequency and the jump consistency ratio.

[0014] Preferably, the step of determining whether the currently selected storage node has instantaneous fluctuations based on the instantaneous distortion index is as follows: compare the instantaneous distortion index with the distortion threshold; if the instantaneous distortion index is greater than or equal to the distortion threshold, it is determined that the currently selected storage node has instantaneous fluctuations; if the instantaneous distortion index is less than the distortion threshold, it is determined that the currently selected storage node does not have instantaneous fluctuations.

[0015] Preferably, the steps for verifying the identity and permissions of the data access request are as follows: upon receiving a data access request, extract the identity information of the accessing subject from the data access request; based on the identity information, verify the identity validity of the accessing subject to determine whether the identity information matches a registered legitimate accessing subject; if the identity validity verification passes, obtain the access permission information corresponding to the accessing subject; if the identity validity verification fails, determine that the data access request has failed the identity and permission verification; match and verify the access permission information with the access control requirements corresponding to the target data pointed to by the data access request to determine whether the accessing subject has the permission to access the target data; if the accessing subject passes the permission matching verification, determine that the data access request has passed the identity and permission verification; if the accessing subject fails the permission matching verification, determine that the data access request has failed the identity and permission verification.

[0016] The technical effects and advantages of this invention are as follows:

[0017] The system selects storage nodes based on the current load status of the storage system, obtains the operating parameters of the selected storage nodes, calculates the transient distortion index based on the operating parameters, and determines whether the currently selected storage node has transient fluctuations based on the transient distortion index. If the currently selected storage node has transient fluctuations, a new storage node is selected. If the currently selected storage node does not have transient fluctuations, the currently selected storage node is used as the target storage node, effectively improving the reliability and practicality of the data security storage system in high-concurrency scenarios. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a data security storage and privacy protection method for high-concurrency scenarios, as provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. In addition, the forms of the various structures described in the following embodiments are merely illustrative. The data security storage and privacy protection method for high-concurrency scenarios involved in the present invention is not limited to the structures described in the following embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention provides a method for secure data storage and privacy protection in high-concurrency scenarios, such as... Figure 1 As shown, it includes the following steps:

[0021] Step 1: In high-concurrency access scenarios, receive data storage requests from multiple data sources. The data storage requests include the request arrival time, request type, and business identifier. Schedule and process the data storage requests to form a queue of data storage requests to be processed, so as to avoid direct conflicts between concurrent requests and put pressure on the storage system.

[0022] Step 2: Parse the data to be stored in the data storage request queue to obtain the data field type, data purpose and data rules. Classify the data according to the data field type, data purpose and data rules to obtain the classified data to be stored. Each classified data to be stored corresponds to a data type, including but not limited to identity identification data, business record data and behavior trajectory data. Perform sensitivity identification on the classified data to be stored to determine whether there is sensitive data.

[0023] It should be noted that parsing the data to be stored in the data storage request queue is a prior art technique, and this embodiment will not describe its specific steps in detail.

[0024] In this embodiment, it should be specifically explained that the steps for classifying data according to data field type, data purpose, and data rules are as follows:

[0025] For the data to be stored, obtain the information of the data fields it contains, and identify the data field type corresponding to each data field, which is used to characterize the structural composition features of the data to be stored;

[0026] Based on the business attributes or processing scenario of the data storage request to which the data to be stored belongs, determine the data purpose corresponding to the data to be stored, which is used to characterize the purpose of the data to be stored in the system;

[0027] Based on the data field type and data purpose, the system matches the data rules corresponding to the data to be stored from the preset data rules. The data rules are used to limit the general constraints of specific data in the storage, access or processing process.

[0028] Pre-defined data rules refer to a set of data management constraints that are established and maintained uniformly before or during system operation. These rules are used to regulate the basic behavioral requirements of different types of data during storage, access, or processing. These data rules can be configured based on data field characteristics, data usage, or compliance requirements, and are invoked by the system as a set of rules. During data classification and processing, the applicable general constraints for the data to be stored are determined by matching the corresponding data rules.

[0029] Based on the data field type, data purpose, and matching data rules, the data to be stored is classified and determined to identify the data category to which the data to be stored belongs, thereby obtaining the classified data to be stored;

[0030] Generate a data type identifier for the classified data to be stored, corresponding to its data category, so that the classified data to be stored has a clear data type basis in subsequent sensitive identification and privacy protection processing.

[0031] In this embodiment, it is necessary to specifically explain the steps for performing sensitivity identification on the classified data to be stored and determining whether there is sensitive data:

[0032] For the classified data to be stored, obtain the sensitive identification rules corresponding to the data type. The sensitive identification rules include several sensitive data feature conditions. The sensitive identification rules are used to characterize the data features that need to be protected. They include at least one or more of the following: sensitive field identification rules, sensitive data format rules, or sensitive key content matching rules.

[0033] Sensitive identification rules refer to a set of predefined criteria used to determine whether data to be stored contains information requiring special protection. These rules characterize the common features of sensitive data across different data types. They can be set based on data field attributes, data content structure, or data usage scenarios, and are maintained by the system as a set of rules. When performing sensitive identification on categorized data to be stored, the data content is compared with these rules to determine whether any data content meets the characteristics of sensitive data.

[0034] Based on the sensitive identification rules, the content of each data field in the classified data to be stored is compared item by item to determine whether each data field meets the sensitive data feature conditions defined by the sensitive identification rules.

[0035] When at least one data field satisfies the sensitive data feature condition corresponding to the sensitive identification rule, it is determined that there is sensitive data in the currently classified data to be stored; when none of the data fields in the classified data to be stored satisfy the sensitive data feature condition corresponding to the sensitive identification rule, it is determined that there is no sensitive data in the currently classified data to be stored.

[0036] Step 3: If sensitive data is determined to exist, perform security processing on the classified data to be stored after identifying the sensitive data to obtain the data to be stored after security processing. If no sensitive data is determined to exist, the classified data to be stored will be directly used as the data to be stored after security processing.

[0037] Security processing refers to the process of protecting sensitive content in data that is determined to contain sensitive information, according to a preset data security policy. Its purpose is to reduce the risk of leakage of sensitive information during storage and access without affecting data storage and subsequent legitimate use.

[0038] It should be noted that performing security processing operations on the classified data to be stored after identifying sensitive data is existing technology, and this embodiment will not describe the specific steps in detail.

[0039] It should be noted that security processing operations include at least one of data encryption, desensitization, or anonymization to reduce the risk of leakage of raw data during storage and subsequent access.

[0040] Step 4: Select storage nodes based on the current load status of the storage system. It should be noted that the storage system contains several storage nodes. The load status is a state variable used to characterize the current operating load level of the storage node. In this embodiment, the load status can be directly obtained by the storage system monitoring module, and the storage node with the smallest load status value is selected. Obtain the operating parameters of the selected storage node. The operating parameters of the storage node include the write request queuing depth and the sampling timestamp. Calculate the instantaneous distortion index based on the operating parameters of the storage node, and determine whether the currently selected storage node has instantaneous fluctuations based on the instantaneous distortion index.

[0041] In this embodiment, it should be specifically explained that the steps for obtaining the instantaneous distortion index are as follows:

[0042] Obtain the write request queuing depth sequence and sampling timestamp sequence within the current scheduling period, and calculate the write queue pressure coefficient and write queue growth perturbation coefficient based on the write request queuing depth sequence and sampling timestamp sequence;

[0043] Obtain the sampling time interval sequence within the current scheduling period, and calculate the load sampling stability coefficient based on the sampling time interval sequence;

[0044] The write queue pressure coefficient, write queue growth disturbance coefficient, and load sampling stability coefficient are normalized. Specifically, in this embodiment, vector normalization can also be used to normalize these coefficients. Specifically, the three coefficients are combined into a three-dimensional vector. The norm value is obtained by taking the square root of the sum of squares of each component of this three-dimensional vector. The write queue pressure coefficient, write queue growth disturbance coefficient, and load sampling stability coefficient are then divided by this norm value, thereby normalizing the three coefficients. The purpose of this vector normalization is to ensure that the write queue pressure coefficient, write queue growth disturbance coefficient, and load sampling stability coefficient have a uniform order of magnitude and scale standard in the subsequent calculation of the instantaneous distortion index. This avoids the abnormal amplification or weakening of a coefficient in the comprehensive evaluation due to large differences in the original value ranges of different coefficients, thereby improving the stability and reliability of the instantaneous distortion index calculation results. Since vector normalization is an existing technology, its mathematical calculation process has been publicly disclosed and widely used in fields such as multi-parameter comprehensive evaluation, system state analysis, and signal processing. Therefore, this embodiment will not elaborate on its specific algorithm steps. The instantaneous distortion index is calculated based on the normalized write queue pressure coefficient, write queue growth disturbance coefficient, and load sampling stability coefficient. The specific steps for obtaining the index are as follows:

[0045] ;

[0046] In the formula, Expressed as the instantaneous distortion index, This is represented as the normalized write queue pressure coefficient. The greater the write queue pressure a storage node experiences under high-concurrency write scenarios, the more susceptible its current load state is to factors such as short-term congestion and write lag. This can lead to a discrepancy between node selection or status judgment based on this load state and the node's actual processing capacity. By establishing a positive correlation between the write queue pressure coefficient and the transient distortion index, the impact of write pressure on the accuracy of load perception can be reflected. The transient distortion index amplifies synchronously with increasing write pressure, more realistically depicting the risk of storage node status information distortion under high-concurrency conditions. This represents the normalized write queue growth perturbation coefficient. When the write queue length of a storage node experiences significant fluctuations or perturbations, the write load exhibits unstable dynamic characteristics within a short period, making it difficult to accurately and promptly reflect the actual operating state of the storage node. By establishing a positive correlation between the write queue growth perturbation coefficient and the transient distortion index, the impact of the discontinuity and suddenness of the write process on the accuracy of load perception can be demonstrated. The transient distortion index increases accordingly with the degree of write perturbation, thus more effectively characterizing the risk of state distortion caused by write rate fluctuations. This represents the normalized load sampling stability coefficient. The more stable the sampling process and the more regular the sampling rhythm of the storage node's load state, the more accurately the acquired load state information reflects the node's current operating status, thus reducing the risk of state deviation caused by sampling jitter or abnormal sampling rhythm. By establishing an inverse relationship between the load sampling stability coefficient and the transient distortion index, the supporting role of sampling stability in load perception reliability can be demonstrated. This ensures that the transient distortion index remains at a low level when sampling stability is high, thereby improving the accuracy of storage node state judgment and scheduling decisions. , , This represents the weighting coefficients of the normalized write queue pressure coefficient, the normalized write queue growth perturbation coefficient, and the normalized load sampling stability coefficient. , , , The weights are obtained using the Analytic Hierarchy Process (AHP). It should be noted that the AHP is an existing technology, a structured analysis method used to determine the relative importance of multiple factors. This method decomposes complex decision problems into objective, criterion, and indicator layers, and compares each factor pairwise based on their relative importance to obtain the weight coefficients for each evaluation factor. The AHP has been widely used in multi-indicator evaluation, system state analysis, and decision support, and can reasonably reflect the degree of influence of each factor on the comprehensive evaluation result when multiple factors work together. Therefore, this embodiment uses the AHP to determine the weights. As a well-known method for determining weights, its specific calculation process will not be described further.

[0047] In this embodiment, it should be specifically explained that the steps for obtaining the write queue pressure coefficient are as follows:

[0048] For the currently selected storage node, obtain the write request queuing depth sequence and the corresponding sampling timestamp sequence for a consecutive number of sampling points within the current scheduling period;

[0049] Calculate the change in writing queue depth and the sampling time interval for each adjacent sampling point to obtain the sequence of change in writing queue depth and the sequence of sampling time intervals;

[0050] The queue growth rate sequence and queue decline rate sequence are constructed based on the sequence of changes in queue depth and the sequence of sampling time intervals. The specific steps for obtaining these sequences are as follows:

[0051] ;

[0052] ;

[0053] In the formula, This represents the queue growth rate within the j-th time interval. This represents the queue decay rate within the j-th time interval. This is represented as the change in queue depth during the j-th time interval. This represents the actual time interval of the j-th time interval;

[0054] The maximum values ​​are taken from the queue growth rate sequence and the queue decline rate sequence to obtain the maximum growth rate and the maximum decline rate. The short-term growth dominance is then calculated based on the maximum growth rate and the maximum decline rate. The specific steps for obtaining these values ​​are as follows:

[0055] ;

[0056] In the formula, This is represented by the degree of short-term growth dominance. This is represented as the maximum growth rate. This is expressed as the maximum decay rate;

[0057] Get the last and maximum values ​​in the write request queuing depth sequence, and record them as the last queuing depth and the maximum queuing depth respectively. Divide the last queuing depth by the maximum queuing depth to get the queue occupancy. The larger the queue occupancy value, the closer the current queuing depth is to the short-term peak level.

[0058] The write queue pressure coefficient is obtained by multiplying the short-term growth dominance by the queue occupancy and taking the square root.

[0059] By simultaneously characterizing the changing trend of write request queuing depth and the current queuing state's position within a short-term window, the write pressure situation of storage nodes is comprehensively reflected from both dynamic and static dimensions. Specifically, the short-term growth dominance characterizes whether queue pressure is primarily driven by the continuous increase in write requests, thus reflecting the directional characteristics of pressure formation; queue occupancy characterizes the degree to which the current queuing state is close to the short-term peak, thus reflecting the actual pressure level. By nonlinearly fusing these two dimensions, misjudgments caused by relying solely on instantaneous queuing depth or a single changing indicator can be avoided. This makes the write queue pressure coefficient more sensitive to sudden congestion in a short period and more robust to instantaneous jitter or brief dips, thus providing a more stable and representative input for subsequent evaluation of the instantaneous distortion index.

[0060] In this embodiment, it should be specifically explained that the step of obtaining the write queue growth perturbation coefficient is as follows:

[0061] For the currently selected storage node, obtain the write request queuing depth sequence and the corresponding sampling timestamp sequence for a consecutive number of sampling times within the current scheduling period;

[0062] For each adjacent sampling point, calculate the change in write queue depth and the sampling time interval to obtain the sequence of change in write queue depth and the sequence of sampling time interval. For each time interval, divide the corresponding change in write queue depth by the sampling time interval to obtain the write queue change rate.

[0063] Extract the growth rate subsequence from the rate of change sequence. The specific steps to obtain it are as follows:

[0064] ;

[0065] In the formula, This represents the growth rate of the j-th time interval; if the queue does not grow, it is set to 0.

[0066] The median of the growth rate subsequence is taken to obtain a robust central value of the growth rate, so as to reduce the dominance of the extreme rate caused by a single burst write on the disturbance assessment.

[0067] The maximum and minimum values ​​of the growth rate subsequence are obtained and the difference is calculated to obtain the growth rate dispersion amplitude. The growth rate dispersion amplitude is used to characterize the fluctuation range of the growth rate within the current scheduling period.

[0068] The growth perturbation coefficient of the write queue is calculated based on the dispersion amplitude of the growth rate and the robust central value of the growth rate. The specific steps for obtaining this coefficient are as follows:

[0069] ;

[0070] In the formula, This is represented as the write queue growth perturbation coefficient. This is expressed as the discrete magnitude of the growth rate. This is represented as the robust central value of the growth rate.

[0071] By characterizing the fluctuations in the rate of change of the write queue length over a short period, the stability of the write load variation process is reflected. Using the dispersion of the growth rate as a disturbance criterion, by comparing the fluctuation range of the growth rate within the scheduling period with its typical level, non-stationary states where the write queue does not continuously increase but frequently experiences sudden increases and decreases can be effectively identified. This approach focuses on describing the dynamic instability of the write process itself, enabling it to maintain high sensitivity to instantaneous disturbances caused by factors such as sudden writes and scheduling jitter under high concurrency conditions, while avoiding excessive amplification of the evaluation results by a single abnormal rate, thus providing a representative disturbance measure for the instantaneous distortion index.

[0072] It should be noted that although both the write queue pressure coefficient and the write queue growth perturbation coefficient are calculated based on the information of the write request queuing depth changing over time, they focus on different evaluation dimensions. The former is used to reflect the write pressure bearing status, while the latter is used to reflect the fluctuation and perturbation characteristics of the write process. Therefore, corresponding change rate sequences are constructed for independent evaluation.

[0073] In this embodiment, it should be specifically explained that the steps for obtaining the load sampling stability coefficient are as follows:

[0074] For the currently selected storage node, obtain the sampling timestamp sequence corresponding to a number of consecutive load samples within the current scheduling period;

[0075] The difference between adjacent sampling timestamps is calculated to obtain a sampling time interval sequence, and the difference between adjacent sampling time intervals in the sampling time interval sequence is further calculated to obtain an interval increment sequence, which is used to describe the acceleration or deceleration of the sampling rhythm.

[0076] Obtain the maximum and minimum sampling intervals in the sampling time interval sequence, and calculate the difference to obtain the sampling interval fluctuation amplitude;

[0077] For the interval increment in the interval increment sequence, determine the direction of change of each adjacent interval increment, that is, determine whether each interval increment is positive, negative or unchanged, in order to characterize whether the sampling interval becomes longer or shorter in adjacent time periods.

[0078] The direction of change of adjacent interval increments in the interval increment sequence is compared segment by segment. When the direction of change of two adjacent interval increment segments is inconsistent, it is recorded as a symbol flip. The number of symbol flips that occur in the current scheduling period is counted, and the number of symbol flips is divided by the total number of comparable adjacent interval increment segments to obtain the symbol flip frequency.

[0079] It should be noted that in this embodiment, the total number of comparable adjacent interval increment segments is the number of samples minus 3, and the calculation of the load sampling stability coefficient is based on the comparison between the changing directions of adjacent sampling time intervals. Let the number of samples be m. Since only m-1 sampling time intervals can be obtained from a sampling timestamp sequence with m samples, only m-2 changes in adjacent sampling time intervals can be constructed. Since the determination of sign flipping requires comparing two adjacent changes, the number of adjacent interval increment segments that can be compared for sign flipping in the entire sampling sequence is the number of samples minus 3.

[0080] Take the absolute value of all interval increments in the interval increment sequence to obtain the absolute change amplitude, and normalize each absolute change amplitude with the sampling interval fluctuation amplitude in the current scheduling period to obtain the normalized change amplitude sequence, so as to eliminate the influence of different sampling period lengths on the comparison results.

[0081] Obtain the median of the normalized change magnitude sequence as the typical level of the normalized change magnitude sequence. Using the typical level as a reference, judge each normalized change magnitude segment by segment and count the number of change segments whose change magnitude reaches or exceeds the typical level, which is recorded as the number of jumps.

[0082] Dividing the number of jumps by the total number of comparable change segments yields the jump consistency ratio, which characterizes whether the changes in the sampling time interval occur in a few abnormal periods or occur generally in multiple adjacent periods. The larger the jump consistency ratio, the more consistent and continuous the abnormal changes in the sampling rhythm are, thus reflecting higher sampling stability.

[0083] It should be noted that in this embodiment, the total number of comparable variation segments is the number of samples minus 2. The variation segments are statistically analyzed based on the variation amplitude of adjacent sampling time intervals. Let the number of samples be m. Since only m-1 adjacent sampling time intervals can be obtained from a sampling timestamp sequence with m sampling times, and each sampling time interval itself can constitute an independent variation segment to characterize the change in sampling rhythm within that time interval, the total number of variation segments that can be used for variation amplitude statistics and comparison in the entire sampling sequence is the number of samples minus 2.

[0084] The load sampling stability coefficient is calculated based on the symbol flip frequency and the proportion of transition consistency. The specific steps for obtaining this coefficient are as follows:

[0085] ;

[0086] In the formula, This is expressed as the load sampling stability coefficient. Represented as sign-to-flip frequency, This is represented as the percentage of transition consistency.

[0087] The load sampling stability coefficient is calculated based on the rhythmic characteristics of the sampling time interval changes. By characterizing the switching frequency of the sampling interval between longer and shorter intervals and the consistency of the distribution of the variation amplitude across different time periods, it comprehensively reflects the stability of the load sampling process itself. This method can distinguish between occasional sampling jitter and continuous sampling rhythm imbalance, avoiding misjudgments of stability based solely on a single abnormal sampling interval, thus more realistically reflecting the reliability of load status information under high-concurrency operating conditions. By introducing the load sampling stability coefficient into the evaluation of the instantaneous distortion index, load status distortion caused by sampling rhythm fluctuations can be effectively identified.

[0088] In this embodiment, it should be specifically explained that the step of determining whether the currently selected storage node has instantaneous fluctuations based on the instantaneous distortion index is as follows:

[0089] The instantaneous distortion index is compared with a distortion threshold. If the instantaneous distortion index is greater than or equal to the distortion threshold, the currently selected storage node is determined to have instantaneous fluctuations; if the instantaneous distortion index is less than the distortion threshold, the currently selected storage node is determined not to have instantaneous fluctuations. The distortion threshold is obtained through an adaptive thresholding method. It should be noted that the adaptive thresholding method is an existing technology. The basic idea is to dynamically adjust the judgment threshold based on historical data or statistical characteristics during system operation, rather than using a fixed threshold parameter. This method has been widely used in anomaly detection, status determination, and system operation monitoring technologies to improve the adaptability of the threshold under different operating conditions and the stability of the judgment results. Therefore, the adaptive thresholding method is only presented as a well-known means, and its specific implementation will not be described in detail.

[0090] Step 5: If it is determined that the currently selected storage node has instantaneous fluctuations, then a new storage node is selected; if it is determined that the currently selected storage node does not have instantaneous fluctuations, then the currently selected storage node is used as the target storage node.

[0091] Step 6: Write the securely processed data to be stored into the target storage node to obtain the stored data. This achieves ordered storage and load sharing of data in high-concurrency scenarios. At the same time, configure access control policies for the stored data. It should be noted that the access control policy is used to limit the access permissions of different access subjects to the data, and to verify the access request according to the access control policy when the data access request occurs, so as to prevent unauthorized access behavior.

[0092] Step 7: Upon receiving a data access request for stored data, verify the identity and permissions of the data access request. If the verification is successful, provide the accessing subject with data content that is within the scope of permissions according to the data type and the corresponding privacy protection rules, thereby ensuring data availability while protecting privacy. If the verification fails, reject the data access request.

[0093] In this embodiment, it should be specifically explained that the steps for verifying identity and authorization of data access requests are as follows:

[0094] Upon receiving a data access request, the identity information of the access subject is extracted from the data access request. The identity information is used to uniquely identify the access subject that initiated the data access request.

[0095] Based on the identity information, the access subject's identity validity is verified to determine whether the identity information matches the registered legitimate access subject, so as to confirm whether the data access request comes from a legitimate access subject;

[0096] If the identity validity verification passes, the access permission information corresponding to the access subject is obtained. The access permission information is used to represent the data access scope granted to the access subject in the system. If the identity validity verification fails, the data access request is determined to have failed the identity verification and permission verification.

[0097] The access permission information is matched and verified with the access control requirements corresponding to the target data pointed to by the data access request to determine whether the access subject has the permission to access the target data.

[0098] If the accessing entity passes the permission matching verification, the data access request is determined to have passed both identity and permission verification; if the accessing entity fails the permission matching verification, the data access request is determined to have failed both identity and permission verification.

[0099] Privacy protection rules refer to a set of data access and presentation constraints pre-defined according to data type and maintained by the system. These rules limit the scope and presentation format of data content that different users can access after identity verification and permission checks. The privacy protection rules include at least data field visibility rules, data anonymization rules, and data access granularity control rules, determined by matching data type with the access user's permission level. When a user initiates a data access request and verification is successful, the system performs field filtering, content anonymization, or data trimming on the target data according to the privacy protection rules, returning only data content that conforms to the user's access permissions. This ensures data availability while preventing unnecessary exposure of sensitive information.

[0100] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0101] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for secure data storage and privacy protection in high-concurrency scenarios, characterized in that, Includes the following steps; Step 1: Receive data storage requests, schedule and process the data storage requests, and form a queue of data storage requests to be processed; Step 2: Parse the data to be stored in the data storage request queue to obtain the data field type, data purpose and data rules. Classify the data according to the data field type, data purpose and data rules to obtain the classified data to be stored. Each classified data to be stored corresponds to a data type. Perform sensitivity identification on the classified data to be stored to determine whether there is sensitive data. Step 3: If sensitive data is determined to exist, perform security processing on the classified data to be stored after identifying the sensitive data to obtain the data to be stored after security processing. If no sensitive data is determined to exist, the classified data to be stored will be directly used as the data to be stored after security processing. Step 4: Select storage nodes based on the current load status of the storage system, obtain the operating parameters of the selected storage nodes, including write request queuing depth and sampling timestamp, calculate the transient distortion index based on the storage node operating parameters, and determine whether the currently selected storage node has transient fluctuations based on the transient distortion index; Step 5: If it is determined that the currently selected storage node has instantaneous fluctuations, then a new storage node is selected; if it is determined that the currently selected storage node does not have instantaneous fluctuations, then the currently selected storage node is used as the target storage node. Step 6: Write the securely processed data to be stored into the target storage node to obtain the stored data, and configure access control policies for the stored data at the same time; Step 7: Upon receiving a data access request for stored data, verify the identity and permissions of the data access request. If the verification is successful, provide the accessing subject with data content that is within the scope of permissions. If the verification fails, reject the data access request. The steps for obtaining the instantaneous distortion index are as follows: Obtain the write request queuing depth sequence and sampling timestamp sequence within the current scheduling period, and calculate the write queue pressure coefficient and write queue growth perturbation coefficient based on the write request queuing depth sequence and sampling timestamp sequence; Obtain the sampling time interval sequence within the current scheduling period, and calculate the load sampling stability coefficient based on the sampling time interval sequence; The write queue pressure coefficient, write queue growth disturbance coefficient, and load sampling stability coefficient are normalized, and the instantaneous distortion index is calculated based on the normalized write queue pressure coefficient, write queue growth disturbance coefficient, and load sampling stability coefficient.

2. The data security storage and privacy protection method for high-concurrency scenarios according to claim 1, characterized in that: The step of performing sensitivity identification on the classified data to be stored to determine whether there is sensitive data is as follows: For the classified data to be stored, obtain the sensitive identification rules corresponding to the data type. The sensitive identification rules include several sensitive data feature conditions. Based on the sensitive identification rules, the content of each data field in the classified data to be stored is compared item by item to determine whether each data field meets the sensitive data feature conditions defined by the sensitive identification rules. When at least one data field satisfies the sensitive data feature condition corresponding to the sensitive identification rule, it is determined that there is sensitive data in the data to be stored after the current classification. If none of the data fields in the classified data to be stored meet the sensitive data feature conditions corresponding to the sensitive identification rule, it is determined that there is no sensitive data in the currently classified data to be stored.

3. The data security storage and privacy protection method for high-concurrency scenarios according to claim 1, characterized in that, The steps for obtaining the write queue pressure coefficient are as follows: For the currently selected storage node, obtain the write request queuing depth sequence and the corresponding sampling timestamp sequence for a series of consecutive sampling points within the current scheduling period; Calculate the change in writing queue depth and the sampling time interval for each adjacent sampling point to obtain the sequence of change in writing queue depth and the sequence of sampling time intervals; The queue growth rate sequence and the queue decay rate sequence are constructed based on the sequence of changes in queue depth and the sequence of sampling time intervals. The maximum values ​​are taken from the queue growth rate sequence and the queue decline rate sequence to obtain the maximum growth rate and the maximum decline rate. The short-term growth dominance is calculated based on the maximum growth rate and the maximum decline rate. Get the last and maximum values ​​in the write request queuing depth sequence, and record them as the last queuing depth and the maximum queuing depth respectively. Divide the last queuing depth by the maximum queuing depth to get the queue occupancy. The write queue pressure coefficient is obtained by multiplying the short-term growth dominance by the queue occupancy and taking the square root.

4. The data security storage and privacy protection method for high-concurrency scenarios according to claim 1, characterized in that: The steps for obtaining the write queue growth perturbation coefficient are as follows: For the currently selected storage node, obtain the write request queuing depth sequence and the corresponding sampling timestamp sequence for a consecutive number of sampling times within the current scheduling period; For each adjacent sampling point, calculate the change in write queue depth and the sampling time interval to obtain the sequence of change in write queue depth and the sequence of sampling time interval. For each time interval, divide the corresponding change in write queue depth by the sampling time interval to obtain the write queue change rate. Extract the growth rate subsequence from the change rate sequence; The median of the growth rate subsequence is taken to obtain the robust central value of the growth rate; The maximum and minimum values ​​of the growth rate subsequence are obtained and the difference is calculated to obtain the discrete amplitude of the growth rate. The growth perturbation coefficient of the write queue is calculated based on the dispersion of the growth rate and the robust central value of the growth rate.

5. A data security storage and privacy protection method for high-concurrency scenarios according to claim 1, characterized in that: The steps for obtaining the load sampling stability coefficient are as follows: For the currently selected storage node, obtain the sampling timestamp sequence corresponding to a number of consecutive load samples within the current scheduling period; The difference between adjacent sampling timestamps is calculated to obtain a sampling time interval sequence, and the difference between adjacent sampling time intervals in the sampling time interval sequence is further calculated to obtain an interval increment sequence; Obtain the maximum and minimum sampling intervals in the sampling time interval sequence, and calculate the difference to obtain the sampling interval fluctuation amplitude; Determine the direction of change of each adjacent interval increment in the interval increment sequence; The direction of change of adjacent interval increments in the interval increment sequence is compared segment by segment. When the direction of change of two adjacent interval increment segments is inconsistent, it is recorded as a symbol flip. The number of symbol flips that occur in the current scheduling period is counted, and the number of symbol flips is divided by the total number of comparable adjacent interval increment segments to obtain the symbol flip frequency. Take the absolute value of all interval increments in the interval increment sequence to obtain the absolute change amplitude, and normalize each absolute change amplitude with the sampling interval fluctuation amplitude in the current scheduling period to obtain the normalized change amplitude sequence. Obtain the median of the normalized change magnitude sequence as the typical level of the normalized change magnitude sequence. Using the typical level as a reference, judge each normalized change magnitude segment by segment and count the number of change segments whose change magnitude reaches or exceeds the typical level, which is recorded as the number of jumps. Divide the number of jumps by the total number of comparable change segments to obtain the jump consistency ratio. The load sampling stability coefficient is calculated based on the symbol flip frequency and the percentage of transition consistency.

6. A data security storage and privacy protection method for high-concurrency scenarios according to claim 1, characterized in that: The step of determining whether the currently selected storage node has instantaneous fluctuations based on the instantaneous distortion index is as follows: The instantaneous distortion index is compared with the distortion threshold. If the instantaneous distortion index is greater than or equal to the distortion threshold, it is determined that the currently selected storage node has instantaneous fluctuations; if the instantaneous distortion index is less than the distortion threshold, it is determined that the currently selected storage node does not have instantaneous fluctuations.

7. A data security storage and privacy protection method for high-concurrency scenarios according to claim 1, characterized in that: The steps for verifying identity and authorization of data access requests are as follows: Upon receiving a data access request, extract the identity information of the accessing subject from the data access request; Based on identity information, the access subject's identity validity is verified to determine whether the identity information matches the registered legitimate access subject; If the identity verification passes, the access permission information corresponding to the access subject is obtained; if the identity verification fails, the data access request is determined to have failed the identity verification and permission verification. The access permission information is matched and verified with the access control requirements corresponding to the target data pointed to by the data access request to determine whether the access subject has the permission to access the target data. If the accessing entity passes the permission matching verification, then the data access request is determined to have passed the identity verification and permission verification. If the accessing entity fails the permission matching verification, the data access request is determined to have failed the identity verification and permission verification.

Images