Data lifecycle-oriented encryption and desensitization collaborative processing method and system

Abstract

The application discloses a data life cycle-oriented encrypted desensitization collaborative processing method and system, and relates to the technical field of data collaborative processing. The method comprises the following steps: S1, collecting link collaborative operation indexes in real time, and performing data preprocessing; S2, comprehensively evaluating the time delay risk and accumulation pressure of the current link, and determining whether to enter an urgent operation state; S3, identifying the life cycle stage of data, constructing a collaborative scheme, evaluating the stage adaptation degree of the candidate collaborative scheme, and screening to form a feasible collaborative scheme set; and S4, constructing an operation state prediction model, predicting the potential effect of the feasible collaborative scheme in terms of tail delay and queue relief, and executing an optimal collaborative scheduling scheme. The problems that, due to the superposition of multiple processing links in a high-concurrency scene, the calculation overhead is introduced, and the encrypted desensitization collaborative processing cannot simultaneously consider the link real-time performance and operation stability are solved.

Description

Technical Field

[0001] This invention relates to the field of data collaborative processing technology, specifically to an encrypted and desensitized collaborative processing method and system for data lifecycle. Background Technology

[0002] In an operational environment where business data flows continuously and frequently, business data typically undergoes multiple data processing procedures according to established rules at different stages of generation, transmission, processing, and storage to meet business operation and compliance management requirements. Related systems usually build a unified processing framework around the data processing chain, uniformly organizing and managing the data field processing methods, processing order, and execution strategies involved in different stages. Through chain monitoring and operational control mechanisms, they ensure the continuous execution and stable operation of multiple processing links in complex business scenarios.

[0003] For example, invention patent CN114157709B discloses a method for encrypting and de-identifying distributed service data, including the following steps: 1. Marking APIs requiring security policies in the service with different descriptive information using a set of metadata; 2. After starting the service, the application parses the APIs marked with metadata; 3. The service reports the APIs parsed by the application and the corresponding metadata to the information security center; 4. Hashing and storing the APIs and corresponding metadata (in memory or Redis); 5. The gateway intercepts all requested resources (APIs) and parses them to obtain the metadata of the requested resources (APIs); 6. Executing the security policy of the requested resources (APIs) based on the parsed metadata. This invention ensures the information security of the collaborative service itself while eliminating data barriers caused by security policies, guaranteeing the security and high availability of distributed services.

[0004] For example, invention patent CN115333809B discloses a data desensitization and restoration method based on symmetric encryption. The data desensitization process includes: sequentially marking fields in the dataset or table to be desensitized with a first and second mark; requesting a random encryption salt and generating a corresponding random encryption key based on the database name or table name, recording and saving these; initiating a data acquisition request; and desensitizing the acquired data using a symmetric encryption algorithm, generating a desensitized data file, saving it, and creating a corresponding table structure. The data restoration process includes: acquiring the structural information of the data file in the dataset or table to be restored; sequentially requesting and acquiring an encryption salt and encryption key; and restoring the data file using a symmetric encryption algorithm based on the acquired encryption salt and encryption key, generating restored data, saving it, and creating a corresponding table structure based on the file structure. Compared with existing technologies, this invention effectively improves the security of data encryption and data restoration.

[0005] However, in payment clearing, real-time anti-fraud, and high-frequency risk control scenarios, the data processing chain needs to meet strict tail latency constraints. However, field-level encryption, dynamic de-identification, and policy verification all introduce additional computational overhead. When the chain load fluctuates or concurrent transactions surge, if there is a lack of a collaborative granular scheduling mechanism oriented towards the lifecycle state, problems such as increased encryption strength leading to P95 latency exceeding limits, de-identification task backlog, or policy execution degradation may occur. This makes it difficult to maintain stable encryption and de-identification collaborative processing under real-time conditions, affecting closed-loop control capabilities.

[0006] Therefore, in response to the above problems, there is an urgent need for encryption and desensitization collaborative processing methods and systems that address the data lifecycle. Summary of the Invention

[0007] Technical problems to be solved

[0008] To address the shortcomings of existing technologies, this invention provides a collaborative processing method and system for encryption and desensitization oriented towards the data lifecycle. It solves the problem that the collaborative processing of encryption and desensitization is unable to simultaneously ensure both real-time performance and operational stability due to the computational overhead introduced by multiple processing stages in high-concurrency scenarios.

[0009] Technical solution

[0010] To achieve the above objectives, the present invention provides the following technical solution: a data lifecycle-oriented encryption and desensitization collaborative processing method, comprising the following steps: S1, real-time collection of link collaborative operation indicator sets and data preprocessing of the link collaborative operation indicators; S2, based on the preprocessed link collaborative operation indicator sets, a comprehensive assessment of the latency risk and backlog pressure of the current link is conducted, and a determination is made as to whether to enter an urgent operation state based on the assessment results; S3, in the urgent operation state, the lifecycle stage of the data is identified, a collaborative scheme for encryption, desensitization, and policy verification is constructed, and the stage adaptability of candidate collaborative schemes is assessed based on the link collaborative operation indicator sets of historical lifecycle stages, and a set of feasible collaborative schemes is selected; S4, an operation status prediction model is constructed based on the link collaborative operation indicator sets, and the potential effects of feasible collaborative schemes on tail latency and queue mitigation are predicted and judged in combination with the operation status prediction model, and the optimal collaborative scheduling scheme is selected for execution to achieve controlled adjustment and closed-loop optimization.

[0011] Furthermore, the specific process of real-time collection of link collaborative operation indicator set and data preprocessing of link collaborative operation indicator is as follows: A sliding time window is constructed. Through payment gateway link tracing and server-side APM statistics, the end-to-end request-response latency sequence of transaction requests within the window is collected in real time, and quantile statistics are performed to obtain the request-response P95 latency. Simultaneously, the P95 latency of the encryption processing segment, the de-identification processing segment, and the policy verification segment within the window are collected through corresponding processing link embedding statistics. The length of the de-identification task queue and the length of the policy verification task queue within the window are collected synchronously. The length of the de-identification task queue is obtained by counting the number of pending items in the de-identification operator scheduling queue, and the length of the policy verification task queue is obtained by counting the length of the request queue in the policy verification engine. Based on the request-response P95... 5. Construct a link collaborative operation indicator set by considering latency, P95 latency for encryption processing, P95 latency for desensitization processing, P95 latency for policy verification, length of the desensitization task queue, and length of the policy verification task queue. Add sampling timestamps to the link collaborative operation indicator set, map it to a time axis with a unified sampling period, and align it to the same sliding time window statistical interval using the nearest neighbor alignment method within the window. Identify outliers in the link collaborative operation indicator set using the median absolute deviation anomaly detection method, and correct outliers using the quantile truncation method within the window. When there are short-term missing samples, use the forward hold method within the window to complete them. Perform min-max normalization processing on the link collaborative operation indicator set. Establish an encryption and desensitization collaborative processing database to store the original and preprocessed link collaborative operation indicator sets.

[0012] Furthermore, based on the preprocessed link collaborative operation indicator set, a comprehensive assessment of the latency risk and backlog pressure of the current link is conducted. The specific process for determining whether to enter an urgent operation state based on the assessment results is as follows: Read the preprocessed request response P95 latency, desensitization task queue length, and policy verification task queue length, perform natural exponential calculations on each, sum the results of the three exponential calculations, add one to the sum, and take the natural logarithm to obtain the link urgency value; associate the link urgency value with the corresponding link collaborative operation indicator set and compare it with the urgency threshold; when the link urgency value is greater than or equal to the urgency threshold, generate a link urgency status flag and trigger the lifecycle stage constraint modeling process; when the link urgency value is less than the urgency threshold, maintain the currently effective collaborative processing granularity configuration of encryption, desensitization, and policy verification, and only continuously monitor the link collaborative operation indicator set and the link urgency value.

[0013] Furthermore, under urgent operating conditions, the specific process of identifying the data's lifecycle stage and constructing a collaborative solution for encryption, desensitization, and policy verification is as follows: When a link urgency status marker is identified, the lifecycle stage constraint modeling process is executed: the data lifecycle stage marker carried by the business processing flow is read, the transaction processing stage, clearing and settlement stage, and archiving and reporting stage of the current data are distinguished, the collaborative processing range of field-level encryption granularity, desensitization granularity, and policy verification strength is determined according to different lifecycle stages, and a candidate collaborative combination set is generated. Each candidate collaborative combination includes encryption granularity options, desensitization granularity options, and policy verification strength options.

[0014] Furthermore, based on the historical lifecycle stage link collaborative operation index set, the specific process for evaluating the stage adaptability of candidate collaborative schemes is as follows: For each candidate collaborative combination, read the P95 latency of the encryption processing segment, the P95 latency of the desensitization processing segment, and the P95 latency of the policy verification processing segment corresponding to each option in the candidate collaborative combination within the most recent sliding time window under the current lifecycle stage, and sum them to obtain the total candidate processing time; calculate the proportion of the P95 latency of the encryption processing segment, the P95 latency of the desensitization processing segment, and the P95 latency of the policy verification segment in the total candidate processing time, respectively, to obtain the segmented processing time proportion distribution, and construct the candidate resource occupation distribution vector; statistically analyze the segmented processing time proportion distribution of the same historical lifecycle stage from the encryption and desensitization collaborative processing database to obtain the stage target processing structure distribution; calculate the KL divergence between the candidate resource occupation distribution vector and the stage target processing structure distribution, take the negative value of the KL divergence result and perform natural exponentiation to obtain the feasibility margin value of the candidate collaborative combination under the current lifecycle stage.

[0015] Furthermore, the specific process of screening to form a set of feasible collaborative solutions is as follows: calculate the feasibility margin value of each candidate collaborative combination and compare it with the structure matching threshold. If the feasibility margin value is greater than or equal to the structure matching threshold, the candidate collaborative combination is marked as a feasible candidate collaborative combination; otherwise, the candidate collaborative combination is removed from the set of candidate collaborative combinations. Output the subset of feasible candidate collaborative combinations and the corresponding feasibility margin value.

[0016] Furthermore, the specific process of constructing the operation status prediction model based on the link collaborative operation index set is as follows: The link collaborative operation index set within the historical sliding time window is used as the input feature; the encryption granularity option, desensitization granularity option, and policy verification strength option of the candidate collaborative combination are used as the condition input; the request response P95 latency, desensitization task queue length, and policy verification task queue length actually collected within the subsequent sliding time window adjacent to the corresponding window are used as the supervision output label; the operation status prediction model is trained using a linear regression prediction algorithm based on historical time-series samples; the current link collaborative operation index set and feasible candidate collaborative combinations are input into the operation status prediction model, and the predicted request response P95 latency, predicted desensitization task queue length, and predicted policy verification task queue length corresponding to each feasible candidate collaborative combination are output.

[0017] Furthermore, the specific process for predicting and judging the potential effects of feasible collaborative schemes on tail latency and queue mitigation using the operational state prediction model is as follows: For each feasible candidate collaborative combination, the difference between the current request response P95 latency and the predicted request response P95 latency is divided by the current request response P95 latency to obtain the request response latency improvement value; the sum of the current de-identification task queue length and the policy verification task queue length is calculated to obtain the total queue pressure; the sum of the predicted de-identification task queue length and the predicted policy verification task queue length is calculated to obtain the total queue pressure prediction value; the difference between the current total queue pressure and the total queue pressure prediction value is divided by the current total queue pressure to obtain the queue pressure improvement value; the natural exponential operation is performed on the request response latency improvement value and the queue pressure improvement value respectively, and the sum is used to obtain the latency queue improvement joint term; the latency queue improvement joint term is added to one and the natural logarithm is taken as the denominator, and the corresponding feasible margin value is taken as the numerator; the numerator is divided by the denominator to obtain the scheduling priority value of the candidate collaborative combination.

[0018] Furthermore, the specific process of selecting the optimal collaborative scheduling scheme to achieve controlled adjustment and closed-loop optimization is as follows: Select the feasible candidate collaborative combination with the largest scheduling priority value from the subset of feasible candidate collaborative combinations as the optimal collaborative scheduling action for the life cycle stage corresponding to the current window, and generate a collaborative processing strategy version identifier corresponding to the optimal collaborative scheduling action; send the optimal collaborative scheduling action to the field-level encryption executor, desensitization operator and strategy verification engine, update the corresponding encryption granularity, desensitization granularity and strategy verification strength configuration, and set controlled effective period restrictions for processing links involving granularity relaxation; at the same time, feed back the execution feedback of the optimal collaborative scheduling action to the running status prediction model.

[0019] The second aspect of this invention provides an encryption and desensitization collaborative processing system for data lifecycle, comprising: a link operation data processing module for real-time collection of link collaborative operation indicator sets and preprocessing of the link collaborative operation indicators; a link urgency assessment module for comprehensively assessing the latency risk and backlog pressure of the current link based on the preprocessed link collaborative operation indicator set, and determining whether to enter an urgent operation state based on the assessment results; a stage collaborative constraint candidate module for identifying the lifecycle stage of the data in the urgent operation state, constructing a collaborative scheme for encryption, desensitization, and policy verification, evaluating the stage adaptability of candidate collaborative schemes based on the link collaborative operation indicator set of historical lifecycle stages, and screening to form a set of feasible collaborative schemes; and a constraint collaborative modeling and scheduling module for constructing an operation state prediction model based on the link collaborative operation indicator set, predicting and judging the potential effects of feasible collaborative schemes on tail delay and queue mitigation based on the operation state prediction model, and selecting the optimal collaborative scheduling scheme for execution to achieve controlled adjustment and closed-loop optimization.

[0020] Beneficial effects

[0021] The present invention has the following beneficial effects:

[0022] (1) This invention constructs a set of link collaborative operation indicators and introduces a comprehensive evaluation mechanism based on tail delay and queue pressure to realize continuous perception of the operation status of the transaction processing link. It also determines whether to enter the urgent operation state through the urgency threshold, thereby avoiding frequent adjustment of encryption, desensitization and policy verification processing granularity under unnecessary circumstances and improving the overall operation stability.

[0023] (2) This invention combines the link urgency state with the data lifecycle stage, and limits the executable encryption granularity, desensitization granularity and policy verification strength range by stage under the urgency operation state. This effectively avoids the disordered combination of collaborative processing schemes, makes the collaborative processing granularity generation process consistent with the processing characteristics of different business stages, and enhances the stage adaptability of scheme selection.

[0024] (3) This invention introduces an adaptation evaluation mechanism based on the processing structure of the same life cycle stage in history, compares and analyzes the resource occupation structure of candidate collaborative schemes in the current stage, and selects a set of feasible collaborative schemes accordingly, thereby reducing the number of schemes that do not meet the processing characteristics of the stage from entering the scheduling decision process and improving the reliability and controllability of collaborative scheduling.

[0025] (4) This invention constructs a running state prediction model to predict and judge the potential effects of different feasible collaborative schemes on tail delay and queue mitigation, and sorts and selects scheduling priorities based on the prediction results, so that the collaborative scheduling process can be dynamically adjusted under the premise of satisfying tail delay constraints, and forms a closed loop optimization through subsequent running result feedback, thereby improving the continuous running capability of the system in high concurrency scenarios.

[0026] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0027] Figure 1 A flowchart of a collaborative data lifecycle-oriented encryption and desensitization processing method;

[0028] Figure 2 This is a structural diagram of an encryption and desensitization collaborative processing system oriented towards the data lifecycle.

[0029] Figure 3 A diagram showing the comparison of priority values ​​for feasible candidate collaborative combination scheduling;

[0030] Figure 4 This is a schematic diagram illustrating the evolution of collaborative processing granularity throughout the data lifecycle. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. As those skilled in the art will understand, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see Figures 1-4 This invention provides a technical solution: a collaborative processing method for encryption and desensitization oriented towards the data lifecycle, such as... Figure 1As shown, the process includes the following steps: S1, real-time collection of link collaborative operation indicator sets and data preprocessing of the link collaborative operation indicators; S2, based on the preprocessed link collaborative operation indicator sets, a comprehensive assessment of the latency risk and backlog pressure of the current link is conducted, and the assessment results determine whether to enter an urgent operation state; S3, in the urgent operation state, the life cycle stage of the data is identified, a collaborative scheme for encryption, desensitization, and policy verification is constructed, and the stage adaptability of candidate collaborative schemes is evaluated based on the link collaborative operation indicator sets of historical life cycle stages, and a set of feasible collaborative schemes is selected; S4, an operation status prediction model is constructed based on the link collaborative operation indicator sets, and the potential effects of feasible collaborative schemes on tail delay and queue mitigation are predicted and judged in combination with the operation status prediction model, and the optimal collaborative scheduling scheme is selected for execution to achieve controlled adjustment and closed-loop optimization.

[0033] Specifically, the process of real-time collection of link collaborative operation index set and data preprocessing of the link collaborative operation index is as follows: Construct a sliding time window, which is an interval that is updated according to a preset statistical period. It is used to continuously statistically analyze the link operation status. The length of the sliding time window is determined according to the historical average request arrival rate and the tail latency fluctuation period of the link, preferably 100 to 1000 requests to ensure the stability of P95 statistics. The statistical period is the sliding update step size, preferably 1 to 10 seconds, and the statistical period is less than 1 / 5 of the window length to ensure time continuity and data smoothness, and avoid statistical jitter caused by too short a window or response lag caused by too long a window. By using payment gateway link tracing and server-side APM statistics, the end-to-end request-response latency sequence of transaction requests within a window is collected in real time. This sequence comprises the time taken for the complete call chain from when the request enters the payment gateway to when the business process is completed and a result is returned. Quantile statistics are then performed, and the P95 quantile is selected to obtain the P95 request-response latency. The P95 quantile is determined by sorting the latency sequence within the window in ascending order and selecting the [number missing] quantile. The number of samples is obtained by sampling, where n is the number of samples within the window. When the number of samples is less than the minimum number of samples, it is preferable to extend the statistical window to ensure the reliability of the statistical results. Simultaneously, the P95 latency of the encryption processing segment, the P95 latency of the desensitization processing segment, and the P95 latency of the policy verification segment within the corresponding processing link are collected through statistical data collection using corresponding processing link data points. The P95 latency of each processing segment is obtained by setting timestamp data points at the entry and exit points of the field-level encryption module, the desensitization operator module, and the policy verification module to obtain the latency sequence and then performing window quantile statistics. This is used to characterize the independent contribution of the corresponding processing link to the overall link latency. The P95 quantile can reflect the overall response level while highlighting the impact of a small number of high-latency requests on business experience and link stability. Compared to the average value, it is less susceptible to instantaneous jitter interference and has better stability and repeatability than extreme value statistics. It is suitable for continuous monitoring and scheduling control decisions regarding tail latency risks in concurrent request scenarios. The lengths of the de-identification task queue and the policy verification task queue are collected synchronously within the window. The length of the de-identification task queue is obtained by counting the number of pending items in the de-identification operator scheduling queue, while the length of the policy verification task queue is obtained by counting the length of the request queue in the policy verification engine. Both the lengths of the de-identification task queue and the policy verification task queue are instantaneous queue sizes within the window's statistical period, reflecting the dynamic relationship between processing capacity and request arrival rate. Based on the request-response P95 latency, encryption processing P95 latency, de-identification processing P95 latency, policy verification P95 latency, de-identification task queue length, and policy verification task queue length, a link collaborative operation index set is constructed. This index set is used to uniformly characterize the comprehensive operational status of the link within the current time window in terms of performance and load, and serves as the basic data input for subsequent urgency assessment, stage constraint modeling, and scheduling decisions. A sampling timestamp is appended to the link collaborative operation index set to identify the collection time of each index data, mapping it to a time axis with a unified sampling period, and using a nearest-neighbor alignment method within the window to align it to the same sliding time window statistical interval, thus eliminating differences in sampling frequency between different indicators. The time offset caused by the difference; outliers in the link collaborative operation index set are identified by the median absolute deviation anomaly detection method. Specifically, within the sliding time window, the median of each operation index is calculated, and the absolute deviation between each sampled value and the median is calculated. The median of the absolute deviation is used as the deviation benchmark. When the absolute deviation corresponding to a certain sampled value exceeds the preset multiple threshold of the deviation benchmark, the sampled value is determined to be an outlier. The preset multiple threshold is preferably between 2.5 and 4.5, preferably 3. The multiple threshold can be calibrated according to the balance result of the anomaly detection rate and false detection rate of historical link operation data. MAD statistics are calculated based on the samples within the current sliding time window and are updated in real time as the window scrolls.Outliers include atypical sampling points caused by transient jitter, network fluctuations, and system transient interference. An in-window quantile truncation method is used to correct outliers to ensure the stability and robustness of the statistical results. The upper and lower limits of the quantile truncation are preferably the 5th and 95th quantiles within the window. When an outlier exceeds the upper limit, it is corrected to the upper limit value; when it falls below the lower limit, it is corrected to the lower limit value to ensure the continuity of the indicator distribution. When short-term missing samples exist, an in-window forward hold method is used for completion, that is, the missing position is filled with the most recent valid sample value to avoid the missing data affecting subsequent continuous analysis. The short-term missing data criterion is that the continuous missing time does not exceed twice the sampling period, preferably 2 to 20 seconds. When the missing time exceeds 1 / 3 of the window length, it is preferable to mark the current window data as invalid and trigger a resampling mechanism to avoid long-term missing data misleading the statistical results. The link collaborative operation index set is subjected to min-max normalization to map indices of different dimensions to a unified numerical range, so as to meet the requirements of numerical stability for subsequent exponential calculation, distance calculation and model training. An encrypted and desensitized collaborative processing database is established to store the original and preprocessed link collaborative operation index set. This database is used to save historical operation data under different life cycle stages, different collaborative granularity configurations and different time windows, so as to provide data support for stage target structure statistics, operation status prediction model training and closed-loop optimization.

[0034] In this implementation plan, key operational indicators of the link are uniformly collected and preprocessed through a sliding time window. P95 quantile statistics, queue size awareness, and robust anomaly correction mechanisms are introduced to achieve a stable characterization of latency risks and processing backlog pressure at the tail of the link. Time alignment, missing data completion, and normalization processes ensure the consistency and comparability of multi-source indicators in terms of time series and dimensions. Historical operational data is also stored in a structured manner to provide a reliable data foundation for subsequent urgency assessment, lifecycle stage constraint modeling, collaborative granular scheduling, and predictive model training, thereby improving the real-time performance, stability, and feasibility of encryption, desensitization, and policy verification collaborative control.

[0035] Specifically, based on the preprocessed link collaborative operation indicator set, a comprehensive assessment of the latency risk and backlog pressure of the current link is conducted. The process for determining whether to enter an urgent operation state based on the assessment results is as follows: The preprocessed request-response P95 latency, desensitization task queue length, and policy verification task queue length are read, and natural exponential calculations are performed on each. The results of the three exponential calculations are summed, and the sum is added by one and the natural logarithm is taken to obtain the link urgency value. The request-response P95 latency characterizes the end-to-end tail latency level of the link, while the desensitization task queue length and policy verification task queue length characterize the queuing backlog pressure in the security processing stage. These three indicators reflect the link operation status from the perspectives of performance and load, respectively. The natural exponential calculation makes the indicators more sensitive to changes in the high-risk range, thereby amplifying the impact of increased tail latency or worsened queue backlog on the urgency assessment results. By summing the exponential results and taking the logarithm, the risks of multiple indicators are smoothly integrated, avoiding drastic fluctuations in the assessment results caused by instantaneous jitter of a single indicator, and ensuring the continuity and determinability of the link urgency value over time. The link urgency value provides a unified characterization of key operational indicators that directly reflect the performance risks and pressure buildup in the safety handling stages of a link. Through sensitive amplification and smoothing fusion of high latency and high backlog states, it forms a comprehensive representation of the current operational stress level of the link. This is used to determine whether a link has transitioned from a stable operating state to an urgent operating state requiring coordinated granular scheduling intervention, thus providing a reliable basis for subsequent scheduling triggering and control strategy selection. The link urgency value is associated with the corresponding set of link coordinated operation indicators and compared with an urgency threshold. The urgency threshold is a pre-set judgment benchmark based on historical link urgency values, used to distinguish between normal operating intervals and urgent operating intervals requiring coordinated scheduling intervention. The association between the link urgency value and the set of link coordinated operation indicators maintains a traceable relationship between the urgency judgment result and the original operating state, facilitating subsequent scheduling decision interpretation and auditing. When the link urgency value is greater than or equal to the urgency threshold, a link urgency status flag is generated, triggering the lifecycle stage constraint modeling process. This link urgency status flag serves as a trigger condition for subsequent lifecycle stage identification and collaborative granularity candidate generation. It limits the complex collaborative scheduling calculation process to only when there is significant latency risk or processing backlog pressure on the link, thus reducing additional computational overhead during normal operation. When the link urgency value is less than the urgency threshold, the currently effective collaborative processing granularity configuration for encryption, desensitization, and policy verification is maintained, and only the link collaborative operation indicator set and link urgency value are continuously monitored. This approach avoids frequent adjustments to security processing granularity when the link operation is stable, ensuring the stability of business processing strategies. Simultaneously, the link collaborative operation indicator set and link urgency value are continuously updated, providing real-time judgment criteria for potential subsequent urgent operating states.

[0036] The specific formula for the link urgency value is as follows:

[0037] ;

[0038] In the formula, The link urgency value is used to characterize whether the link is in a high-risk or high-load state within the current sliding time window, and serves as the basis for determining whether to trigger lifecycle stage constraint modeling and subsequent collaborative scheduling processes, thereby avoiding one-sided judgments based solely on a single latency indicator or a single queue indicator. This indicates the P95 latency of the request-response pair, used to characterize the tail latency level of the transaction processing link within the current window, reflecting the stress level of the overall real-time constraints of the link. This indicates the length of the de-identification task queue, which reflects the task backlog pressure in the current window during the de-identification processing stage and describes the relationship between the processing capacity of the de-identification operator and the load. This indicates the length of the policy verification task queue, reflecting the queuing and waiting status of the policy verification processing stage within the current window, and characterizing the processing pressure during the policy verification execution phase.

[0039] This implementation scheme unifies the characterization of link tail latency risk and queue backlog pressure in security processing, enabling continuous and stable assessment of link operational stress. Based on a threshold determination mechanism, it distinguishes between stable and urgent operating states, triggering subsequent collaborative scheduling processes only when there are definite performance and load risks. This ensures that P95 tail latency is controllable while avoiding frequent adjustments to encryption, desensitization, and policy verification granularity under normal operating conditions, balancing system stability, scheduling response timeliness, and overall processing efficiency. It provides a reliable and determinable triggering basis for subsequent lifecycle stage constraint modeling and collaborative scheduling decisions.

[0040] Specifically, under urgent operational conditions, the process of identifying the data's lifecycle stage and constructing a collaborative solution for encryption, desensitization, and policy verification is as follows: When an urgent state marker is identified in the link, the lifecycle stage constraint modeling process is executed: The data lifecycle stage marker carried by the business processing flow is read. The data lifecycle stage marker is a stage attribute field transmitted along the link with the business request, used to indicate the current position of the data in the business processing flow. The data lifecycle stage marker is generated by the business system when the request enters the payment access gateway and is consistently transmitted in subsequent processing. Preferably, the data lifecycle stage marker is generated after the payment access gateway completes transaction type parsing, written to the stage attribute field in the unified request context object, and simultaneously written to a fixed field name in the HTTP request header. During synchronous service calls, the stage marker is automatically passed through the unified service framework's call context object. Before each internal RPC call, the calling service writes the stage attribute field to the call context, and the called service reads this field from the context when receiving the request. In asynchronous message processing scenarios, the stage marker is written to a fixed attribute field in the message header before message sending. The message consumer reads this field and restores it to the service context environment of the currently executing thread. Before performing field-level encryption, de-identification, and policy verification, each service node reads the stage attribute fields from the current service context and determines the currently effective collaborative processing scope based on a preset stage mapping table. The stage mapping table is loaded into the memory configuration area during initialization, uniformly distributed by the configuration center, and periodically retrieved and updated by each node to ensure consistency of stage identification rules across the entire chain. After the stage marker is generated, an internal signature verification value is attached. This signature verification value is generated based on an internal key, and its validity is synchronously verified when the service reads the marker to prevent unauthorized tampering of the stage marker in the chain. Simultaneously, the association between the stage marker and the request identifier is recorded in the log system to support subsequent auditing and backtracking. Based on the data lifecycle stage markers, the current data is distinguished into the transaction processing stage, clearing and settlement stage, and archiving and reporting stage. The transaction processing stage addresses the processing of business requests with high real-time requirements; the clearing and settlement stage corresponds to the summarization, reconciliation, and settlement of transaction results; and the archiving and reporting stage corresponds to the data retention and compliance reporting process after the business is completed.The collaborative processing range for the currently allowed field-level encryption granularity, de-identification granularity, and policy verification strength is determined based on different lifecycle stages. This collaborative processing range is a pre-configured set of optional processing levels for different lifecycle stages, used to constrain the encryption strength, de-identification methods, and policy verification strictness that can be adopted in the corresponding stage, to avoid introducing unacceptable processing overhead in stages with high real-time requirements. Field-level encryption granularity includes, but is not limited to: performing strong encryption on core sensitive fields, selective encryption on extended sensitive fields, and no encryption on non-sensitive fields. Core sensitive fields are key fields directly related to funds, identity, or account security, while extended sensitive fields are business fields with associated risks in specific business scenarios. De-identification granularity includes, but is not limited to: performing full mask de-identification on field values, performing partial mask de-identification on field values, and performing only format preservation or weak de-identification on field values. Full mask de-identification completely hides the true content of the field, partial mask de-identification preserves the field structure but hides characters in key positions, and weak de-identification reduces the degree of sensitivity exposure while ensuring readability. The policy verification strength includes, but is not limited to: performing full rule verification, performing core rule verification, and performing simplified rule verification. Full rule verification is used to conduct multi-dimensional and multi-condition security and compliance checks on requests; core rule verification is used to verify only key risk points; and simplified rule verification is used to reduce the number of policy matching operations and thus reduce processing latency in high-concurrency scenarios. Each processing granularity adopts a hierarchical configuration mechanism, preferably divided into three processing levels, which can be expanded according to business complexity. Each level is preset as configurable parameters and corresponds one-to-one with specific algorithm types, rule quantities, and mask strength, thereby ensuring quantifiable differences and executable feasibility between different levels. During the transaction processing stage, strong encryption is preferably applied to core sensitive fields, and selective encryption is applied to extended sensitive fields, combined with partial masking or weak desensitization methods, and core rule verification is performed to ensure critical data security while meeting real-time requirements. During the clearing and settlement stage, strong encryption is preferably applied to both core and extended sensitive fields, combined with partial or full masking desensitization methods, and full rule verification is performed to ensure the security and consistency of clearing results. During the archiving and reporting phase, it is preferable to perform strong encryption on all sensitive fields, combined with full masking for data masking, and execute full rule verification or compliance verification rules to meet the security requirements for data retention, auditing, and compliant reporting. Through these methods, the granularity of field-level encryption, data masking, and policy verification at different lifecycle stages has a clear selectable range and corresponding relationship. This ensures that the scope of collaborative processing not only conforms to the characteristics of the business stage but also can be directly configured and executed, thereby enhancing the feasibility and interpretability of the collaborative scheduling process.A candidate collaborative combination set is generated, which is obtained by enumerating the various processing options allowed in the current lifecycle stage. This set covers multiple collaborative processing configuration schemes that can be legally effective within the stage. Each candidate collaborative combination includes encryption granularity options, desensitization granularity options, and policy verification strength options. The encryption granularity option indicates the coverage or algorithm level of field-level encryption, the desensitization granularity option indicates the data desensitization method and mask strength, and the policy verification strength option indicates the execution depth or frequency of policy verification rules. All of these options are directly configurable and executable processing parameters to ensure that the candidate collaborative combination can be directly called and executed by the subsequent scheduling module.

[0041] In this implementation scheme, when the link enters an urgent operational state, the data lifecycle stage is introduced as the core constraint dimension for collaborative scheduling. By carrying and transmitting lifecycle stage markers in business requests, the processing position of the data can be accurately identified, and the selectable range of field-level encryption granularity, de-identification granularity, and policy verification strength can be limited accordingly. This prevents the misuse of security processing configurations that do not conform to the characteristics of the stage from the source. At the same time, by combining and enumerating the allowed processing options within the stage, a set of candidate collaborative combinations that can be directly configured and executed is formed, ensuring that subsequent scheduling only selects from a legal and feasible solution space. While ensuring the security and compliance requirements of critical data, this method effectively takes into account the differentiated needs for real-time performance and processing overhead at different lifecycle stages, improving the pertinence, stability, and interpretability of collaborative scheduling decisions, and providing a clear and reliable foundation for subsequent adaptive scheduling and closed-loop optimization.

[0042] Specifically, the process of evaluating the stage adaptability of candidate collaborative schemes based on the link collaborative operation index set of historical lifecycle stages is as follows: For each candidate collaborative combination, read the options in the candidate collaborative combination within the most recent sliding time window under the current lifecycle stage, including encryption granularity options, desensitization granularity options, and policy verification strength options. For the encryption processing segment P95 latency, desensitization processing segment P95 latency, and policy verification processing segment P95 latency corresponding to the same configuration, and sum them to obtain the total processing time of the candidate. Among them, the most recent sliding time window is the most recent statistical window under the corresponding lifecycle stage where the processing configuration of the candidate collaborative combination has actually taken effect. It is used to reflect the latest operating performance of the combination under the current stage, thereby avoiding the distortion of evaluation results due to the use of outdated data. The proportions of the P95 latency of the encryption processing segment, the P95 latency of the desensitization processing segment, and the P95 latency of the policy verification segment in the total candidate processing time are calculated separately to obtain the segmented processing time proportion distribution, and a candidate resource occupancy distribution vector is constructed. The segmented processing time proportion distribution is used to characterize the relative contribution of different security processing stages to the overall processing time. The candidate resource occupancy distribution vector describes the resource and time occupancy structure of the three processing segments (encryption, desensitization, and policy verification) in a unified vector form, facilitating comparison and analysis with historical stage structures. The segmented processing time proportion distribution of the same lifecycle stage is statistically analyzed from the encryption and desensitization collaborative processing database to obtain the stage target processing structure distribution. The stage target processing structure distribution is obtained by statistically summarizing the segmented processing time proportion distributions under normal operation conditions in multiple historical sliding time windows within the corresponding lifecycle stage. It is used to characterize the typical processing structure characteristics of the current lifecycle stage under expected operating conditions, does not rely on data from a single moment, and has good stability and representativeness. The KL divergence between the candidate resource occupancy distribution vector and the target processing structure distribution of the current stage is calculated. The negative value of the KL divergence result is then used to perform a natural exponential operation to obtain the feasibility margin value of the candidate collaborative combination in the current lifecycle stage. The KL divergence measures the degree of difference between the resource occupancy structure corresponding to the candidate collaborative combination and the target processing structure of the historical stage; a smaller difference indicates that the candidate collaborative combination better matches the typical processing characteristics of the current lifecycle stage. By negatively taking the KL divergence result and performing a natural exponential operation, the feasibility margin value monotonically increases as the structural difference decreases, thus mapping structural similarity to a numerically comparable feasibility indicator, facilitating subsequent threshold screening and scheduling ranking. This achieves a quantitative assessment of the stage adaptability of candidate collaborative combinations in the current lifecycle stage, enabling the screening process to consider not only the magnitude of a single latency but also the rationality of different security processing links within the overall processing structure.

[0043] The specific formula for the feasible margin value is as follows:

[0044] ;

[0045] In the formula, It represents the feasibility margin value, which is used to characterize whether the candidate collaborative combination meets the processing characteristic constraints of the current life cycle stage, and serves as the basic indicator for subsequent screening of feasible collaborative schemes and participation in scheduling priority calculation, thereby avoiding collaborative schemes that do not conform to the stage processing structure from entering the real-time scheduling decision process. This represents the candidate collaborative combination index, used to distinguish different collaborative configuration schemes for encryption, de-identification, and policy verification. This represents the candidate resource occupancy distribution vector, used to describe the relative time overhead structure of candidate collaborative combinations in the three processing stages of encryption processing, desensitization processing, and policy verification processing; This indicates the distribution of the target processing structure at each stage, reflecting the typical resource consumption ratio characteristics of each processing stage in the same historical lifecycle stage.

[0046] In this implementation scheme, when the link enters an urgent operating state, the processing structures of different encryption, desensitization, and policy verification collaborative schemes can be compared and evaluated based on the stable operating characteristics of the same life cycle stage in the past, avoiding coarse-grained decision-making based solely on instantaneous performance indicators. By introducing an evaluation of the matching degree between the proportion of segmented processing time and the stage target structure, the selected collaborative scheme can meet the tail delay constraint while maintaining consistency with the typical processing mode of the current life cycle stage, thereby improving the stability, interpretability, and feasibility of the collaborative scheduling results, and providing a reliable scheme foundation for subsequent adaptive scheduling and closed-loop optimization.

[0047] Specifically, the process of selecting and forming a set of feasible collaborative solutions is as follows: The feasibility margin value of each candidate collaborative combination is calculated and compared with the structure matching threshold. The feasibility margin value quantifies the degree to which the candidate collaborative combination comprehensively satisfies tail delay constraints and processing structure adaptability at the current lifecycle stage. The structure matching threshold distinguishes between collaborative solutions that are stably executable in the current stage and those with stage mismatch or potential risks. If the feasibility margin value is greater than or equal to the structure matching threshold, the candidate collaborative combination is marked as a feasible candidate collaborative combination, indicating that the candidate collaborative combination meets the requirements of processing structure distribution, resource consumption characteristics, and historical operation. In terms of performance, all candidates exhibit high consistency with the current lifecycle stage, serving as effective alternatives for subsequent scheduling decisions under urgent operational conditions. Conversely, candidate collaborative combinations are removed from the candidate collaborative combination set to avoid mis-selection of encryption, desensitization, or policy verification configurations that are mismatched with the stage in subsequent scheduling, thereby causing additional latency fluctuations and processing backlog risks. The feasible candidate collaborative combination subset and its corresponding feasibility margin value are output as the input basis for subsequent operational state prediction, scheduling priority calculation, and optimal collaborative scheduling action selection, thereby improving the accuracy and stability of collaborative scheduling decisions while ensuring that the computational scale is controlled.

[0048] In this implementation scheme, by quantitatively evaluating the feasibility margin of candidate collaborative combinations and using threshold screening, this method can effectively eliminate processing configurations that are incompatible with the current lifecycle stage or have high potential risks under urgent operating conditions, retaining only collaborative schemes that have good stage adaptability in terms of processing structure, resource consumption, and historical operating characteristics. This significantly reduces the search space and probability of misjudgment in subsequent scheduling decisions, improves the stability, controllability, and implementation efficiency of collaborative scheduling, and provides a reliable and feasible candidate scheme basis for subsequent prediction evaluation and optimal scheduling execution.

[0049] Specifically, the process of constructing a predictive model for operational status based on the link collaborative operation index set is as follows: The link collaborative operation index set within the historical sliding time window is used as the input feature. The link collaborative operation index set includes request response P95 latency, P95 latency of each processing segment, and the length of the desensitization task queue and the length of the policy verification task queue. It is used to characterize the comprehensive operational status of the link in terms of performance and load. Its time sequence corresponds one-to-one with the sliding time window, thus forming a historical sample sequence with temporal correlation. The index values ​​corresponding to each time window are processed using a unified normalized scale before input to ensure the stability of different dimensional indicators in the numerical space. The encryption granularity option, desensitization granularity option, and policy verification strength option of the candidate collaborative combination are used as conditional inputs. These conditional inputs characterize the differences in operational features that the link may exhibit under different security processing configurations, enabling the prediction model to explicitly perceive the impact of changes in collaborative granularity on subsequent operational states. Specifically, each encryption granularity option, desensitization granularity option, and policy verification strength option has a pre-established discrete level mapping table, and each level is mapped to a fixed-length discrete encoding vector before being input into the prediction model. Preferably, each option employs an independent encoding method, mapping the encryption granularity level, desensitization granularity level, and policy verification strength level to independent discrete encoding vectors. The parameters are concatenated at the input layer to form a conditional feature vector. Discrete encoding vectors can be constructed using one-hot encoding, ensuring each level has a unique identifier in the vector space and avoiding misleading numerical order between different levels. When the number of candidate co-op combinations or levels is large, a fixed-dimensional embedding mapping method can be further used to map discrete levels into low-dimensional continuous vectors before inputting them into the model, reducing the input dimensionality and improving training stability. The above encoding mapping relationships are determined and maintained consistently before model training, ensuring that the model uses a unified conditional input expression during both training and inference phases, thereby guaranteeing the repeatability and feasibility of the prediction logic. During feature construction, the temporal feature vector and the conditional input vector are concatenated along the feature dimension to form a complete model input vector. For each historical sample, the input vector and its corresponding subsequent time window supervision label constitute a training sample pair. Preferably, the number of training samples is no less than 1000 sets to ensure the statistical stability of the model parameters. The actual request response P95 latency, desensitization task queue length, and policy verification task queue length collected in the subsequent sliding time window adjacent to the corresponding window are used as supervision output labels. The supervision output labels are used to characterize the actual performance results and backlog changes of the link in the next time window under the given historical running state and collaborative processing configuration, thereby constructing a learnable mapping relationship between input and output.A linear regression prediction algorithm based on historical time-series samples is used to train the operational status prediction model. Specifically, a multivariate linear regression model is constructed, with the predicted request response P95 latency, the predicted desensitization task queue length, and the predicted strategy verification task queue length as independent regression output targets. The model parameters are estimated using the least squares method. The loss function is preferably the sum of the squares of the prediction errors of each output variable, used to measure the deviation between the model's predicted values ​​and the actual observed values. The regression coefficients are iteratively updated until the loss function converges or reaches the preset training epoch limit, thereby obtaining stable model parameters. The model parameter training is completed by minimizing the error between the predicted values ​​and the actual observed values, enabling the model to stably depict the evolution trend of the link's operational status. The current set of collaborative operation indicators and feasible candidate collaborative combinations are input into the operation status prediction model. While keeping the model structure unchanged, forward prediction calculations are performed on different feasible candidate collaborative combinations. The predicted request response P95 latency, predicted desensitization task queue length, and predicted policy verification task queue length are output for each feasible candidate collaborative combination. The above prediction results are used to characterize the tail latency level and queue backlog status that the link may reach in the subsequent time window under different collaborative processing configurations. This provides a direct and quantifiable basis for subsequent collaborative scheduling priority calculation and optimal scheduling decision.

[0050] In this implementation, a predictive model of operational status based on a historical sliding time window is constructed, and the granularity of collaborative processing is used as an explicit conditional input. This enables the system to make forward-looking judgments on tail latency changes and queue accumulation trends under different collaborative schemes before scheduling, thereby avoiding adjustments based solely on post-event feedback. This method organically links the historical operational status of the links, differences in collaborative configuration, and subsequent performance results, enabling predictable evaluation of scheduling effects, providing a quantitative basis for selecting the optimal collaborative scheme, enhancing the accuracy, stability, and controllability of collaborative scheduling decisions, and improving the overall system's adaptive scheduling and closed-loop optimization capabilities under tight operational conditions.

[0051] Specifically, the process of predicting and judging the potential effects of feasible collaborative schemes on tail latency and queue mitigation using the operational status prediction model is as follows: For each feasible candidate collaborative combination, the difference between the current request response P95 latency and the predicted request response P95 latency is divided by the current request response P95 latency to obtain the request response latency improvement value. Here, the current request response P95 latency is taken from the link operation indicators actually collected within the current sliding time window, used to characterize the baseline tail latency level before scheduling execution. The predicted request response P95 latency is output by the operational status prediction model under the given candidate collaborative combination conditions, used to characterize the tail latency result that may be reached in subsequent time windows after the collaborative combination takes effect. The request response latency improvement value is used to characterize the relative improvement of collaborative scheduling on the link tail performance. The sum of the current desensitization task queue length and the policy verification task queue length is calculated to obtain the total queue pressure. The total queue pressure is used to comprehensively characterize the queuing backlog of security processing-related operators within the current time window, reflecting the real-time operational pressure level of the link in terms of load dimension. The predicted queue length for de-identification tasks and the predicted queue length for policy verification tasks are summed to obtain the total queue pressure prediction value. This total queue pressure prediction value is obtained by summing the output parameters of the runtime prediction model and is used to characterize the potential queuing backlog trend in the link within subsequent time windows under the condition that the candidate collaborative combination is effective. The difference between the current total queue pressure and the predicted total queue pressure is divided by the current total queue pressure to obtain the queue pressure improvement value. This improvement value is used to quantify the relative effectiveness of candidate collaborative combinations in alleviating backlog in de-identification and policy verification processing, making the differences in load mitigation capabilities of different candidate collaborative combinations comparable. The natural exponential operation is performed on the improvement values ​​of request response latency and queue pressure, and the sum is obtained to obtain the joint term of latency and queue pressure improvement. Through the natural exponential operation, the synergistic combination with more significant improvement effects in tail latency or queue pressure is numerically amplified, thereby enhancing the ability of scheduling priority calculation to distinguish high-yield solutions. The joint term of latency and queue pressure improvement is added to one and the natural logarithm is used as the denominator. The exponential result is smoothed by logarithmic transformation to avoid the scheduling priority value being unbalanced due to the excessive improvement of a single indicator, thus ensuring the stability and decidability of the scheduling evaluation results. The corresponding feasibility margin value is used as the numerator, which reflects the degree of fit between the candidate synergistic combination and the processing structure of the current life cycle stage. The scheduling priority value of the candidate synergistic combination is obtained by dividing the numerator by the denominator. The scheduling priority value quantifies and weighs the stage adaptability and actual operational benefits of the collaborative scheduling scheme in a unified manner: by using the feasibility margin of the candidate collaborative combination in the current life cycle stage as the numerator, it ensures that the scheduling scheme first meets the stage constraints and compliance boundaries; at the same time, the effect of tail latency improvement and queue pressure relief is integrated and evaluated, and after exponential amplification and logarithmic smoothing, it is used as the denominator to characterize the performance and load adjustment costs brought by the scheduling scheme.Therefore, scheduling priority values ​​can prioritize collaborative solutions that achieve greater latency and backlog mitigation benefits with less adjustment cost, while ensuring stage adaptability, without being dominated by a single indicator. This provides a stable, interpretable, and implementable decision-making basis for adaptive scheduling.

[0052] The specific formula for scheduling priority is as follows:

[0053] ;

[0054] In the formula, This represents the scheduling priority value. The denominator is used to characterize the potential difficulty of improving the link operation status after the candidate collaborative combination is executed, and the numerator is used to characterize the structural feasibility of the candidate collaborative combination in the life cycle stage dimension. The two work together to achieve the priority selection of feasible and significantly improved collaborative solutions, avoiding single-dimensional decision-making based solely on stage adaptability or short-term performance improvement. This index represents feasible candidate collaborative combinations, used to distinguish different combinations of encryption granularity, de-identification granularity, and policy verification strength. This represents the feasible margin value, which reflects the degree of matching between the candidate collaborative combination and the target processing structure at the current stage of the data lifecycle. It is a quantitative expression of the adaptability to stage constraints. This indicates the P95 delay in the request response, used to characterize the current link tail delay level; This indicates the predicted request-response P95 latency, used to reflect the potential impact of candidate cooperative combinations on tail latency improvement; This represents the total queue pressure, used to characterize the overall queue backlog pressure level of the current link; This represents the predicted total queue pressure, used to reflect the potential impact of candidate synergistic combinations on queue backlog mitigation.

[0055] In this embodiment, Table 1 is a data table of scheduling priority values ​​for feasible candidate cooperative combinations. The current request response P95 latency is 120, and the current total queue pressure is 200. The table details the feasibility margin, predicted request response P95 latency, predicted total queue pressure, and scheduling priority values ​​for five feasible candidate cooperative combinations. Specifically, feasible candidate cooperative combination 1 has a feasibility margin of 0.82, a predicted request response P95 latency of 105, a predicted total queue pressure of 150, and a scheduling priority of 0.66731; feasible candidate cooperative combination 2 has a feasibility margin of 0.90, a predicted request response P95 latency of 98, a predicted total queue pressure of 130, and a scheduling priority of 0.69954; feasible candidate cooperative combination 3 has a feasibility margin of 0.76, a predicted request response P95 latency of 98, a predicted total queue pressure of 130, and a scheduling priority of 0.69954; and feasible candidate cooperative combination 3 has a feasibility margin of 0.76, a predicted request response P95 latency of 98, a predicted request response P95 latency of 98, a predicted total queue pressure of 130, and a scheduling priority of 0.69954. The latency is 110, the predicted total queue pressure is 160, and the scheduling priority is 0.63522; the feasibility margin of feasible candidate coordination combination 4 is 0.88, the predicted request response P95 latency is 102, the predicted total queue pressure is 140, and the scheduling priority is 0.70058; the feasibility margin of feasible candidate coordination combination 5 is 0.65, the predicted request response P95 latency is 95, the predicted total queue pressure is 120, and the scheduling priority is 0.49443.

[0056] Table 1. Priority values ​​of feasible candidate cooperative combination scheduling

[0057]

[0058] like Figure 3 The image shows a comparison of scheduling priority values ​​for feasible candidate collaborative combinations. It illustrates the distribution of scheduling priority values ​​for different feasible candidate collaborative combinations within the current sliding time window. The horizontal axis represents the feasible candidate collaborative combination number, and the vertical axis represents the scheduling priority value of each candidate collaborative combination after comprehensively considering the feasibility margin of the lifecycle stage, the improvement in request-response tail latency, and the effect of queuing pressure relief. The height of the bars reflects the comprehensive scheduling ranking result of each candidate collaborative combination under the current operating state and predictive constraints. (This is in conjunction with Table 1 and...) Figure 3It can be seen that there are significant differences in the scheduling priority values ​​of different feasible candidate coordination combinations, indicating that under the same lifecycle stage constraints, different combinations of encryption granularity, desensitization granularity, and policy verification strength have different comprehensive impacts on tail latency and queue pressure. Candidate coordination combination 4 has the highest scheduling priority value, indicating that under the current link operating state and within the feasible margin constraint, the combination provides the most balanced comprehensive improvement effect on request response P95 latency and queue pressure. Candidate coordination combination 5 has a relatively low scheduling priority value. Even though it predicts lower request response P95 latency and queue pressure, its lower feasible margin value inhibits its performance in the comprehensive evaluation. The overall results demonstrate that scheduling decisions are not determined solely by a single performance indicator, but are constrained by both the degree of lifecycle stage adaptation and the ability to improve operating state, avoiding policy imbalances caused by simply pursuing latency or queue minimization.

[0059] In this implementation scheme, a scheduling priority calculation mechanism based on prediction results is introduced to uniformly and quantitatively evaluate the adaptability of candidate collaborative schemes at each lifecycle stage and their potential improvement effects on tail latency and queue backlog. This transforms scheduling decisions from passive responses to proactive selections that can be determined in advance. Under the premise of meeting stage constraints and compliance boundaries, this method can prioritize collaborative schemes that offer higher overall benefits in terms of performance improvement and load mitigation, and whose adjustment costs are more controllable. This avoids scheduling imbalances caused by single-indicator driving, thereby improving the stability, interpretability, and closed-loop optimization capabilities of the collaborative scheduling process.

[0060] Specifically, the process of selecting the optimal collaborative scheduling scheme to achieve controlled adjustment and closed-loop optimization is as follows: From the subset of feasible candidate collaborative combinations, the feasible candidate collaborative combination with the highest scheduling priority value is selected as the optimal collaborative scheduling action for the corresponding lifecycle stage of the current window. The scheduling priority value characterizes the comprehensive scheduling value of each feasible candidate collaborative combination under the conditions of the current link operating state, lifecycle stage constraints, and predicted improvement effects. Selecting the candidate collaborative combination with the highest scheduling priority value can achieve the optimal balance between performance gains and scheduling costs among multiple feasible schemes. A collaborative processing strategy version identifier corresponding to the optimal collaborative scheduling action is generated. This identifier uniquely identifies the collaborative processing configuration effective within the current scheduling cycle and records the corresponding encryption granularity, de-identification granularity, and policy verification strength options to support subsequent operational status backtracking, scheduling behavior auditing, and policy evolution management. The optimal collaborative scheduling action is sent to the field-level encryption executor, the de-identification operator, and the policy verification engine, updating the corresponding encryption granularity, de-identification granularity, and policy verification strength configuration. Among them, the field-level encryption executor, the de-identification operator, and the policy verification engine are all processing modules that can be independently configured and dynamically loaded in the link, and can complete the configuration switching without interrupting business requests, thereby ensuring that the collaborative scheduling action has the ability to take effect in real time. Furthermore, a controlled effective period limit is set for the processing steps involving granularity relaxation. The effective period limit is used to limit the duration of the granularity relaxation configuration. After meeting the current tail latency and queue mitigation requirements, it automatically reverts to the default security level configuration to prevent the granularity relaxation from persisting for a long time and increasing the risk of data exposure. The controlled effective period limit specifically includes an effective period length parameter, a period early termination condition, and a period expiration rollback mechanism. The effective period length parameter is a pre-set maximum duration used to limit the longest duration of the granularity relaxation configuration from the effective time. It is preferably an integer value between 30 seconds and 600 seconds, or preferably the length of 1 to 5 sliding time windows. The effective period length parameter is written into the collaborative processing policy version identifier when the policy version is generated and is synchronously distributed to the corresponding execution module. The early termination condition for the cycle is as follows: if the link urgency value is lower than the urgency threshold for M consecutive sliding time windows within the effective cycle, and the request response P95 latency recovers to below the preset security limit, and the lengths of the de-identification task queue and the policy verification task queue are both lower than their respective historical percentile benchmarks, then granularity rollback is immediately triggered; where M is preferably an integer between 2 and 5. The cycle expiration rollback mechanism is as follows: when the effective cycle reaches the preset effective cycle length parameter, regardless of the current link urgency status, the encryption granularity, de-identification granularity, and policy verification strength are automatically restored to the default security level configuration corresponding to the lifecycle stage, and a rollback operation record is generated and written to the collaborative processing policy version library to ensure that the scheduling behavior is auditable and traceable.The timing optimization of the controlled effective period is managed by the link operation data processing module based on a unified timestamp system. Millisecond-level timestamps are used to record the effective time and rollback time of scheduling actions, ensuring the accuracy and consistency of period control. Simultaneously, the execution feedback of the optimal collaborative scheduling action is fed back to the operation status prediction model. This feedback includes the effective time of the scheduling action, the corresponding collaborative processing strategy version identifier, and the actual request-response P95 latency collected before and after scheduling execution, the P95 latency of each processing segment, the length of the desensitization task queue, and the length of the strategy verification task queue. This information is used as new training samples input into the operation status prediction model, thereby achieving continuous updating of model parameters and gradual improvement of prediction accuracy. This forms a closed-loop optimization process covering scheduling decision-making, execution, effectiveness, and effect evaluation, ensuring that the collaborative scheduling strategy can continuously adapt and evolve with changes in link operation characteristics and business load, improving overall stability and controllability in complex operating scenarios.

[0061] like Figure 4The diagram illustrates the evolution of collaborative processing granularity throughout the data lifecycle. It showcases the collaborative evolution of field-level encryption granularity, de-identification granularity, and policy verification strength at different lifecycle stages in data lifecycle-oriented encryption and de-identification collaborative processing. The horizontal axis represents the evolution of data through the lifecycle stages of the business request chain, including the transaction processing stage, clearing and settlement stage, and archiving and reporting stage. The vertical axis corresponds to the three security processing dimensions: field-level encryption granularity, de-identification granularity, and policy verification strength. The diagram uses different levels to indicate the allowed processing strength for each stage, visually reflecting the balance between real-time and security requirements at different stages. In the transaction processing stage, a lower or medium processing level is preferred to reduce the impact on end-to-end latency while ensuring core security constraints. In the clearing and settlement stage, as real-time constraints decrease, the processing levels of encryption, de-identification, and policy verification are allowed to increase to enhance the security and consistency of the data processing process. In the archiving and reporting stage, the processing level is further increased to meet the security requirements of data retention, auditing, and compliance reporting scenarios. The "Urgent State Trigger (Controlled Adjustment)" label in the diagram illustrates when the link enters an urgent operating state. The red vertical line above is the "Urgent State Trigger Alignment Line," used to identify the current moment when the link enters an urgent operating state and its corresponding position on the lifecycle evolution axis. Urgent state triggering is completed by the link urgency assessment module. Specifically, it calculates the link urgency value by reading the preprocessed request response P95 latency, the desensitization task queue length, and the policy verification task queue length, and compares it with the urgency threshold. This triggers controlled adjustment, entering the lifecycle stage constraint modeling and collaborative granularity candidate generation process. When in an urgent operating state, the collaborative scheduling module can perform controlled adjustments to the processing granularity within the allowable range of the current stage, but without exceeding the predefined collaborative processing boundaries of the stage. For example, it only selects the optimal collaborative scheduling action within the allowed combination space of field-level encryption granularity, desensitization granularity, and policy verification strength level for that stage. The diagram intuitively reflects the overall design concept of this method, which uses the data lifecycle stage as a constraint basis to configure and dynamically coordinate the processing granularity of encryption, desensitization, and policy verification in stages, enhancing the interpretability and implementability of the collaborative processing strategy.

[0062] In this implementation plan, by selecting the best scheduling action based on the scheduling priority value in the feasible collaborative scheme and distributing the scheduling configuration to each security processing module in a versioned manner, the controllability, traceability and rollback adjustment of the collaborative processing granularity are realized. At the same time, the prediction model is continuously corrected by combining scheduling execution feedback, so that the scheduling decision and the actual operation effect form a closed loop linkage. Thus, under the premise of ensuring stage constraints and security boundaries, the tail delay control and queue backlog mitigation needs are dynamically balanced, thereby improving the system's adaptive scheduling capability and long-term operational stability under complex load scenarios.

[0063] Reference Figure 2 As shown, the second aspect of the present invention provides an encryption and desensitization collaborative processing system for data lifecycle, applied to the aforementioned encryption and desensitization collaborative processing method for data lifecycle, comprising: a link operation data processing module, used to collect a set of link collaborative operation indicators in real time and preprocess the link collaborative operation indicators; a link urgency assessment module, used to comprehensively assess the latency risk and backlog pressure of the current link based on the preprocessed set of link collaborative operation indicators, and determine whether to enter an urgent operation state based on the assessment results; a stage collaborative constraint candidate module, used to identify the lifecycle stage of the data in the urgent operation state, construct a collaborative scheme for encryption, desensitization and policy verification, evaluate the stage adaptability of candidate collaborative schemes based on the link collaborative operation indicator set of historical lifecycle stages, and screen to form a set of feasible collaborative schemes; and a constraint collaborative modeling and scheduling module, used to construct an operation state prediction model based on the set of link collaborative operation indicators, combine the operation state prediction model to predict and judge the potential effects of feasible collaborative schemes in terms of tail delay and queue mitigation, select the optimal collaborative scheduling scheme for execution, so as to achieve controlled adjustment and closed-loop optimization.

[0064] In this implementation plan, by organically integrating link operation monitoring, urgency assessment, lifecycle stage constraint modeling, and collaborative scheduling decision-making into a unified system architecture, the system achieves full-process collaborative management of encryption, desensitization, and policy verification processes. The system can maintain stable processing policies when the link is running smoothly, and perform adaptive scheduling based on lifecycle stages and historical operation characteristics under urgent conditions. It also combines predictive models to achieve pre-assessment and closed-loop optimization, thereby improving tail latency control capabilities and queue backlog mitigation effects while ensuring stage adaptability and security processing boundaries. This enhances the system's controllability, stability, and evolvability in complex business scenarios.

[0065] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0066] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. As those skilled in the art will understand, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A collaborative processing method for encryption and desensitization oriented towards the data lifecycle, characterized in that, Includes the following steps: S1 collects the link collaborative operation indicator set in real time and performs data preprocessing on the link collaborative operation indicator; S2, based on the preprocessed link collaborative operation index set, comprehensively assesses the latency risk and backlog pressure of the current link, and determines whether to enter the urgent operation state based on the assessment results; S3, under urgent operating conditions, identifies the lifecycle stage of the data, constructs a collaborative scheme for encryption, desensitization and policy verification, evaluates the stage adaptability of candidate collaborative schemes based on the link collaborative operation index set of historical lifecycle stages, and selects to form a set of feasible collaborative schemes. The specific process of identifying the lifecycle stage of data and constructing a collaborative solution for encryption, de-identification, and policy verification under urgent operating conditions is as follows: When a link urgency status marker is identified, the lifecycle stage constraint modeling process is executed: read the data lifecycle stage marker carried by the business processing process, distinguish the current data's transaction processing stage, clearing and settlement stage, and archiving and reporting stage, determine the range of collaborative processing of field-level encryption granularity, de-identification granularity, and policy verification strength that are currently allowed to be executed according to different lifecycle stages, and generate a set of candidate collaborative combinations. Each candidate collaborative combination includes encryption granularity options, de-identification granularity options, and policy verification strength options. S4. Based on the link collaborative operation index set, construct the operation status prediction model. Combine the operation status prediction model to predict and judge the potential effects of feasible collaborative schemes on tail delay and queue mitigation, select the optimal collaborative scheduling scheme for execution, so as to achieve controlled adjustment and closed-loop optimization.

2. The data lifecycle-oriented encryption and desensitization collaborative processing method according to claim 1, characterized in that, The specific process of collecting the real-time link collaborative operation index set and preprocessing the link collaborative operation index is as follows: A sliding time window is constructed, and the end-to-end request-response latency sequence of transaction requests within the window is collected in real time through payment gateway link tracing and server-side APM statistics. Quantile statistics are then performed to obtain the request-response P95 latency. Simultaneously, the P95 latency of the encryption processing segment, the de-identification processing segment, and the policy verification segment within the window are collected through corresponding processing link tracking. The lengths of the de-identification task queue and the policy verification task queue within the window are also collected synchronously. The length of the de-identification task queue is obtained by counting the number of pending items in the de-identification operator scheduling queue, and the length of the policy verification task queue is obtained by counting the length of the request queue in the policy verification engine. Based on the request-response P95 latency, encryption processing segment P95 latency, de-identification processing segment P95 latency, policy verification segment P95 latency, de-identification task queue length, and policy verification task queue length, a set of link collaborative operation indicators is constructed. A sampling timestamp is appended to the link collaborative operation index set, mapped to a time axis with a unified sampling period, and aligned to the same sliding time window statistical interval using the nearest neighbor alignment method within the window; outliers in the link collaborative operation index set are identified using the median absolute deviation anomaly detection method, and outliers are corrected using the quantile truncation method within the window; when there are short-term missing samples, the forward hold method within the window is used to complete them; min-max normalization processing is performed on the link collaborative operation index set; an encrypted and desensitized collaborative processing database is established to store the original and preprocessed link collaborative operation index sets.

3. The data lifecycle-oriented encryption and desensitization collaborative processing method according to claim 1, characterized in that, The process of comprehensively assessing the latency risk and backlog pressure of the current link based on the preprocessed link collaborative operation index set, and determining whether to enter an urgent operation state based on the assessment results, is as follows: Read the preprocessed request response P95 latency, the desensitization task queue length, and the policy verification task queue length, perform natural exponential calculations on each, sum the results of the three exponential calculations, add one to the summation result, and take the natural logarithm to obtain the link urgency value. Associate the link urgency value with the corresponding link collaborative operation indicator set and compare it with the urgency threshold; When the link urgency value is greater than or equal to the urgency threshold, a link urgency status flag is generated, triggering the lifecycle stage constraint modeling process; When the link urgency value is less than the urgency threshold, the current effective encryption, desensitization, and policy verification collaborative processing granularity configuration is maintained, and only the link collaborative operation indicator set and link urgency value are continuously monitored.

4. The data lifecycle-oriented encryption and desensitization collaborative processing method according to claim 1, characterized in that, The specific process for evaluating the stage adaptability of candidate collaboration schemes based on the link collaboration operation index set based on historical lifecycle stages is as follows: For each candidate collaboration combination, read the P95 latency of the encryption processing segment, the P95 latency of the desensitization processing segment, and the P95 latency of the policy verification processing segment corresponding to each option in the candidate collaboration combination within the most recent sliding time window under the current life cycle stage, and sum them to obtain the total processing time of the candidate. Calculate the proportion of the encryption processing segment P95 delay, the desensitization processing segment P95 delay, and the policy verification segment P95 delay in the total candidate processing time, respectively, to obtain the segmented processing time proportion distribution, and construct the candidate resource occupancy distribution vector. The distribution of segmented processing time proportions for the same lifecycle stage in the encrypted and desensitized collaborative processing database is statistically analyzed to obtain the stage target processing structure distribution. Calculate the KL divergence between the candidate resource occupancy distribution vector and the stage target processing structure distribution. Take the negative value of the KL divergence result and perform natural exponent calculation to obtain the feasible margin value of the candidate collaborative combination under the current life cycle stage.

5. The data lifecycle-oriented encryption and desensitization collaborative processing method according to claim 1, characterized in that, The specific process of screening to form a set of feasible collaborative solutions is as follows: Calculate the feasibility margin value for each candidate synergy combination and compare it with the structure matching threshold. If the feasibility margin value is greater than or equal to the structure matching threshold, the candidate synergy combination is marked as a feasible candidate synergy combination; otherwise, the candidate synergy combination is removed from the candidate synergy combination set. Output the feasible candidate collaborative combination subset and its corresponding feasible margin value.

6. The data lifecycle-oriented encryption and desensitization collaborative processing method according to claim 1, characterized in that, The specific process of constructing the operation status prediction model based on the link collaborative operation index set is as follows: The set of link collaborative operation indicators within the historical sliding time window is used as the input feature, and the encryption granularity option, desensitization granularity option, and policy verification strength option of the candidate collaborative combination are used as the condition input. The request response P95 latency, desensitization task queue length, and policy verification task queue length actually collected in the subsequent sliding time window adjacent to the corresponding window are used as the supervision output labels. A linear regression prediction algorithm based on historical time series samples is used to train the operation status prediction model. The current link collaborative operation index set and feasible candidate collaborative combinations are input into the operation status prediction model, and the predicted request response P95 latency, the predicted desensitization task queue length and the predicted strategy verification task queue length are output for each feasible candidate collaborative combination.

7. The data lifecycle-oriented encryption and desensitization collaborative processing method according to claim 1, characterized in that, The specific process of predicting and judging the potential effects of feasible collaborative solutions on tail delay and queue mitigation by combining the operational state prediction model is as follows: For each feasible candidate collaboration combination, the difference between the current request response P95 latency and the predicted request response P95 latency is divided by the current request response P95 latency to obtain the request response latency improvement value. Calculate the sum of the current de-identification task queue length and the policy verification task queue length to obtain the total queue pressure; Calculate the sum of the predicted desensitization task queue length and the predicted strategy verification task queue length to obtain the total queue pressure prediction value; divide the difference between the current total queue pressure and the total queue pressure prediction value by the current total queue pressure to obtain the queue pressure improvement value. Perform natural exponential operations on the improvement values ​​of request response latency and queue pressure, respectively, and sum them to obtain the joint term of delay queue improvement. Add the joint term of delay queue improvement to one and take the natural logarithm as the denominator. Take the corresponding feasible margin value as the numerator and divide the numerator by the denominator to obtain the scheduling priority value of the candidate cooperative combination.

8. The data lifecycle-oriented encryption and desensitization collaborative processing method according to claim 1, characterized in that, The specific process of selecting the optimal cooperative scheduling scheme for execution to achieve controlled adjustment and closed-loop optimization is as follows: Select the feasible candidate collaborative combination with the highest scheduling priority from the subset of feasible candidate collaborative combinations as the optimal collaborative scheduling action for the lifecycle stage corresponding to the current window, and generate a collaborative processing strategy version identifier corresponding to the optimal collaborative scheduling action; distribute the optimal collaborative scheduling action to the field-level encryption executor, desensitization operator and strategy verification engine, update the corresponding encryption granularity, desensitization granularity and strategy verification strength configuration, and set controlled effective period restrictions for processing links involving granularity relaxation; at the same time, feed back the execution feedback of the optimal collaborative scheduling action to the running status prediction model.

9. A collaborative processing system for encrypted and desensitized data throughout its lifecycle, characterized in that: include: The link operation data processing module is used to collect the link collaborative operation indicator set in real time and perform data preprocessing on the link collaborative operation indicator. The link urgency assessment module is used to comprehensively assess the latency risk and backlog pressure of the current link based on the preprocessed link collaborative operation indicator set, and determine whether to enter the urgent operation state based on the assessment results. The phase-based collaborative constraint candidate module is used to identify the lifecycle stage of data under urgent operating conditions, construct collaborative schemes for encryption, desensitization and policy verification, evaluate the phase adaptability of candidate collaborative schemes based on the link collaborative operation index set of historical lifecycle stages, and screen to form a set of feasible collaborative schemes. The specific process of identifying the lifecycle stage of data and constructing a collaborative solution for encryption, de-identification, and policy verification under urgent operating conditions is as follows: When a link urgency status marker is identified, the lifecycle stage constraint modeling process is executed: read the data lifecycle stage marker carried by the business processing process, distinguish the current data's transaction processing stage, clearing and settlement stage, and archiving and reporting stage, determine the range of collaborative processing of field-level encryption granularity, de-identification granularity, and policy verification strength that are currently allowed to be executed according to different lifecycle stages, and generate a set of candidate collaborative combinations. Each candidate collaborative combination includes encryption granularity options, de-identification granularity options, and policy verification strength options. The constrained collaborative modeling and scheduling module is used to construct a predictive model of the operating status based on the set of link collaborative operation indicators. It combines the predictive model of the operating status to predict and judge the potential effects of feasible collaborative schemes on tail delay and queue mitigation, and selects the optimal collaborative scheduling scheme to be executed in order to achieve controlled adjustment and closed-loop optimization.

Images