Data collaborative directory management method and system

Abstract

The invention discloses a data collaborative directory management method and system, and relates to the technical field of government affair informatization, and the method comprises the steps: generating a directory snapshot containing a global hash value; calculating a directory node hash value of each edge directory node based on the directory snapshot identifier; eliminating clock drift interference through time sequence alignment and dynamic tolerance filtering; inputting the Hash difference time sequence into an isolated forest model to judge a substantial change node; based on the difference entry number and the historical calling weight, combining a dual-threshold rule and an online dichotomy model to hierarchically synchronize requirements; according to a grading result, matching an incremental push mode or a full pull mode, and constructing a synchronous transaction context containing an exponential backoff retry mechanism; synchronous operation is executed through the two-stage state model, and compensation rollback is triggered when the synchronous operation fails; and calculating a health index of the substantially changed node, dynamically selecting a self-healing action and optimizing system parameters. The problem of misjudgment caused by time sequence drift is effectively solved, the synchronization efficiency is improved, and the consistency of directory versions is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of government information technology, and more specifically, to a data collaborative directory management method and system. Background Art

[0002] Data collaboration requires maintaining unified directory metadata such as interface definitions and field structures across multiple departmental edge directory nodes. Existing technologies have serious flaws:

[0003] Timing drift leads to misjudgment: Edge directory nodes are out of sync with the central system clock, and network latency fluctuations cause a discrepancy between the hash value calculation time of edge directory nodes and the time when the central directory snapshot is generated. Traditional methods cannot distinguish between real changes and clock drift, resulting in a large number of false synchronization requests.

[0004] Inaccurate change determination: The single hash difference threshold is easily affected by network jitter, misjudging temporary data disturbances as actual changes, resulting in invalid synchronization.

[0005] Inefficient synchronization: Full synchronization is performed on all nodes without distinguishing the scale of changes. This results in a large amount of redundant data being transmitted across cross-region private networks, resulting in significant latency.

[0006] Version consistency is difficult to ensure: There is no closed-loop fallback mechanism after synchronization failure. The directory version of the edge directory node is inconsistent with that of the center, resulting in interface call exceptions.

[0007] Therefore, there is an urgent need for a management solution that can eliminate timing drift interference, accurately identify substantial changes, synchronize on demand, and ensure version consistency.

[0008] In view of the above problems, the present invention proposes a solution. Summary of the Invention

[0009] The purpose of the present invention is to provide a data collaborative directory management method and system for solving the problems of low synchronization efficiency and inconsistent directory versions caused by timing drift and misjudgment in data collaborative directory management.

[0010] The purpose of the present invention can be achieved through the following technical solutions:

[0011] A data collaborative directory management method includes the following steps:

[0012] Generates a directory snapshot containing directory metadata structure, interface information and directory center hash value based on a preset period and records the directory snapshot generation timestamp;

[0013] Calculate the directory node hash value of each edge directory node and record its calculation time; if the difference between the directory node hash value calculation time and the directory snapshot generation timestamp falls within the dynamic tolerance range, retain the corresponding directory node hash value and mark it as a time series matching record; otherwise, discard it;

[0014] Compare the hash value of each directory node in the time series matching record with the hash value of the directory center to construct a hash difference time series. This hash difference time series is then input into the isolation forest anomaly detection model to output an anomaly score corresponding to the hash value of each directory node. The anomaly score is then compared with a preset anomaly threshold to determine whether each edge directory node has a substantial change. Edge directory nodes that have undergone substantial changes are marked as substantially changed nodes.

[0015] For each node that has actually changed, the corresponding current edge directory snapshot is compared with the last successfully synchronized edge directory snapshot based on the historical edge directory snapshot information, and the number of different edge directory nodes is counted; and the ratio of the number of different edge directory nodes to the total number of edge directory nodes and the historical call weight are compared with the corresponding preset threshold to obtain the synchronization demand grading result, and the synchronization mode is preliminarily matched based on the synchronization demand grading result.

[0016] As a further solution of the present invention: the directory center hash value is the directory hash value corresponding to the directory sender; the edge directory node is each directory receiver, and the directory node hash value is the directory hash value received by each directory receiver; the synchronization requirement grading result includes lightweight synchronization requirements and heavy synchronization requirements; the synchronization mode includes incremental push mode and full pull mode.

[0017] As a further solution of the present invention: the process of the isolation forest anomaly detection model outputting the anomaly score corresponding to the hash value of each directory node includes: extracting four indicators of the hash difference time series, namely the current hash difference, the sliding window moving average, the sliding window standard deviation and the overall mean of the hash difference time series, and combining them into a four-dimensional feature vector; inputting the feature vector into the isolation forest anomaly detection model, and obtaining the corresponding anomaly score by calculating the average path length of the four-dimensional feature vector in multiple isolated trees and normalizing it. The anomaly score is calculated according to the formula ;in, is a four-dimensional eigenvector, for The path length of each tree in the anomaly detection model, E is the average path length in the forest, c is the path length normalization factor.

[0018] As a further solution of the present invention: the synchronization demand classification result is specifically obtained by the following rules: the number of difference edge directory nodes DC and the historical call weights of these difference edge directory nodes Compare it with its corresponding preset threshold; when and When , the node with substantial changes is marked as lightweight synchronization; otherwise, it is marked as heavy synchronization; is the total number of directory snapshots, α is the preset ratio threshold, and β is the preset weight threshold.

[0019] As a further solution of the present invention: matching the preliminary synchronization mode according to the synchronization requirement grading result of each substantial change node; in order to further fine-tune the selection of the preliminary synchronization mode, it also includes calculating the grading confidence of each synchronization requirement grading result, and comparing its grading confidence with the preset confidence threshold. Adjusting the synchronization mode according to the grading confidence of the synchronization requirement grading result includes: if the synchronization requirement grading result is lightweight synchronization, and its grading confidence is greater than or equal to the preset confidence threshold, then maintaining the incremental push mode; otherwise, adjusting the synchronization mode to the full pull mode; if the synchronization requirement grading result is heavy synchronization, and its grading confidence is greater than or equal to the preset confidence threshold, then maintaining the full pull mode; otherwise, adjusting the synchronization mode to the incremental push mode.

[0020] As a further solution of the present invention: an incremental push transaction context is constructed for the essential change node of lightweight synchronization, which includes the node identifier, target snapshot identifier, transaction ID, patch version number and base snapshot identifier; a full pull transaction context is constructed for the essential change node of heavy synchronization, which includes the node identifier, target snapshot identifier, idempotent snapshot ID and compressed directory list; and the above-mentioned incremental push transaction context and full pull transaction context are recorded as synchronization transaction context.

[0021] As a further solution of the present invention: a two-stage state model is used based on the synchronization transaction context to perform synchronization of the substantially changed nodes, and the two-stage state model includes: a first transaction stage: the substantially changed node performs operations according to the synchronization mode, the quantity mode downloads and verifies the patch package and then updates the edge directory node, and the full mode downloads the complete snapshot and overwrites the edge directory node after verifying the signature; if the operation is successful, the state is transferred to applied, and if it fails or times out, the state is transferred to rolled back; the second transaction stage: the substantially changed node performs structural self-checking and functional self-checking, and when the structural consistency is full and the interface error rate is lower than the preset upper limit, it is transferred to confirmed, otherwise it rolls back to the previous version.

[0022] As a further solution of the present invention: if the synchronization of the substantive change node in the two-stage state model fails, a retry mechanism is triggered:

[0023] Dynamically calculate retry parameters based on the current synchronous transaction context, the retry parameters include initial delay, backoff factor and number of retries; the initial delay Based on the formula ; The backoff factor According to the formula ;in, is the average network delay, To preset the initial buffer multiple, is the packet loss rate, is the minimum waiting time allowed by the system; and based on the initial delay and backoff factor The retry delay is calculated, and the maximum number of retries is inversely distributed according to the hierarchical confidence of the synchronization mode; retries are performed based on the retry delay and the maximum number of retries. If multiple retries are still unsuccessful, the synchronization is considered to have failed, the rollback mechanism is triggered, and the node with substantial changes is marked as rolled back.

[0024] As a further solution of the present invention: when the substantially changed node enters the rolled back state, it also includes: based on a preset monitoring period, collecting health indicators of the substantially changed node, including heartbeat success rate, average synchronization delay, rollback ratio, time since last confirmation, and resource occupancy load; calculating the health value of the substantially changed node through preset weights, and determining whether a self-healing operation is required based on the health value of the substantially changed node; if the health value of the substantially changed node is less than a preset health threshold, the substantially changed node is recorded as a node requiring self-healing and added to a self-healing candidate list; if the health value of the substantially changed node is greater than or equal to the preset health threshold, it is marked as a healthy node; based on the health value of the substantially changed node, the optimal self-healing operation is selected from the action set, and a recovery operation is performed.

[0025] A data collaborative catalog management system includes a central catalog service cluster that performs snapshot generation, time series alignment, anomaly detection, and policy decision-making;

[0026] Edge directory node agent: deployed on each edge directory node, responsible for local hash calculation, state machine execution and self-test;

[0027] Distributed storage module: stores directory snapshots and synchronizes transaction contexts;

[0028] Reinforcement learning engine: manages the self-healing action decision-making and parameter optimization closed loop;

[0029] Visual monitoring platform: displays status distribution, health index and alarm information in real time.

[0030] Beneficial effects of the present invention:

[0031] (1) This invention achieves precise alignment and filtering of edge directory node hash calculation time and snapshot generation time by introducing a timing tolerance range dynamically calculated based on the heartbeat period and a preset tolerance coefficient. This mechanism can effectively eliminate false difference records caused by local clock drift or network delays in edge directory nodes, significantly reduce the false positive rate, avoid a large number of invalid synchronization operations, and significantly improve network utilization and edge directory node processing efficiency.

[0032] (2) This invention constructs a hybrid grading system that integrates dual-threshold rules and an online binary classification model, and generates synchronization requirement records with confidence indicators based on this system. On the one hand, the dual-threshold rules can quickly distinguish the severity of the change scale; on the other hand, the online binary classification model can adaptively adjust the grading decision to improve the accuracy of the grading. Ultimately, the system only performs the corresponding synchronization on the nodes that truly need a full update or incremental push, which not only ensures the timely consistency of the directory version, but also minimizes unnecessary resource overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below with reference to the accompanying drawings.

[0034] Figure 1 This is a flow chart of a data collaborative directory management method of the present invention;

[0035] Figure 2 This is a schematic diagram of edge directory node synchronization difference detection in step 1 of a data collaborative directory management method of the present invention;

[0036] Figure 3 This is a schematic diagram of the synchronization and hierarchical structure of edge directory nodes in step 1 of a data collaborative directory management method of the present invention;

[0037] Figure 4 This is a schematic diagram of the implementation of the state model in step 3 of the data collaborative directory management method of the present invention;

[0038] Figure 5 This is a schematic diagram of implementing step 4 of a data collaborative directory management method of the present invention;

[0039] Figure 6 It is a structural diagram of a data collaborative directory management system of the present invention. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0041] Example 1

[0042] See also Figure 1 As shown, the present invention is a data collaborative directory management method and system, comprising the following steps:

[0043] Step 1: Generate a directory snapshot based on a preset snapshot generation cycle and assign a unique snapshot identifier; each edge directory node extracts the directory snapshot identifier within the heartbeat cycle through the directory perception agent, calculates the directory node hash value and records the directory node hash value calculation time, calls the difference detection model, and sets a dynamic tolerance range to filter the directory node hash value records; constructs a hash difference time series; and combines the moving average, standard deviation and isolation forest anomaly detection model of the hash difference time series to identify substantial changes; counts the number of difference entries and historical call weights of the substantial change nodes, applies a dual threshold rule or an online binary classification model, divides the substantial change nodes into lightweight synchronization or heavy synchronization, and generates a synchronization requirement record; the directory snapshot includes the directory metadata structure, interface information and directory center hash value;

[0044] In this embodiment, the central directory system is connected to each edge directory node via a secure private network, and the central directory system generates directory snapshots on the distributed storage nodes at a preset period. The directory snapshots contain: the interface name, field description, parameter list, interface call address, and authorization information. The above contents are concatenated and then calculated to obtain a global hash result GHash. Each directory snapshot is also assigned a unique snapshot identifier (SID) and the directory snapshot generation timestamp is recorded.

[0045] Each edge directory node deploys a directory-aware agent and accesses the time series database and message queue. The directory-aware agent The directory snapshot indicated by the latest unique snapshot identifier is obtained through the pull instruction carried by the heartbeat message or the active push of the center; the directory-aware agent locally performs the same hash operation as the center on the directory snapshot content to generate a directory node hash value LHash record; and obtains the directory node hash value calculation time PTime and network quality indicators including average round-trip delay and packet loss rate together with the node identifier NID and records them in the time series database;

[0046] like Figure 2 As shown, the central directory system obtains the directory node hash value calculation time PTime and the corresponding directory node hash value LHash of the specified edge directory node in each preset detection cycle, and aligns the time sequence with the corresponding global directory snapshot generation timestamp CTime; the alignment method is: for each directory node hash value, calculate the difference between its directory node hash value calculation time PTime and the directory snapshot generation timestamp CTime to obtain the clock deviation , and set the dynamic tolerance range ; Wherein, k is the preset tolerance adjustment multiple and k>1; timing filtering is performed based on the set dynamic tolerance range;

[0047] Only select The directory node hash values ​​of the dynamic tolerance range will be used for subsequent comparisons. Directory node hash values ​​that exceed the dynamic tolerance range will be considered as timing mismatches and automatically discarded.

[0048] Mark the m directory node hash values ​​retained by time alignment as directory node hash value records, and calculate the difference between them and the global hash result GHash in turn to obtain the hash difference value of each directory node Combined with the corresponding edge directory node hash value calculation time PTime, construct the hash difference time series ;in, , i is the index of the reserved directory node hash value record;

[0049] Convert hash difference time series into anomaly detection model input feature vector ;in, and are the moving average and standard deviation of the current hash difference time series, is the overall mean of the hash difference time series;

[0050] Based on the input feature vector The process of using the isolation forest anomaly detection model to obtain the synchronization demand classification results is as follows: Input Calculating anomaly scores , the anomaly score is based on the formula ;in, for The path length of each tree in the anomaly detection model, E is the average path length in the forest, c is the path length normalization factor;

[0051] It should be noted that the anomaly detection model is an isolation forest model trained using historical normal difference sequence data, including multiple isolated trees;

[0052] The calculated anomaly score and preset abnormal threshold Make comparisons;

[0053] when If it is determined to be a substantially changed node, then the edge directory node is considered to have undergone a real change; otherwise, it is considered a false difference and no synchronization is triggered;

[0054] For nodes that are determined to be substantially changed, such as Figure 3As shown, the batch analysis platform is further called to count the difference edge directory node set ΔEntries between this edge directory snapshot and the last successful synchronization edge directory snapshot, and calculate the number of difference edge directory nodes DC and the historical call weights of these difference edge directory nodes. The historical call weight is obtained by aggregating operation and maintenance logs on the big data platform;

[0055] Further adopting the dual threshold classification rule to classify the synchronization requirements of each substantial change node includes: and When , the node with substantial changes is marked as lightweight synchronization; otherwise, it is marked as heavy synchronization; is the total number of directory snapshot metadata entries in the central directory system, α is the preset ratio threshold, and β is the preset weight threshold;

[0056] Specifically, the ratio threshold α and weight threshold β are automatically tuned through offline clustering analysis of historical data or Bayesian optimization algorithm to balance network overhead and update efficiency;

[0057] To further enhance the robustness of grading decisions, this embodiment also deploys an online binary classification model. After the dual-threshold rule outputs a grading result, the online binary classification model is automatically activated to perform intelligent tuning of the synchronous demand grading results. This includes: the online binary classification model calculates the grading confidence of the synchronous demand grading results obtained based on the dual-threshold rule; and compares the grading confidence with a preset confidence threshold.

[0058] If the synchronization requirement is lightweight synchronization and its hierarchical confidence is greater than or equal to the preset confidence threshold, the incremental push mode is maintained; otherwise, the synchronization mode is adjusted to full pull mode; if the synchronization requirement is heavy synchronization and its hierarchical confidence is greater than or equal to the preset confidence threshold, the full pull mode is maintained; otherwise, the synchronization mode is adjusted to incremental push mode;

[0059] It should be noted that the online binary classification model uses the number of different edge directory nodes and the historical call weights of these different edge directory nodes, the network delay between the edge directory nodes and the center, and the network packet loss rate between the edge directory nodes and the center as its characteristics. Through online training, the model parameters are continuously adjusted, and the model output is integrated with the rule classification results to generate the final synchronization requirement identification of the substantially changed node and its corresponding classification confidence.

[0060] Finally, the synchronization requirement grading results of each substantial change node are summarized to form a synchronization requirement record; the synchronization requirement record provides data input for subsequent steps, thereby ensuring the efficiency and reliability of subsequent incremental patches or full snapshot pushes; the synchronization requirement record includes the edge directory node identifier, snapshot unique identifier, the number of difference edge directory nodes, historical call weight, and grading result label and its corresponding grading confidence.

[0061] Step 2: Based on the synchronization requirement record, synchronization mode matching is performed according to its graded result label. Synchronization mode matching is performed according to the node lightweight / heavy synchronization label, and an incremental push or full pull mode operation identifier is generated. A synchronization transaction context is constructed, which includes the transaction unique identifier, mode type, and network quality information. If an ACK is not received within the confirmation timeout, a retry mechanism is triggered based on the current retry count and exponential backoff parameters.

[0062] Receive synchronization request records, which include node identifier NID, target snapshot identifier MSID, difference edge directory node number DC, and historical call weight. , average network delay , packet loss rate , grading labels and their corresponding grading confidences; further, after receiving each synchronization requirement record, idempotent deduplication and validity verification are first performed: if the current node identifier plus the target snapshot identifier has been processed or the field is incomplete, a warning log is recorded and the process is skipped; otherwise, the process proceeds to the next step;

[0063] Further synchronization pattern matching is performed between lightweight synchronization and heavy synchronization based on hierarchical labels, and pattern matching is fine-tuned by combining cross-verification rules based on the number of difference entries DC and the average network delay. The following steps are included:

[0064] The preliminary mode matching includes: when the synchronization requirement classification result is lightweight synchronization, the initial synchronization mode is set to incremental push mode; when the synchronization requirement classification result is heavy synchronization, the initial synchronization mode is set to full pull mode;

[0065] After determining the synchronization mode, a synchronization transaction context object is constructed, which uniformly encapsulates the following common fields: node identifier NID, target snapshot identifier MSID, synchronization mode, decision timestamp, confirmation timeout threshold Tconf, and message sequence number;

[0066] For incremental push mode, the synchronization transaction context is further supplemented with: transaction ID, patch version number, and base snapshot identifier;

[0067] For full pull mode, the following fields are added: idempotent snapshot ID, compressed directory entry list, and pull authorization. All these dedicated fields ensure that the edge directory node can accurately perform the corresponding incremental or full synchronization operation after receiving the synchronization transaction context.

[0068] The idempotent snapshot ID and transaction ID are mode operation identifiers of their corresponding synchronization modes respectively;

[0069] To implement a reliable retry mechanism, the exponential backoff parameters are dynamically calculated based on the current network quality and the graded confidence: initial delay ; Backoff factor ;in, is the average network delay, 1.5 is the preset initial buffer multiple in this embodiment, is the packet loss rate, The minimum waiting time allowed by the system; the maximum number of retries According to the classification confidence setting, specifically, the classification confidence is compared with the preset standard confidence; when the classification confidence is greater than or equal to the preset standard confidence, the allocation = ; When the classification confidence is less than the preset standard confidence, the allocation = ;and , and All are positive integers; and the above initial delay, backoff factor and maximum number of retries are defined as the retry field;

[0070] This configuration ensures that the delay and number of retries are appropriate to the confidence level of the classification.

[0071] After receiving the synchronization transaction context C, the directory-aware agent of the edge directory node immediately persists the synchronization transaction context to the local storage and starts the confirmation timer: it sends an ACK confirmation to the center within the confirmation timeout threshold, otherwise it is considered timed out; if the center does not receive the ACK from the specified node, it will be based on the initial delay and backoff factor Calculate the next retry delay, obtain the latest network metrics from the local or central server before retrying, and update the retry field again; each retry causes the retry field to increment automatically until the maximum number of retries is reached. ;

[0072] During the synchronization process, each synchronization transaction context delivery, ACK return from the edge directory node, and restart action are reported to the log collection system: key fields such as event type, timestamp, retry field, and latest network indicators are recorded for real-time reference in subsequent steps.

[0073] Reference Figure 4 ,Step 3: According to the mode operation identifier, drive the state model and enter the pending application state; and initiate the first transaction phase. After the node applies the corresponding synchronization mode, it switches to the applied state and reports persistently; then initiate the second transaction phase. The node performs structure and function self-tests. If it passes, it switches to the confirmed state. If it fails or times out, it executes compensation transactions and switches to the rolled back state and reports; all state transitions are logged; after the two phases are completed, anomaly detection is performed;

[0074] Receive the synchronization transaction context, extract the node identifier and mode operation identifier of the synchronization transaction context; based on the unique node identifier and mode operation identifier, query whether there is a completion record in the local lightweight storage; if it exists, discard the synchronization transaction context and do not need to repeat the execution; if it does not exist, create a state model record locally, including the current operation marked as pending and the reception time, for subsequent status tracking and delay statistics, and immediately report to the center that it has entered the pending state;

[0075] Once the local state transitions from pending execution to pending application, the state model enters the first transaction phase—the application phase. During this phase, nodes execute incremental patch applications or complete snapshot writes based on the synchronization mode type, while simultaneously collecting and reporting multi-dimensional performance metrics.

[0076] For incremental patches, the synchronization agent downloads the patch package from the center or the nearest mirror server, calculates the digest locally using the same hash algorithm, and compares it with the patch digest carried in the context package;

[0077] For complete snapshots, the entire directory snapshot file is downloaded and a digital signature verification mechanism is used to confirm the data source and integrity. During the process, the patch download time and data verification time are recorded separately to provide key data for subsequent performance baseline construction.

[0078] In incremental mode, the patch application script is executed in the file system or database to update the directory metadata information and interface definition;

[0079] In full mode, the complete snapshot file is decompressed and written to local storage, and the global hash digest is updated;

[0080] During the actual writing process, the node continuously monitors CPU utilization, memory usage, disk I / O throughput, and network bandwidth usage, and reports these resource usage to the center at a frequency of seconds, which is stored in real time through the time series database.

[0081] Determine whether the preset performance threshold is met based on the actual collected download and write duration:

[0082] If the time taken by all operations is less than the corresponding threshold and the data verification is correct, the state model determines that the application is successful and the execution state transitions to Applied;

[0083] If any step takes too long or a verification error occurs, the application is immediately deemed to have failed, the execution state transitions to Rolled Back, and the compensation process is triggered.

[0084] Each state change is written into a unified log in the form of a structured event, along with corresponding performance indicators and error information, providing sufficient basis for subsequent statistical analysis and rapid location.

[0085] When the state model enters the applied state, it automatically switches to the second transaction phase, the confirmation phase. In this phase, the node verifies that the local directory update complies with the central rules through two self-checking activities and evaluates the results through statistical analysis. These include:

[0086] The node calls the local metadata comparison service to perform an in-depth comparison of the current directory entry number, field names, and types with the central snapshot definition. It records the time taken for structure verification and the number of inconsistent items, and writes the structure consistency score and verification time into the log for subsequent trend analysis.

[0087] Use predefined interface test scripts to send typical service requests to each key interface, collect response status codes and network round-trip delay sequences, summarize average response delays and error rates in logs, and plot interface performance distribution curves to determine directory interface availability.

[0088] The self-check and state transition based on the comprehensive structure consistency score and interface error rate include:

[0089] When the structural consistency score reaches the maximum score and the error rate is lower than the preset error limit, the self-check is considered to have passed and the execution status is transferred to confirmed;

[0090] Otherwise, the self-test is determined to have failed, the execution state is transferred to rolled back, and the reason for the self-test failure and key performance indicators are recorded.

[0091] Furthermore, a large number of structured events and performance indicator logs are generated during the entire state model operation, which can be aggregated and used for real-time anomaly detection using the following methods:

[0092] Count the percentage of nodes in each state to create a state distribution dashboard. Calculate application success rate, self-test pass rate, compensation success rate, and average time consumption to provide quantitative indicators for service health assessment. Apply simple outlier detection algorithms such as Z-score or pattern matching algorithms within a continuous sliding window to identify repeated fallback-try-fallback cycles.

[0093] When the same transaction on a node with substantial changes enters the rolled-back state three times consecutively within a short period of time, a high-priority alarm is triggered and the node is marked as requiring manual intervention. Otherwise, it enters the self-healing decision-making stage.

[0094] All indicators and alarms are displayed in real time on the visual large screen.

[0095] Please refer to Figure 5,Step 4: When the substantive change node enters the rolled-back state, ,multidimensional monitoring data is collected according to the preset monitoring ,refresh cycle, and the substantive change node health index is calculated to ,perform a substantive change node health assessment; when the health score is lower than the ,threshold, a self-healing action decision is made by combining ,reinforcement learning strategy, and the selected action is issued and ,the effect is monitored, the convergence time and success rate are fed back to the ,learning pool, and the parameters are dynamically iterated and ,optimized;

[0096] Based on the preset monitoring refresh cycle, multi-source data is synchronized from the time series database and log storage service: heartbeat response records: recording the event timestamps and error codes of consecutive heartbeat successes or failures of the substantive change node, synchronization transaction delay logs containing the time it takes for the synchronization transaction context to be successfully delivered to the node application, state model event distribution, compensation execution count and time consumption statistics, and local resource utilization of the substantive change node. After the above data is aggregated in the center, it is cleaned and aligned through the ETL process to form a time series feature table for each substantive change node;

[0097] For each substantive change node j, in the latest monitoring cycle, the following indicators are extracted from the cleaned feature table: heartbeat success rate , average synchronization delay , fallback ratio , Time since last confirmation , resource usage load ; Initial assignment weights based on expert experience , calculate the node health index ; Compare the calculated results with the preset health threshold Compare, if The node with substantial changes is marked as requiring self-healing and added to the self-healing candidate list; otherwise, it is marked as a healthy node; the rollback ratio is the ratio of the number of compensation triggers to the total number of transactions;

[0098] For each node that requires self-healing and substantial changes, the optimal self-healing action is selected based on its health index and current cluster load status, combined with the action recommendations output by the reinforcement learning strategy model. Action types include:

[0099] Grayscale restart: Restart the agent process in a step-by-step manner to restore the status;

[0100] Incremental re-push: re-issue the differential patch package based on the current synchronization transaction context;

[0101] Full re-pull: Switch to full snapshot pull mode to ensure that all unapplied changes are covered;

[0102] Each decision simultaneously records the node feature vector, the selected action type, and the decision confidence;

[0103] Once a self-healing command is issued, monitor the self-healing process and collect the following key process metrics: action start and end time, action success flag, number of new rollbacks or failures during the self-healing process, and peak resource consumption for a single node and cluster.

[0104] The effect report generated by this process is pushed to the visualization screen and operation and maintenance alarm system in real time for management personnel to monitor. In addition, after the action is completed, the health index and transaction convergence rate before and after self-healing are compared to quantify the self-healing effect; the success of the action is marked by the node re-entering the confirmed state;

[0105] All self-healing action execution logs and effect reports will be written into the reinforcement learning feedback pool, forming a state-action-reward triple: ; Where s is the health feature vector of the node before self-healing, a is the self-healing action executed, and r is the quantitative indicator of the action return, such as the convergence rate improvement, the negative value of the convergence time, or the success rate;

[0106] The feedback pool is periodically taken for offline policy network training, using a deep Q network or policy gradient algorithm to calculate the new action value function Or directly output the improved strategy ;

[0107] Based on the output of the reinforcement learning policy network, the following key parameters are dynamically adjusted: snapshot generation period, hash difference threshold, confirmation timeout threshold, exponential backoff parameter, etc. The adjustment strategy follows the principle of smooth evolution: parameters are updated in batches only when multiple feedback triggers are consistent, and are first verified on a subset of nodes in a grayscale manner. The updated effect is compared with the old configuration, and the decision on whether to fully recycle the new parameters is made based on the benefit-cost ratio.

[0108] Please refer to Figure 6 ,This embodiment also includes a data collaborative directory management system, including a central directory service cluster: performing snapshot generation, time series alignment, difference detection and policy decision-making;

[0109] Edge directory node agent: deployed on each edge directory node, responsible for local hash calculation, state machine execution and self-test;

[0110] Distributed storage module: stores directory snapshots and synchronizes transaction contexts;

[0111] Reinforcement learning engine: manages the self-healing action decision-making and parameter optimization closed loop;

[0112] Visual monitoring platform: displays status distribution, health index and alarm information in real time.

[0113] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0114] The above embodiments may be implemented in whole or in part through software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0115] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application of the technical solution and the invention constraints. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0117] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0118] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data collaborative directory management method, characterized in that: The steps include: Generates a directory snapshot containing directory metadata structure, interface information and directory center hash value based on a preset period and records the directory snapshot generation timestamp; Calculate the directory node hash value of each edge directory node and record its calculation time; When the difference between the directory node hash value calculation time and the directory snapshot generation timestamp falls within the dynamic tolerance range, the corresponding directory node hash value is retained and marked as a time sequence matching record; otherwise, it is discarded; Compare the hash value of each directory node in the time series matching record with the hash value of the directory center to construct a hash difference time series. This hash difference time series is input into the isolation forest anomaly detection model to obtain the anomaly score corresponding to the hash value of each directory node. The anomaly score is then compared with the preset anomaly threshold to determine whether there is a substantial change in each edge directory node. And mark the edge directory nodes with substantial changes as substantial change nodes; For each substantially changed node, compare the corresponding current edge directory snapshot with the last successfully synchronized edge directory snapshot based on the historical edge directory snapshot information, and count the number of different edge directory nodes; The ratio of the number of difference edge directory nodes to the total number of edge directory nodes and the historical call weight are compared with the corresponding preset thresholds to obtain the synchronization demand grading results, and the synchronization mode is preliminarily matched based on the synchronization demand grading results.

2. A data collaborative directory management method according to claim 1, characterized in that: The directory center hash value is the directory hash value corresponding to the directory sender; the edge directory nodes are the directory receivers, and the directory node hash values ​​are the directory hash values ​​received by the directory receivers; the synchronization requirement grading results include lightweight synchronization requirements and heavy synchronization requirements; the synchronization modes include incremental push mode and full pull mode.

3. A data collaborative directory management method according to claim 1, characterized in that: The process of outputting the anomaly score corresponding to the hash value of each directory node by the isolation forest anomaly detection model includes: extracting the current hash difference, the sliding window moving average, the sliding window standard deviation and the overall mean of the hash difference time series from the hash difference time series, and combining them into a four-dimensional feature vector; inputting the four-dimensional feature vector into the isolation forest anomaly detection model, and obtaining the corresponding anomaly score by calculating the average path length of the four-dimensional feature vector in multiple isolated trees and normalizing it. The anomaly score is calculated according to the formula ;in, is a four-dimensional eigenvector, for The path length of each tree in the anomaly detection model, E is the average path length in the forest, c is the path length normalization factor.

4. A data collaborative directory management method according to claim 1, characterized in that: The synchronization requirement classification result is specifically obtained by the following rules: the number of difference edge directory nodes DC and the historical call weights of these difference edge directory nodes Compare it with its corresponding preset threshold; when and When , the node with substantial changes is marked as lightweight synchronization; otherwise, it is marked as heavy synchronization; is the total number of directory snapshots, α is the preset ratio threshold, and β is the preset weight threshold.

5. A data collaborative directory management method according to claim 1, characterized in that: According to the synchronization requirement grading results of each substantial change node, the preliminary synchronization mode is matched; in order to further fine-tune the selection of the preliminary synchronization mode, it also includes calculating the grading confidence of each synchronization requirement grading result, and comparing its grading confidence with the preset confidence threshold. The synchronization mode is adjusted according to the grading confidence of the synchronization requirement grading results, including: if the synchronization requirement grading result is lightweight synchronization, and its grading confidence is greater than or equal to the preset confidence threshold, then maintain the incremental push mode; otherwise, adjust the synchronization mode to the full pull mode; if the synchronization requirement grading result is heavy synchronization, and its grading confidence is greater than or equal to the preset confidence threshold, then maintain the full pull mode; otherwise, adjust the synchronization mode to the incremental push mode.

6. A data collaborative directory management method according to claim 5, characterized in that: For nodes with substantive changes that are synchronized lightly, an incremental push transaction context is constructed, including the node ID, target snapshot ID, transaction ID, patch version number, and base snapshot ID. For nodes with substantive changes that are synchronized heavily, a full pull transaction context is constructed, including the node ID, target snapshot ID, idempotent snapshot ID, and compressed directory list. The incremental push transaction context and the full pull transaction context are recorded as synchronous transaction context.

7. A data collaborative directory management method according to claim 6, characterized in that: It also includes executing substantial change node synchronization based on the synchronization transaction context using a two-stage state model, specifically the first transaction stage: the substantial change node executes the operation according to the synchronization mode, the quantity mode downloads and verifies the patch package and then updates the edge directory node, the full mode downloads the complete snapshot and overwrites the edge directory node after verifying the signature; if the operation is successful, the state transfers to applied, if it fails or times out, the state transfers to rolled back; the second transaction stage: the substantial change node executes structural self-check and functional self-check, and when the structural consistency is full and the interface error rate is lower than the preset upper limit, it is converted to confirmed, otherwise it is executed rollback.

8. A data collaborative directory management method according to claim 7, characterized in that: If the synchronization of the substantive change node in the second-stage state model fails, a retry mechanism is triggered, including: Dynamically calculate retry parameters based on the current synchronous transaction context, the retry parameters include initial delay, backoff factor and number of retries; the initial delay According to the formula: ; The backoff factor According to the formula: ;in, is the average network delay, To preset the initial buffer multiple, is the packet loss rate, is the minimum waiting time allowed by the system; and based on the initial delay and backoff factor The retry delay is calculated, and the maximum number of retries is inversely distributed according to the hierarchical confidence of the synchronization mode; retries are performed based on the retry delay and the maximum number of retries. If multiple retries are still unsuccessful, the synchronization is considered to have failed, the rollback mechanism is triggered, and the node with substantial changes is marked as rolled back.

9. A data collaborative directory management method according to claim 8, characterized in that: When the substantial change node enters the rolled back state, it also includes: based on a preset monitoring cycle, collecting health indicators of the substantial change node, including heartbeat success rate, average synchronization delay, rollback ratio, time since last confirmation, and resource occupancy load; calculating the substantial change node health value through preset weights, and determining whether a self-healing operation is required based on the substantial change node health value; if the substantial change node health value is less than the preset health threshold, the substantial change node is recorded as a node requiring self-healing and added to the self-healing candidate list; if the substantial change node health value is greater than or equal to the preset health threshold, it is marked as a healthy node; based on the substantial change node health value, the optimal self-healing operation is selected from the action set to execute the recovery operation.

10. A data collaborative directory management system, used to implement the data collaborative directory management method according to any one of claims 1 to 9, characterized in that: Includes a central directory service cluster: performs snapshot generation, time series alignment, anomaly detection, and policy decision-making; Edge directory node agent: deployed on each edge directory node, responsible for local hash calculation, state machine execution and self-test; Distributed storage module: stores directory snapshots and synchronizes transaction contexts; Reinforcement learning engine: manages the self-healing action decision-making and parameter optimization closed loop; Visual monitoring platform: displays status distribution, health index and alarm information in real time.

Images