Distributed storage method of multi-level resource data

Abstract

The application relates to the technical field of data storage, and discloses a distributed storage method for multi-level resource data, which comprises the following steps: a distributed memory node acquires consistency arbitration conflict load data and routing migration track data caused by lock conflicts, calculates a virtual topology reconstruction threshold parameter, and switches a vertical step-by-step submission addressing routing mode to a horizontal topology dependent convergence migration mode when the load data exceeds the parameter; the distributed memory node generates a mapping buffer unit and covers an underlying physical addressing pointer in a place to dissolve concurrent throughput pressure in a horizontal self-healing closed loop of an edge distributed memory pool; the application utilizes shunt adjustment to inhibit upward source return, avoids core node consistency arbitration deadlock risk, and enhances congestion resistance throughput performance.

Description

Technical Field

[0001] This invention belongs to the field of data storage technology, and in particular relates to a distributed storage method for multi-level resource data. Background Technology

[0002] Currently, in wide-area heterogeneous topology networks, mainstream solutions rely on multi-level independent database clusters matched with file server clusters to construct distributed storage space, coordinate multi-level synchronization of cross-regional resource data, and provide hierarchical storage response channels to physical terminals. In this case, the read / write throughput performance of the distributed storage medium depends on the network topology and transmission link bandwidth, and the data consistency between nodes at each level depends on the lock control logic of the coordination mechanism. When high-dimensional feature data flows horizontally across multiple levels of topology, the multi-level storage medium builds logical indexes level by level in the vertical direction, and maintains local hot and cold data by controlling the prefetching and eviction mechanism of cache pages. As the state transitions and the scale of distributed edge nodes expand, the high-frequency bursts of concurrent read and write data triggered by multi-source heterogeneous terminals exert transient physical throughput pressure on the wide-area storage topology network, causing performance conflicts in the hierarchical storage architecture. When a local hotspot bursts of traffic occur at the edge level, conventional storage solutions usually schedule data to be transmitted vertically upwards to maintain global timing, causing high-latency network transmission resistance in the backbone network. This long-latency interaction state triggers large-scale cache failures of upper-level storage media. Multiple concurrent read and write actions exacerbate hotspot congestion in the edge memory pool, bringing the potential risk of a break in the flexible adjustment chain of the storage system.

[0003] Physical link linear scalability is limited, and logical scheduling protocols for multi-process concurrent environments struggle to avoid systemic deadlocks and performance bottlenecks. For example, Chinese invention patent application CN113127210A discloses a storage management method, device, and storage medium for distributed systems. It solves the identifier conflict problem during abnormal process restarts by introducing sparse volume binding to processes and using a volume reentrant lock mechanism. However, this approach relies on maintaining lock states within a single node or under a fixed-level mapping, representing a static mapping logic patch optimization. In wide-area heterogeneous topologies across provinces, cities, counties, and schools, the central coordination node faces challenges due to massive horizontal data transfers. The resulting high-load arbitration deadlock cannot be resolved by relying on local node lock reentrancy mechanisms to eliminate backbone network throughput resistance. Furthermore, maintaining complex sparse volume mapping relationships increases memory load, and the mismatch between the underlying architecture and dynamic fluctuation conditions causes the existing technology to have fundamental flaws in extreme concurrency environments. To alleviate such congestion, linear improvement approaches such as increasing network link bandwidth or unidirectionally expanding the memory capacity of core nodes not only increase the overall construction cost but also cause high-frequency timing conflicts due to the synchronous interaction of massive concurrent data, exacerbating the strong consistency arbitration deadlock of core coordination nodes. This demonstrates that conventional design solutions have fundamental flaws that are difficult to reconcile.

[0004] Therefore, the technical problem to be solved by this invention is how to open up the local lateral addressing path of hierarchical storage nodes in a large-scale heterogeneous wide-area topology network, break the fixed routing mode of vertical upward transmission, and establish a strong causal closed loop of bidirectional dependence between physical topology hardware and dynamic scheduling rules, so as to achieve adaptive dynamic correction of physical addressing pointers under the condition of bursty concurrent read and write interleaved flow. Summary of the Invention

[0005] This invention aims to solve the problems of long backbone network latency, large-scale distributed cache failure, and core node lock arbitration deadlock caused by vertical upward transmission when multi-level heterogeneous wide-area storage topology networks face the impact of high-concurrency read and write interleaved streams.

[0006] In this technical solution, a distributed storage method for multi-level resource data includes the following steps: Step S101: The lower-level distributed memory nodes deployed in the linkage network architecture consisting of 1 provincial central platform, 18 municipal regional service centers, 158 county-level localized service centers and 28,000 school-level terminal application nodes obtain the consistency arbitration conflict load data of the central coordination node caused by lock conflicts and the routing migration trajectory characteristic data in the high-concurrency data synchronization and interaction environment of the wide-area heterogeneous topology network. Step S102: The lower-level distributed memory nodes use the consistency arbitration conflict load data of the central coordinating node and the routing migration trajectory feature data to calculate and define the virtual topology reconstruction threshold parameter. When the consistency arbitration conflict load data of the central coordinating node exceeds the virtual topology reconstruction threshold parameter, the storage scheduling architecture is switched from the vertical step-by-step addressing routing mode to the horizontal topology dependency convergence migration mode in situ. In step S103, the lower-level distributed memory node generates a mapping buffer unit based on the horizontal convergence reconstruction logic in the horizontal topology dependency convergence migration mode. The mapping buffer unit is called to cover the underlying physical addressing pointer in the vertical step-by-step addressing routing mode in situ, thereby dissolving the concurrent physical throughput pressure triggered by multi-source heterogeneous storage media into the horizontal self-healing closed loop of the edge distributed memory pool.

[0007] Preferably, step S101 includes the following sub-steps: Step S1011, the lower-level distributed memory node is deeply connected to the application terminal through the integrated management component of the application platform layer, and captures the high-frequency burst high-dimensional characteristic data stream triggered by multi-source heterogeneous terminals in real time; Step S1012, the lower-level distributed memory node divides the high-frequency burst high-dimensional characteristic data stream into time windows and monitors the load, extracts the addressing route discontinuous change parameters that characterize the risk of deadlock under the strong consistency arbitration under the lock control logic, and determines the addressing route discontinuous change parameters as the route migration trajectory characteristic data.

[0008] Preferably, the method for calculating and defining the virtual topology reconstruction threshold parameter in step S102 is as follows: the lower-level distributed memory nodes calculate the virtual topology reconstruction threshold parameter using the physical throughput load baseline value and the dynamic convergence index, and the calculation formula is: ,in, Threshold parameters for virtual topology reconstruction. The preset dynamic convergence index, The preset physical throughput load baseline value is used; when the lower-level distributed memory node detects that the current central coordinating node consistency arbitration conflict load data is greater than the virtual topology reconstruction threshold parameter, a trigger gating instruction is generated.

[0009] Preferably, in step S102, the storage scheduling architecture is switched from the vertical hierarchical addressing routing mode to the horizontal topology dependency convergence migration mode, which includes the following: when the trigger gate instruction is activated, the lower-level distributed memory node controls the flow adjustment of the addressing state, suppresses the upward back to the source, and constructs a horizontal topology virtual connectivity graph among the edge distributed memory nodes of the same level.

[0010] Preferably, step S103, which generates a mapping buffer unit based on the horizontal convergence reconstruction logic, includes the following sub-steps: Step S1031, the lower-level distributed memory node determines the horizontal migration path of the local memory page according to the horizontal topology virtual connectivity graph; Step S1032, the lower-level distributed memory node opens up a logical reconstruction storage area in the local distributed memory pool that is independent of the underlying hardware physical addressing space, and defines the logical reconstruction storage area as a mapping buffer unit.

[0011] Preferably, the method of calling the mapping buffer unit to overwrite the underlying physical addressing pointer in the vertical progressive addressing routing mode in step S103 is as follows: the lower-level distributed memory node directly writes the logical segment address of the mapping buffer unit in place and overwrites the underlying physical addressing pointer in the vertical progressive addressing routing mode, and disconnects the vertical data flow consistency arbitration channel pointing to the central coordination node.

[0012] Preferably, the method in step S103 to resolve the concurrent physical throughput pressure triggered by multi-source heterogeneous storage media through the horizontal self-healing closed loop of the edge distributed memory pool is as follows: the lower-level distributed memory nodes converge the received high-dimensional resource data streams within the localized distributed memory pool at the same level to complete the horizontal mutual backup storage of data blocks between multiple nodes, thereby avoiding deadlocks caused by high-frequency back-to-source data flow.

[0013] Preferably, after the horizontal self-healing closed loop of the edge distributed memory pool is completed, the following steps are also included: Step S104, the lower-level distributed memory nodes continuously record and save historical addressing route migration trajectory data, and count the addressing frequency based on the historical addressing route migration trajectory data. When the addressing frequency continues to rise within 10 consecutive monitoring cycles, the quota of the storage area is reconstructed in advance in the localized distributed memory pool to reconstruct the dynamic allocation structure of the edge distributed memory pool storage space.

[0014] Preferably, the distributed storage method operates in a full-dimensional security system environment. Before obtaining the consistency arbitration conflict load data and routing migration trajectory feature data of the central coordination node in step S101, the following steps are also included: Step S105, the lower-level distributed memory node calls the resource integration module, reading activity development module and data integration module through the support platform layer in the linkage network architecture to implement distributed data hierarchical governance on the stored resource data and complete encryption protection using privacy computing rules, and determines the encrypted resource data as the local storage base of the lower-level distributed memory node.

[0015] Preferably, the distributed storage method further includes the following addressing state return step: Step S106, when the distributed addressing transient disturbance data in the network interaction environment falls back to below the virtual topology reconstruction threshold parameter, the lower-level distributed memory node cancels the mapping buffer unit and restores the underlying physical addressing pointer in the vertical step-by-step delivery addressing routing mode, so that the network storage scheduling architecture returns to the original four-layer construction system.

[0016] Compared with existing technologies, the distributed storage method for multi-level resource data of the present invention has the following advantages: 1. In the distributed storage of multi-level resource data, the infrastructure layer is configured with independent user structure, object structure and media structure, and works with edge computing nodes located at the edge of the multi-level heterogeneous topology network to process the original concurrent data stream; the edge computing nodes collect unstructured interactive data generated by distributed physical terminals in situ, and extract state vectors containing hierarchical depth, access frequency and feature dimensions on site, and complete feature decoupling and local small file merging of multi-source heterogeneous data at the source of the multi-level network, reduce the data transmission load of the wide area backbone network under peak conditions, and avoid the memory hot spot congestion caused by massive high-concurrency fragmented storage requests directly impacting the core storage medium.

[0017] 2. The data exchange unit and data processing unit of the support platform layer work together to dynamically calculate the adaptive survival route weight factor based on the transient change slope parameter of the cache miss rate in the running status register. The multi-level heterogeneous topology network uses the survival route weight factor to regulate the physical addressing path of the distributed storage media at each level. When a sudden read-write interleaved traffic congestion occurs at a network node, it adaptively reconstructs the horizontal virtual topology addressing route and redirects the memory addressing pointer, diverting the physical topology pressure transmitted vertically upward to the distributed buffer nodes at the same level and adjacent levels, thus avoiding the problem of large-scale failure of the provincial center platform memory pool caused by global high-frequency cross-network physical back-to-source.

[0018] 3. A dedicated data pool tightly coupled with a standardized directory system, along with tiered storage media, establishes a distributed database cluster architecture with spatial isolation features. The architecture employs hardware-accelerated encryption algorithms and secure virtual private network tunnels to transmit multi-level synchronized data. Through multi-level self-healing closed loops, it horizontally converges physical throughput pressure, limiting the data dependencies between nodes to a local level, breaking the global time-series competition state, and avoiding deadlocks caused by lock conflicts in high-concurrency data synchronization interactions in large-scale wide-area storage topology networks, which lead to strong consistency arbitration deadlocks in the central coordination node. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the implementation steps of the distributed storage method for multi-level resource data according to the present invention. Figure 2 This is a structural diagram of the technical elements of the distributed storage method for multi-level resource data of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0021] A distributed storage method for multi-level resource data includes the following steps: Step S101: The lower-level distributed memory nodes deployed in the linkage network architecture consisting of 1 provincial central platform, 18 municipal regional service centers, 158 county-level localized service centers and 28,000 school-level terminal application nodes obtain the consistency arbitration conflict load data of the central coordination node caused by lock conflicts and the routing migration trajectory characteristic data in the high-concurrency data synchronization and interaction environment of the wide-area heterogeneous topology network. Step S102: The lower-level distributed memory nodes use the consistency arbitration conflict load data of the central coordinating node and the routing migration trajectory feature data to calculate and define the virtual topology reconstruction threshold parameter. When the consistency arbitration conflict load data of the central coordinating node exceeds the virtual topology reconstruction threshold parameter, the storage scheduling architecture is switched from the vertical step-by-step addressing routing mode to the horizontal topology dependency convergence migration mode in situ. In step S103, the lower-level distributed memory node generates a mapping buffer unit based on the horizontal convergence reconstruction logic in the horizontal topology dependency convergence migration mode. The mapping buffer unit is called to cover the underlying physical addressing pointer in the vertical step-by-step addressing routing mode in situ, thereby dissolving the concurrent physical throughput pressure triggered by multi-source heterogeneous storage media into the horizontal self-healing closed loop of the edge distributed memory pool.

[0022] Preferably, step S101 includes the following sub-steps: Step S1011, the lower-level distributed memory node is deeply connected to the application terminal through the integrated management component of the application platform layer, and captures the high-frequency burst high-dimensional characteristic data stream triggered by multi-source heterogeneous terminals in real time; Step S1012, the lower-level distributed memory node divides the high-frequency burst high-dimensional characteristic data stream into time windows and monitors the load, extracts the addressing route discontinuous change parameters that characterize the risk of deadlock under the strong consistency arbitration under the lock control logic, and determines the addressing route discontinuous change parameters as the route migration trajectory characteristic data.

[0023] Preferably, the method for calculating and defining the virtual topology reconstruction threshold parameter in step S102 is as follows: the lower-level distributed memory nodes calculate the virtual topology reconstruction threshold parameter using the physical throughput load baseline value and the dynamic convergence index, and the calculation formula is: ,in, Threshold parameters for virtual topology reconstruction. The preset dynamic convergence index, The preset physical throughput load baseline value is used; when the lower-level distributed memory node detects that the current central coordinating node consistency arbitration conflict load data is greater than the virtual topology reconstruction threshold parameter, a trigger gating instruction is generated.

[0024] Preferably, in step S102, the storage scheduling architecture is switched from the vertical hierarchical addressing routing mode to the horizontal topology dependency convergence migration mode, which includes the following: when the trigger gate instruction is activated, the lower-level distributed memory node controls the flow adjustment of the addressing state, suppresses the upward back to the source, and constructs a horizontal topology virtual connectivity graph among the edge distributed memory nodes of the same level.

[0025] Preferably, step S103, which generates a mapping buffer unit based on the horizontal convergence reconstruction logic, includes the following sub-steps: Step S1031, the lower-level distributed memory node determines the horizontal migration path of the local memory page according to the horizontal topology virtual connectivity graph; Step S1032, the lower-level distributed memory node opens up a logical reconstruction storage area in the local distributed memory pool that is independent of the underlying hardware physical addressing space, and defines the logical reconstruction storage area as a mapping buffer unit.

[0026] Preferably, the method of calling the mapping buffer unit to overwrite the underlying physical addressing pointer in the vertical progressive addressing routing mode in step S103 is as follows: the lower-level distributed memory node directly writes the logical segment address of the mapping buffer unit in place and overwrites the underlying physical addressing pointer in the vertical progressive addressing routing mode, and disconnects the vertical data flow consistency arbitration channel pointing to the central coordination node.

[0027] Preferably, the method in step S103 to resolve the concurrent physical throughput pressure triggered by multi-source heterogeneous storage media through the horizontal self-healing closed loop of the edge distributed memory pool is as follows: the lower-level distributed memory nodes converge the received high-dimensional resource data streams within the localized distributed memory pool at the same level to complete the horizontal mutual backup storage of data blocks between multiple nodes, thereby avoiding deadlocks caused by high-frequency back-to-source data flow.

[0028] Preferably, after the horizontal self-healing closed loop of the edge distributed memory pool is completed, the following steps are also included: Step S104, the lower-level distributed memory nodes continuously record and save historical addressing route migration trajectory data, and count the addressing frequency based on the historical addressing route migration trajectory data. When the addressing frequency continues to rise within 10 consecutive monitoring cycles, the quota of the storage area is reconstructed in advance in the localized distributed memory pool to reconstruct the dynamic allocation structure of the edge distributed memory pool storage space.

[0029] Preferably, the distributed storage method operates in a full-dimensional security system environment. Before obtaining the consistency arbitration conflict load data and routing migration trajectory feature data of the central coordination node in step S101, the following steps are also included: Step S105, the lower-level distributed memory node calls the resource integration module, reading activity development module and data integration module through the support platform layer in the linkage network architecture to implement distributed data hierarchical governance on the stored resource data and complete encryption protection using privacy computing rules, and determines the encrypted resource data as the local storage base of the lower-level distributed memory node.

[0030] Preferably, the distributed storage method further includes the following addressing state return step: Step S106, when the distributed addressing transient disturbance data in the network interaction environment falls back to below the virtual topology reconstruction threshold parameter, the lower-level distributed memory node cancels the mapping buffer unit and restores the underlying physical addressing pointer in the vertical step-by-step delivery addressing routing mode, so that the network storage scheduling architecture returns to the original four-layer construction system.

[0031] Example 1: In a distributed architecture consisting of one provincial central platform, 18 municipal regional service centers, 158 county-level localized service centers, and 28,000 school-level terminal application nodes, when a school-level terminal application node experiences an objective condition where the central coordinating node's consistency arbitration conflict load data exceeds the virtual topology reconstruction threshold parameter due to high-concurrency read requests, the lower-level distributed memory nodes immediately initiate a horizontal topology dependency convergence migration mode. The lower-level distributed memory nodes use the physical throughput load benchmark and dynamic convergence index to calculate the virtual topology reconstruction threshold parameter according to a preset logical relationship. When the real-time monitored central coordinating node's consistency arbitration conflict load data exceeds this virtual topology reconstruction threshold parameter, a trigger gating instruction is generated. This instruction triggers the lower-level distributed memory nodes to adjust the addressing state, suppressing upward back-to-source connections, and also adjusting the addressing state between edge distributed memory nodes at the same level. A horizontal topology virtual connectivity graph is constructed. When specifically acquiring consistency arbitration conflict load data of the central coordinating node, considering that the central coordinating node may have experienced physical high-load deadlock in an extremely high-concurrency environment, leading to the blockage of vertical communication links, the lower-level distributed memory nodes do not directly initiate status queries to the central coordinating node. Instead, they perform equivalent mapping representation by monitoring and statistically analyzing the response timeout rate of vertical data synchronization request frames sent to the central coordinating node and the length of the local retry queue in real time. Specifically, the lower-level distributed memory nodes collect the number of vertical arbitration requests that have not received a response within a unit time window, compare them with the local preset communication congestion benchmark mapping table, and reversely calculate the equivalent value of the current consistency arbitration conflict load of the central coordinating node. Thus, under the non-ideal working condition of blocked vertical channels, the conflict load data can be stably generated and updated on the edge side without relying on the reverse communication of the core node.

[0032] The lower-level distributed memory nodes determine the horizontal migration path of local memory pages based on the horizontal topology virtual connectivity graph, and open a logically reconstructed storage area in the local distributed memory pool, independent of the underlying hardware physical addressing space, as a mapping buffer unit. This mapping buffer unit overwrites the underlying physical addressing pointer in the storage scheduling architecture in situ, so that the concurrent throughput pressure is directly eliminated by the horizontal self-healing closed loop of the edge distributed memory pool. Through the above in-situ switching, the storage scheduling architecture changes from a vertical step-by-step addressing routing mode to a horizontal topology-dependent convergence migration mode. This mode, while maintaining the underlying general middleware and network physical bandwidth, realizes cross-object anti-generalization dedicated defense between nodes at all levels of the system, effectively avoiding deadlocks in the consistency arbitration of provincial core nodes caused by high-frequency origin pulls, and improving the system's anti-congestion throughput performance while maintaining the accuracy of data synchronization and interaction. The addressing link maintains stable operation under non-ideal extreme conditions. Specifically, when determining the lateral migration path of local memory pages based on the lateral topology virtual connectivity graph, the lower-level distributed memory nodes use the constructed topological adjacency matrix between peer edge distributed memory nodes as the underlying graph theory control model. Each element in the topological adjacency matrix represents the weighted cost term of real-time communication latency and remaining memory bandwidth between two adjacent memory nodes. Starting from the current set of locally overloaded memory pages, the lower-level distributed memory nodes call the shortest path algorithm to calculate a sequence of adjacent nodes on the virtual connectivity graph that minimizes the overall transmission cost. The target node sequence and the corresponding memory block transmission step size are defined as the lateral migration path of local memory pages, thereby outputting clear data flow guidance and providing accurate address routing basis for opening mapping buffer units in the logically reconstructed storage area.

[0033] Example 2: This example verifies the congestion resistance and storage reliability of a distributed storage method for multi-level resource data under extremely high concurrency conditions. A simulation verification platform with one central node and 18 edge nodes is constructed to simulate the actual business environment of a library cloud platform. The platform's physical throughput load baseline is 1000MB / s, the dynamic convergence index is set to 0.8, and the virtual topology reconstruction threshold parameter is calculated to be 800MB / s through preset logic. Three sample groups are set up in the experiment: the control sample group adopts the conventional vertical submission mode, the intermediate sample group includes trigger gating logic but does not enable lateral topology dependency convergence migration, and the sample group of this invention enables the lateral topology dependency convergence migration mode after trigger gating. This invention establishes a throughput stagnation measurement mechanism caused by lock conflicts by mapping the lock conflict load, which belongs to the logical state level, to the physical throughput dimension. The lower-level distributed memory nodes monitor the total number of local memory pages that are in a suspended state due to waiting for strong consistency lock arbitration by the central coordinating node within a unit of time. The data flow speed corresponding to these blocked and unwriteable memory pages is converted into an equivalent physical throughput resistance loss value. In this way, the abstract lock arbitration deadlock risk is normalized into central coordinating node consistency arbitration conflict load data in MB / s, so that the arbitration load caused by lock conflicts is comparable to the virtual topology reconstruction threshold parameter calculated based on the physical throughput load benchmark value on the same physical dimension.

[0034] During the experiment, simulated high-concurrency interactive requests were continuously injected into the system. When the central coordinating node measured the consistency arbitration conflict load data to reach 750MB / s, each sample group showed a certain degree of latency, and the system maintained vertical routing addressing. When the request load increased and caused the load data to climb to 850MB / s and exceed the virtual topology reconstruction threshold parameter, the sample group of this invention automatically enabled the mapping buffer unit to cover the underlying physical addressing pointer in situ, and redirected the data access path to the edge distributed memory pool. It was observed that the single-node data back-to-source transmission frequency of this sample group decreased by about 85% compared with the control sample group, and the lock contention latency of consistency arbitration was shortened from 450ms in the control sample group to 55ms.

[0035] Further extreme boundary tests were conducted on the prototype of this invention. Load data was set at three gradients: 500MB / s, 800MB / s, and 1200MB / s, for cyclical verification. Under a load of 500MB / s, the system did not trigger the reconstruction mechanism, and the throughput performance was consistent with the control prototype. At the critical point of 800MB / s, the system completed the virtual connectivity graph construction in 12.4ms, with a smooth transition in data addressing paths. Under a high load of 1200MB / s, the prototype of this invention still maintained the horizontal self-healing closed loop of the edge distributed memory pool, and no core arbitration deadlock occurred. In comparison, the control prototype showed superior performance. The sample group experienced a complete deadlock when the load reached 960MB / s. Experimental data confirmed that when the arbitration load data caused by lock conflicts was at the lower limit of the preset range (500MB / s), the storage system maintained normal scheduling with minimal resource overhead. However, when the load data exceeded the virtual topology reconstruction threshold parameter, the horizontal topology dependency convergence migration mode avoided consistency arbitration pressure on core nodes through a traffic splitting adjustment mechanism. The data synchronization interaction stability and throughput efficiency exhibited by the sample group throughout the entire pressure gradient range verified the effectiveness of in-situ mapping buffer units in high-concurrency heterogeneous network environments. The switching technology can effectively realize the dynamic reconstruction and self-healing of storage resources. Its performance curve did not exhibit non-linear degradation after exceeding the critical threshold, confirming that the defined storage architecture and scheduling strategy have clear engineering feasibility and technical superiority. In the implementation of this invention, the preset physical throughput load benchmark value P is determined based on the theoretical maximum throughput bandwidth of the physical network interface cards of the distributed memory nodes, combined with historical normal peak traffic through calibration tests. In this embodiment, it is set to 1000MB / s. The dynamic convergence index γ is limited to a range of 0.5 to 0.9, and its selection is based on the adjustment... The sensitivity of the reconfiguration mechanism is controlled. If γ is lower than 0.5, the virtual topology reconfiguration threshold parameter S will be too low, and the system will be prone to frequent false triggering of mode switching, thereby increasing the horizontal synchronization communication overhead between peer nodes and causing resource waste. If γ is higher than 0.9, the threshold parameter S will approach the physical limit of the network card, causing the system to trigger reconfiguration only after the network is completely paralyzed or deadlock occurs, losing the flexible adjustment and protection function of edge self-healing closed loop. Therefore, this embodiment selects the intermediate value of 0.8 after engineering simulation optimization to achieve the best system elasticity balance between preventing false triggering and forward defense.

[0036] Example 3: In a network architecture comprising 158 county-level localized service centers and 28,000 school-level terminal application nodes, when the communication link quality of school-level terminal application nodes deteriorates due to equipment aging or random jitter in the local network topology, the system monitors in real time that the connectivity response time between distributed nodes fluctuates frequently, triggering invalid storage route update attempts at the central coordination node and causing unnecessary consistency arbitration conflicts. To address this challenge of frequent disturbances at the logical scheduling layer due to physical layer quality degradation, the present invention triggers an adaptive edge defense procedure in the lower-level distributed memory nodes. This procedure monitors the communication interaction sequence between nodes in real time, extracts the first derivative of the link response time as a mutation detection index, and compares the first derivative value with a preset link jitter threshold. If the index exceeds the threshold for three consecutive cycles, the system will detect the mutation. If the value is not specified, the current link is determined to be in an unstable state, thereby triggering a local route locking action. During the operation of this adaptive edge defense procedure, the lower-level distributed memory nodes capture high-dimensional feature data streams through the integrated management component and further extract the addressing route discontinuous change parameters. Specifically, within a fixed time window, the lower-level distributed memory nodes continuously record the logical span of the addressing pointer jumping between topology nodes at each level. By calculating the first-order difference of the absolute value of the logical level difference between two adjacent addressings, the degree of discrete jump in the vertical and horizontal dimensions of the addressing route is quantitatively characterized. If the first-order difference value experiences a nonlinear sudden increase within a single window, the weight of the sudden increase feature term is set to the highest and transformed into addressing route discontinuous change parameters characterizing the risk of strong consistency arbitration deadlock through a normalization algorithm, serving as the deterministic input for subsequent determination of route migration trajectory feature data.

[0037] After triggering the route locking action, the lower-level distributed memory nodes will no longer send regular link update data packets to the central coordinating node. Instead, based on the locally maintained horizontal topology virtual connectivity graph, they will redirect the read / write request address mappings originally pointing to the central coordinating node to the peer-level edge distributed memory nodes. They will then utilize the redundant backup replicas built into the edge distributed memory pool to respond to data requests. This response process is synchronized through the local cache invalidation judgment logic of the edge nodes, ensuring that data consistency updates are completed only within the horizontal self-healing closed loop between edge nodes. This shields the central coordinating node from logical interference from underlying communication jitter. When subsequent monitoring detects that the first derivative of the link response time returns to a stable range and the duration reaches the preset safety window period, the system automatically releases the route locking state and resumes vertical connection. The hierarchical delivery addressing routing mode enables flexible adaptive adjustment of the system under scenarios of fluctuating communication quality. Experimental verification data shows that, under non-ideal conditions where the simulated link response time fluctuation reaches 30% and the packet loss rate rises to 5%, the sample group of the present invention using the above-mentioned routing locking mechanism reduces the average request response latency of the core central coordination node by 62% compared with the conventional vertical delivery mode. Furthermore, the frequency of consistency arbitration conflict record generation caused by physical layer fluctuations is reduced from 180 times per minute to 12 times per minute. This proves that the adopted link response indicators and their judgment logic can effectively isolate the logical layer load fluctuations caused by physical layer link instability, ensuring that the distributed storage system maintains the operational stability of the storage scheduling architecture and the reliability of resource access in complex and ever-changing edge network environments.

[0038] Example 4: In the school-level terminal application node cluster architecture, for large-scale data storage tasks, a baseline parameter model is established for the distributed memory pool, which includes preset data filling rules and a discretization calibration process. This model is used to eliminate the random response latency of distributed nodes during the data writing phase. Before the system formally executes the multi-level resource data storage process, the distributed memory pool is first calibrated offline. This calibration procedure uses a specific physical throughput load as a benchmark. One hundred benchmark test blocks of 1MB each are written to the edge nodes, and the average write response time and link occupancy rate of each edge node are recorded. The above data is used as the storage capacity fingerprint of the batch of nodes. Then, based on the linear proportional relationship between write response time and link occupancy rate, a load balancing parameter matrix of the distributed memory nodes is constructed. This parameter matrix is ​​fixed in the local storage control module of each edge node and serves as the core criterion for dynamic resource reconstruction during the operation of the distributed storage system.

[0039] When the distributed storage system receives a sudden surge in resource access requests, the distributed memory nodes invoke the aforementioned parameter matrix to calculate the storage migration tendency value in real time based on the current load status of each edge node. The specific calculation path involves reading the current remaining physical memory and allocated logical storage capacity of each edge node to calculate the physical memory occupancy rate. This occupancy rate is then substituted into the linear response function established by the offline calibration procedure to output resource allocation weight coefficients for each edge node. Finally, based on the differences in these weight coefficients, the requested data block is directed to the edge node with the highest weight, ensuring that data write operations can occur within the distributed memory pool. To achieve uniform distribution and avoid cache failures under extreme pressure, the system pre-configures boundary discretization processing logic. When the physical memory occupancy of an edge node exceeds 95%, its resource allocation weight coefficient is automatically corrected to 0, thereby forcibly stopping the writing of new data to the overloaded node and triggering smooth data migration between edge nodes. Verification shows that under the condition of request load fluctuation of 20%, the average execution latency of the memory page lateral migration process is controlled within 10ms to 15ms, and the memory occupancy deviation between edge nodes of the distributed memory pool is reduced from 15% to 2%, achieving the expected engineering effect of storage balanced scheduling.

[0040] Example 5: In the preset construction scenario of the distributed memory pool, in order to ensure the storage consistency of multi-level resource data when accessing across regions, the system adopts a distributed storage resource pre-scheduling calibration procedure based on link quality weight. The system deploys a probe signal generator inside the distributed memory pool and sends data probe packets to adjacent distributed memory nodes at a period of 500ms. Each receiving distributed memory node records the round-trip transmission delay and packet loss rate of the probe signal and feeds the data back to the scheduling and control unit of the distributed memory pool. The scheduling and control unit performs standardized quantization processing on the transmission delay and packet loss rate according to the transmission quality feedback of each physical link, and calculates the comprehensive communication stability index of each link.

[0041] In the calculation logic of the comprehensive communication stability index, a response time index with a weight of 0.8 and a packet loss rate index with a weight of 0.2 are weighted and aggregated. When the index value is lower than 0.65 for three consecutive preset measurement periods, the distributed memory pool automatically determines that the corresponding physical link has a risk of resource scheduling congestion. At this time, the scheduling control unit initiates a memory page lateral remapping instruction to the edge distributed memory nodes with lighter loads based on the total remaining physical memory of each edge distributed memory node and the current logical storage mapping relationship. By generating real-time data backups in the logical reconstruction storage area opened in the remapping target node, a redundant storage baseline with lateral topology dependency is established. This allows the system to complete the processing of distributed memory read and write requests under high concurrency before the physical link communication stability index recovers to above 0.8. This effectively avoids the risk of deadlock in the consistency arbitration of the central coordination node caused by instantaneous fluctuations in the physical link, and ensures the multi-level resource data. To ensure access continuity in a distributed storage environment, the data exchange unit supporting the platform layer also monitors the transient change slope parameter of the cache miss rate in real time through the running status register. Specifically, it calculates the ratio of the difference in cache miss rate within adjacent monitoring periods to the time interval to capture the sudden increase trend of traffic surges. The data processing unit fuses this slope parameter with the comprehensive communication stability index using multi-dimensional features to calculate the adaptive survival routing weight factor used to regulate the addressing paths of storage media at all levels. Specifically, when the transient change slope parameter of the cache miss rate exceeds the preset slope threshold and the communication stability index decreases, the adaptive survival routing weight factor is dynamically reduced, thereby forcibly reducing the addressing priority of the congested path and driving the addressing pointer to redirect and divert traffic to distributed buffer nodes with higher weight factors in the same or adjacent levels. Thus, in addition to the above remapping instructions, a flexible dynamic routing scheduling closed loop based on register micro-state awareness is provided.

[0042] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit of this application and the scope of protection of this invention, and all of these forms are within the protection scope of this application.

Claims

1. A distributed storage method for multi-level resource data, characterized in that, Includes the following steps: Step S101: The lower-level distributed memory nodes deployed in the linkage network architecture consisting of 1 provincial central platform, 18 municipal regional service centers, 158 county-level localized service centers and 28,000 school-level terminal application nodes obtain the consistency arbitration conflict load data of the central coordination node caused by lock conflicts and the routing migration trajectory characteristic data in the high-concurrency data synchronization and interaction environment of the wide-area heterogeneous topology network. Step S102: The lower-level distributed memory nodes use the consistency arbitration conflict load data of the central coordinating node and the routing migration trajectory feature data to calculate and define the virtual topology reconstruction threshold parameter. When the consistency arbitration conflict load data of the central coordinating node exceeds the virtual topology reconstruction threshold parameter, the storage scheduling architecture is switched from the vertical step-by-step addressing routing mode to the horizontal topology dependency convergence migration mode in situ. In step S103, the lower-level distributed memory node generates a mapping buffer unit based on the horizontal convergence reconstruction logic in the horizontal topology dependency convergence migration mode. The mapping buffer unit is called to cover the underlying physical addressing pointer in the vertical step-by-step addressing routing mode in situ, thereby dissolving the concurrent physical throughput pressure triggered by multi-source heterogeneous storage media into the horizontal self-healing closed loop of the edge distributed memory pool.

2. The distributed storage method for multi-level resource data according to claim 1, characterized in that, Step S101 includes the following sub-steps: Step S1011, the lower-level distributed memory node deeply connects with the application terminal through the integrated management component of the application platform layer, and captures in real time the high-frequency burst high-dimensional characteristic data stream triggered by multi-source heterogeneous terminals; Step S1012, the lower-level distributed memory node divides the high-frequency burst high-dimensional characteristic data stream into time windows and monitors the load, extracts the addressing route discontinuous change parameters that characterize the risk of deadlock under the strong consistency arbitration under the lock control logic, and determines the addressing route discontinuous change parameters as the route migration trajectory characteristic data.

3. The distributed storage method for multi-level resource data according to claim 1, characterized in that, The method for calculating and defining the virtual topology reconstruction threshold parameter in step S102 is as follows: the lower-level distributed memory nodes calculate the virtual topology reconstruction threshold parameter using the physical throughput load baseline value and the dynamic convergence index. The calculation formula is as follows: ,in, Threshold parameters for virtual topology reconstruction. The preset dynamic convergence index, The preset physical throughput load baseline value is used; when the lower-level distributed memory node detects that the current central coordinating node consistency arbitration conflict load data is greater than the virtual topology reconstruction threshold parameter, a trigger gating instruction is generated.

4. The distributed storage method for multi-level resource data according to claim 3, characterized in that, Step S102, which involves switching the storage scheduling architecture from the vertical progressive addressing routing mode to the horizontal topology-dependent convergence migration mode, includes the following: When the gating instruction is activated, the lower-level distributed memory node controls the routing adjustment of the addressing state, suppresses upward back-to-source, and constructs a horizontal topology virtual connectivity graph among the edge distributed memory nodes of the same level.

5. A distributed storage method for multi-level resource data according to claim 4, characterized in that, Step S103, which generates a mapping buffer unit based on the horizontal convergence reconstruction logic, includes the following sub-steps: Step S1031, the lower-level distributed memory node determines the horizontal migration path of the local memory page according to the horizontal topology virtual connectivity graph; Step S1032, the lower-level distributed memory node opens up a logical reconstruction storage area in the local distributed memory pool that is independent of the underlying hardware physical addressing space, and defines the logical reconstruction storage area as a mapping buffer unit.

6. The distributed storage method for multi-level resource data according to claim 1, characterized in that, The method of calling the mapping buffer unit to overwrite the underlying physical addressing pointer in the vertical progressive addressing routing mode in step S103 is as follows: the lower-level distributed memory node directly writes the logical segment address of the mapping buffer unit in place and overwrites the underlying physical addressing pointer in the vertical progressive addressing routing mode, and disconnects the vertical data flow consistency arbitration channel pointing to the central coordination node.

7. The distributed storage method for multi-level resource data according to claim 1, characterized in that, The method in step S103 to resolve the concurrent physical throughput pressure triggered by multi-source heterogeneous storage media through the horizontal self-healing closed loop of the edge distributed memory pool is as follows: the lower-level distributed memory nodes converge the received high-dimensional resource data streams within the localized distributed memory pool at the same level to complete the horizontal mutual backup storage of data blocks between multiple nodes, thereby avoiding deadlocks caused by high-frequency back-to-source data flow.

8. A distributed storage method for multi-level resource data according to claim 5, characterized in that, After the horizontal self-healing closed loop of the edge distributed memory pool is completed, the following steps are also included: Step S104, the lower-level distributed memory nodes continuously record and save historical addressing route migration trajectory data, and count the addressing frequency based on the historical addressing route migration trajectory data. When the addressing frequency continues to rise within 10 consecutive monitoring periods, the quota of the storage area is reconstructed in advance in the local distributed memory pool to reconstruct the dynamic allocation structure of the edge distributed memory pool storage space.

9. A distributed storage method for multi-level resource data according to claim 1, characterized in that, The distributed storage method operates in a full-dimensional security system environment. Before obtaining the consistency arbitration conflict load data and routing migration trajectory feature data of the central coordination node in step S101, the following steps are also included: Step S105, the lower-level distributed memory nodes call the resource integration module, reading activity development module and data integration module through the support platform layer in the linkage network architecture to implement distributed data hierarchical governance on the stored resource data and complete encryption protection using privacy computing rules, and determine the encrypted resource data as the local storage base of the lower-level distributed memory nodes.

10. A distributed storage method for multi-level resource data according to claim 1, characterized in that, The distributed storage method also includes the following addressing state return step: Step S106, when the distributed addressing transient disturbance data in the network interaction environment falls back to below the virtual topology reconstruction threshold parameter, the lower-level distributed memory node cancels the mapping buffer unit and restores the underlying physical addressing pointer in the vertical step-by-step delivery addressing routing mode, so that the network storage scheduling architecture returns to the original four-layer construction system.

Images