Distributed quotation data collaborative pushing method and system

Abstract

The invention provides a distributed quotation data collaborative pushing method and system. The method comprises the following steps: establishing a dynamic association model with a network congestion trend by collecting transmission path environment fluctuation data, and generating environment network association parameters; real-time quotation data is cut by adopting double dimensions of a geographic grid boundary and a commodity classification hierarchy, and dynamic binding of data blocks and storage nodes is realized through a scalable hash ring; generating a dynamic routing strategy based on the environmental parameter energy integral, the geographic grid coverage density and the commodity level update rate, and calculating a transmission path priority by combining a geographic extension direction and a data volume growth trend; and constructing a path state evolution model, evaluating a node failure probability and spectrum energy attenuation, and when a path correction factor exceeds a threshold value, triggering a multi-channel push mechanism. According to the technical scheme provided by the invention, distributed quotation data pushing has a network state self-adaptive multi-path cooperation capability.

Description

Technical Field

[0001] The present application relates to the field of distributed network communication technology, and in particular to a method and system for collaboratively pushing distributed quotation data. Background Art

[0002] In cross-regional businesses like financial transactions and logistics tracking, the system must support real-time sharded storage and millisecond-level multi-node synchronization of massive amounts of dynamic quote data, such as global commodity price fluctuations and cross-border exchange rate updates. These businesses require sharded data storage that balances regional relevance and load balancing, while also ensuring strong data consistency and low latency through a cross-data center collaborative push mechanism.

[0003] Currently, some enterprises use a centralized master node sharding and batch broadcast synchronization mechanism. This solution statically partitions data into a fixed number of storage nodes using predefined rules. Each region deploys a master node to receive local write requests and relies on a centralized control node to maintain the global shard routing table.

[0004] However, the core problem with this solution lies in the coupling between a centralized architecture and static policies. Predefined sharding rules cannot dynamically detect changes in service load, making sudden traffic spikes easily overload a single master node. Furthermore, shard expansion requires downtime and data migration, causing service interruptions. The basic message queue lacks priority scheduling and adaptive compression capabilities, leading to severe contention for network bandwidth in high-concurrency scenarios. Summary of the Invention

[0005] The present application provides a distributed quotation data collaborative push method and system to solve the problem of low network transmission efficiency in the prior art.

[0006] In a first aspect, the present application provides a distributed quotation data collaborative push method, comprising:

[0007] Collecting environmental fluctuation data of a data transmission path, and dynamically correlating the environmental fluctuation data with a changing trend of a network congestion index to generate an environmental network correlation parameter;

[0008] Divide the real-time updated quotation data into independent data blocks according to the boundaries of the geographic grid and the ownership relationship of the commodity classification level nodes, and dynamically bind the independent data blocks to storage nodes through a scalable hash ring to generate a redundant storage topology;

[0009] Generate a distributed routing strategy based on the energy integration result of the environmental network associated parameters, combined with the coverage density of the geographic grid and the update rate of the commodity classification level;

[0010] collaboratively calculating the transmission path selection priority of each of the independent data blocks based on the communication weight distribution in the distributed routing strategy, combined with the boundary expansion direction of the geographic grid and the data volume growth trend of the commodity classification level;

[0011] A path state evolution model is established based on the node failure probability of the redundant storage topology and the spectrum energy attenuation rate of the environmental network-related parameters, and a path correction factor is generated according to the path state evolution model. When the path correction factor exceeds a preset threshold, the independent data block is pushed to multiple parallel transmission channels in a preset geographical direction according to the transmission path selection priority.

[0012] Optionally, a communication weight distribution set is extracted from a storage node in the distributed routing strategy, a vector decomposition is performed on the boundary extension direction of the geographic grid to generate a main direction component set, and a trend slope parameter is calculated based on an incremental sequence of adjacent time window data at the commodity classification level;

[0013] Inputting the amplitude of the main direction component set and the trend slope parameter into a distributed collaborative computing node, generating a dynamic superposition coefficient set through a product operation, and superimposing the dynamic superposition coefficient set onto the communication weight distribution set according to the cell identifier of the geographic grid to form a dynamically adjusted communication weight distribution;

[0014] Based on the dynamically adjusted communication weight distribution, combined with the coverage density, the dynamic superposition coefficient set, the depth of the commodity classification level node and the real-time update frequency of the independent data block, the transmission path selection priority is generated through the distributed collaborative computing node.

[0015] Optionally, obtaining a mapping relationship table between the unit identifiers of the geographic grid and the communication weight distribution set, and extracting original communication weight values ​​corresponding to the unit identifiers of the geographic grid based on the mapping relationship table;

[0016] Determining a combined ratio of the dynamic superposition coefficient and the original communication weight value based on an inverse relationship between the coverage density of the geographic grid and the depth of the commodity classification level node;

[0017] Matching the unit identifiers in the dynamic superposition coefficient set with the unit identifiers of the geographic grid in the mapping relationship table, and for the successfully matched unit identifiers, proportionally superimposing the corresponding dynamic superposition coefficients and the original communication weight values ​​according to the merging ratio;

[0018] The communication weight values ​​after the superposition processing are rewritten into the communication weight distribution set according to the cell identifiers of the geographic grid, so as to generate a dynamically adjusted communication weight distribution including the correlation characteristics of the geographic expansion direction and the data growth trend.

[0019] Optionally, a communication density parameter is generated by performing a product operation on the communication weight value of the geographic grid unit and the coverage density, and a level depth parameter is generated by performing an inverse proportional conversion between the dynamic superposition coefficient and the depth of the commodity classification level node;

[0020] fusing the communication density parameter, the layer depth parameter, and the update intensity parameter generated by the real-time update frequency of the independent data block according to a preset ratio to generate a dynamic synthesis parameter;

[0021] In the distributed collaborative computing node, a transmission path selection priority of an independent data block in a geographic grid unit is generated based on the composite ranking of the dynamic synthesis parameters.

[0022] Optionally, obtaining a real-time failure probability value of each storage node in the redundant storage topology and a spectrum energy attenuation rate value of a corresponding transmission path in the environmental network association parameter;

[0023] Dynamically assigning a superposition weight of the failure probability value according to the ratio of the historical online time of the storage node to the current spectrum energy value;

[0024] Dynamically combining the failure probability value with the spectrum energy decay rate value according to the assigned superposition weight to generate a path evolution parameter sequence for each transmission path;

[0025] Based on the numerical distribution characteristics of the path evolution parameter sequence, a path state evolution model is established by using a preset evolution parameter interval division rule, wherein the path state evolution model divides the numerical interval into a stable state evolution region, a fluctuating state evolution region, and a failure state evolution region;

[0026] When the value of the path evolution parameter sequence falls into the wave state evolution zone or the failure state evolution zone, a path correction factor is generated according to the deviation amplitude of the value from the critical value of the corresponding evolution zone.

[0027] Optionally, monitoring a current value of the path correction factor, and when the current value is greater than a preset correction threshold, determining a push order of the independent data blocks to be pushed according to the ranking in the transmission path selection priority list;

[0028] Extracting a set of storage nodes associated with the preset geographical direction from the redundant storage topology according to the coverage range of the preset geographical direction, and allocating at least one transmission channel to each storage node in the set of storage nodes;

[0029] Segmenting the independent data blocks to be pushed to generate segmented data blocks equal to the number of transmission channels, and sequentially marking the segmented data blocks with corresponding segment numbers according to the pushing order;

[0030] The segment number is dynamically bound to the channel identifier of the transmission channel to generate a segment channel mapping table, and according to the segment channel mapping table, multiple segment data blocks of the same independent data block are synchronously pushed to multiple parallel transmission channels in a preset geographical direction through corresponding transmission channels.

[0031] Optionally, based on the rectangular boundary coordinates of the geographic grid, the center point coordinates of each geographic grid and the path depth of the commodity classification level node are extracted;

[0032] When the geographic coordinates of the real-time updated quotation data fall within the rectangular boundary of any geographic grid, the product value of the center point coordinates of the geographic grid and the path depth of the commodity classification level node is calculated. If the product value is less than a preset threshold, the quotation data is attributed to the independent data block corresponding to the current geographic grid and commodity classification level node;

[0033] generating a segment identification code for each of the independent data blocks, constructing a data structure of the scalable hash ring, and inputting the segment identification code into a key-value mapping function of the scalable hash ring to calculate a mapping angle value of the segment identification code on the scalable hash ring;

[0034] The storage node corresponding to the first virtual mapping point containing the mapping angle value in the clockwise direction is selected as the primary storage node, and at least two storage nodes separated from the primary storage node by a fixed angle on the scalable hash ring are selected as secondary storage nodes to generate a redundant storage topology bound to the primary and secondary storage nodes.

[0035] Optionally, for each geographical grid, cumulatively calculating energy integration results of the environmental network associated parameters, and superimposing coverage density values ​​of the geographical grids to generate a routing weight for each geographical grid;

[0036] Multiplying the routing weight by the update rate of the same commodity classification level to obtain a dynamic weight coefficient for each commodity classification level, and dividing the routing levels according to the numerical range of the dynamic weight coefficient;

[0037] Allocating a preset number of candidate routing paths to each routing level, and determining a set of transmission directions of the candidate routing paths based on the geographical distribution direction of the storage nodes in the redundant storage topology and the boundary extension direction of the geographical grid;

[0038] The routing level, transmission direction set and corresponding dynamic weight coefficient are associated and stored to generate a distributed routing strategy.

[0039] Optionally, physical interference intensity data and network delay fluctuation data in the environmental fluctuation data are collected, fluctuation peak moments of the two are extracted according to the same time window, and time matching is calculated;

[0040] Proportionally adjusting the fluctuation amplitude of the physical interference intensity data and the sudden increase amplitude of the network delay fluctuation data based on the time matching degree, so that the fluctuation magnitudes of the two are mapped to a unified numerical range;

[0041] The adjusted physical interference fluctuation amplitude and network delay sudden increase amplitude are input into the dynamic weight allocator, and the fusion ratio weight of the two is automatically adjusted according to the density distribution of matching events in the time window to generate the environment network correlation parameter.

[0042] In a second aspect, the present application provides a distributed quotation data collaborative push system, including:

[0043] A collection module collects environmental fluctuation data of the transmission path, dynamically associates the environmental fluctuation data with a change trend of the network congestion index, and generates an environmental network correlation parameter;

[0044] The partitioning module divides the real-time updated quotation data into independent data blocks according to the boundaries of the geographic grid and the ownership relationship of the commodity classification level nodes, and dynamically binds them to the storage nodes through a scalable hash ring to generate a redundant storage topology;

[0045] A generation module generates a distributed routing strategy based on the energy integration result of the environmental network associated parameters, combined with the coverage density of the geographic grid and the update rate of the commodity classification level;

[0046] a calculation module for collaboratively calculating, based on the communication weight distribution in the distributed routing strategy and in combination with the boundary expansion direction of the geographic grid and the data volume growth trend of the commodity classification level, a transmission path selection priority for each of the independent data blocks;

[0047] A push module establishes a path state evolution model based on the node failure probability of the redundant storage topology and the spectrum energy attenuation rate of the environmental network-related parameters, generates a path correction factor according to the path state evolution model, and when the path correction factor exceeds a preset threshold, pushes the independent data block to multiple parallel transmission channels in a preset geographical direction according to the transmission path selection priority.

[0048] In an embodiment of the present application, environmental fluctuation data of a data transmission path is collected and dynamically associated with a changing trend of a network congestion index to generate an environmental network association parameter. Real-time updated quotation data is divided into independent data blocks based on the boundaries of a geographic grid and the ownership relationship of nodes at a commodity classification level, and the independent data blocks are dynamically bound to storage nodes via a scalable hash ring to generate a redundant storage topology. A distributed routing strategy is generated based on the energy integration result of the environmental network association parameter, combined with the coverage density of the geographic grid and the update rate of the commodity classification level. A transmission path selection priority of each independent data block is collaboratively calculated based on the communication weight distribution in the distributed routing strategy, combined with the boundary expansion direction of the geographic grid and the data volume growth trend of the commodity classification level. A path state evolution model is established based on the node failure probability of the redundant storage topology and the spectrum energy decay rate of the environmental network association parameter. A path correction factor is generated based on the path state evolution model. When the path correction factor exceeds a preset threshold, the independent data block is pushed to multiple parallel transmission channels in a preset geographic direction according to the transmission path selection priority.

[0049] This application has the following beneficial effects:

[0050] By dynamically correlating network environment fluctuations and congestion trends, the real-time adaptability and congestion avoidance capabilities of transmission paths are improved; data block sharding and scalable hash ring binding based on geographic grids and commodity classifications enable elastic expansion and load balancing of storage nodes, enhancing local disaster recovery capabilities of regional hotspots; dynamic routing strategies are generated by integrating environmental parameters with business characteristics, optimizing priority scheduling for cross-domain transmission and reducing the cumulative delay of multi-hop transmission; combining path state evolution models with correction factors, fault prediction and self-healing switching of transmission channels are achieved, ensuring push success rate and timeliness stability in high-concurrency scenarios.

[0051] Furthermore, by extracting the principal component vectors of the communication weight distribution and the direction of geographic grid boundary expansion, combined with the incremental trend slope parameters of the commodity classification level, an overlay coefficient is dynamically generated and the communication weight distribution is corrected. Ultimately, multi-dimensional parameters such as coverage density, layer depth, and update frequency are integrated to achieve collaborative calculation of transmission path priority. This mechanism can dynamically perceive the direction of business growth and network resource distribution. Through weight overlay and multi-factor fusion, it adaptively adjusts path priority, achieving bandwidth tilt allocation in hot spots and redundancy avoidance in low-activity areas in cross-domain transmission. This improves path selection accuracy in high-concurrency scenarios, reduces the probability of network congestion, and mitigates end-to-end latency fluctuations caused by path rigidity.

[0052] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A flowchart of a distributed quotation data collaborative push method provided by the present application is shown;

[0055] Figure 2 The figure shows a structural diagram of a distributed quotation data collaborative push system provided by the present application. DETAILED DESCRIPTION

[0056] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0057] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0058] Researchers have discovered that the sharding strategies of existing distributed quotation systems are out of sync with network fluctuations, and cross-domain push relies on fixed routing rules, leading to problems such as poor adaptability to dynamic loads, high synchronization latency, and inefficient disaster recovery switching. Based on this, they propose a distributed collaborative quotation data push method that dynamically adjusts transmission paths through real-time correlation modeling of environmental fluctuations and network congestion. This method also integrates multi-dimensional sharding rules based on geographic grids and commodity classifications to achieve elastic storage of data chunks and priority-driven, low-latency push.

[0059] The technical solution of this application can be applied to distributed data collaboration scenarios such as financial transactions and cross-border e-commerce that need to support high-frequency quotation updates, strong cross-regional consistency and burst traffic scheduling.

[0060] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0061] Figure 1 A flowchart of a distributed quotation data collaborative push method is provided for an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0062] 101. Collect environmental fluctuation data of a data transmission path, and dynamically associate the environmental fluctuation data with a change trend of a network congestion index to generate an environmental network correlation parameter;

[0063] Among them, environmental fluctuation data is a set of indicators reflecting the real-time status of the transmission path collected by distributed network probes, including bandwidth fluctuation rate, link delay jitter amplitude, packet loss rate peak, etc. These data are used to characterize the dynamic changes of the network transmission environment; the network congestion index is a quantitative indicator based on long-short-term memory network training, which predicts the current network congestion level by analyzing the load rate, queue cache overflow probability and routing hop count change trend in historical traffic data; the environmental network correlation parameter is a parameter matrix generated by using a dynamic graph convolutional network to perform correlation analysis on environmental fluctuation data and network congestion index, which is used to describe the dynamic relationship between network environment changes and congestion status.

[0064] During the implementation of step 101, a cluster of intelligent probes deployed at key network nodes first continuously monitors the target path using a sliding time window, extracts high-order statistical features of bandwidth fluctuations using a wavelet packet decomposition algorithm, and denoises the raw delay data using an improved Kalman filter to generate a time-stamped standardized environmental fluctuation dataset. This dataset is then input into a pre-trained spatiotemporal graph neural network, where it is aligned with the congestion index time series data provided by the network control plane using dynamic time warping technology. The correlation coefficients in the time and frequency domains are calculated using a dual-channel attention mechanism. Finally, a joint learning model constructed based on a gated recurrent unit fuses the above-mentioned correlation features with the path topology features, and outputs an environmental network correlation parameter matrix containing the path stability score, congestion correlation strength, and fluctuation propagation factor through a multi-layer perceptron. This matrix will serve as the core input for subsequent routing strategy formulation.

[0065] Optionally, to address the difficulty in quantifying the correlation between environmental fluctuations and network congestion, in some embodiments, dynamically correlating the environmental fluctuation data with the changing trend of the network congestion index in step 101 to generate an environmental network correlation parameter includes:

[0066] 1011. Collect physical interference intensity data and network delay fluctuation data from the environmental fluctuation data, extract fluctuation peak moments of the two data in the same time window, and calculate time matching;

[0067] 1012. Proportionally adjust the fluctuation amplitude of the physical interference intensity data and the sudden increase amplitude of the network delay fluctuation data based on the time matching degree, so that the fluctuation magnitudes of the two are mapped to a unified numerical range;

[0068] 1013. Input the adjusted physical interference fluctuation amplitude and network delay sudden increase amplitude into the dynamic weight allocator, automatically adjust the fusion ratio weight of the two according to the density distribution of matching events in the time window, and generate the environment network correlation parameter.

[0069] In step 1011, a distributed sensor cluster deployed at the physical layer is used to synchronously collect physical interference intensity data (such as RSSI fluctuations and signal-to-noise ratio changes) and network delay fluctuation data using a sliding time window mechanism, and a wavelet transform algorithm is used to extract the fluctuation peak moment series of the two. Subsequently, a dynamic time warping algorithm is applied to align the time axes of the two peak sequences, and a time matching degree is generated by calculating the ratio of the number of matching peak pairs to the total peaks. A higher value indicates a more significant causal relationship between physical interference and network delay.

[0070] Step 1012 adjusts the proportions of the two types of data based on the time matching degree: when the time matching degree is higher than the threshold, the maximum and minimum normalization method is used to scale the physical interference amplitude and the network delay surge amplitude to the range of 0-1 respectively; if the matching degree is lower than the threshold, the scaling factor is adjusted according to the matching degree ratio through linear interpolation. For example, when the time matching degree is 0.6, the physical interference amplitude scaling factor is 0.6×the standard normalization factor, ensuring that the magnitude differences of low-correlation data fluctuations are compressed.

[0071] In step 1013, the adjusted physical interference fluctuation amplitude and network delay spike amplitude are input into the dynamic weight allocator. This module uses an event density-aware algorithm to calculate the distribution density of matching events (i.e., events where physical interference and delay spikes occur simultaneously) within the current time window. When the density is higher than the historical average, a higher weight (e.g., 0.7) is assigned to physical interference; when the density is lower, a higher weight (e.g., 0.6) is assigned to network delay. Finally, a multidimensional correlation parameter matrix is ​​generated using the weighted formula: Environment Network Correlation Parameter = W_Physical × Physical Interference Amplitude + W_Delay × Network Delay Spike (W_Physical + W_Delay = 1), which serves as input for subsequent routing strategies.

[0072] 102. Divide the real-time updated quotation data into independent data blocks according to the boundaries of the geographic grid and the ownership relationship of the commodity classification level nodes, and dynamically bind the independent data blocks to the storage nodes through a scalable hash ring to generate a redundant storage topology;

[0073] Among them, the rectangular boundary coordinates of the geographic grid are the set of coordinate ranges of the regular rectangular grid that divides the geographic area according to longitude and latitude, which is used to discretize the physical space into computable units; the path depth of the commodity classification hierarchy node refers to the hierarchical distance from the root node to the current node of the commodity in the classification tree structure, reflecting the position depth of the commodity in the classification system; the scalable hash ring is a distributed storage structure improved based on the consistent hashing algorithm, which realizes the dynamic binding of data blocks and storage nodes through the virtual node mapping mechanism; the redundant storage topology refers to the data storage architecture constructed through the primary and secondary storage node binding strategy, ensuring that each data block has copies on multiple physical nodes to improve disaster recovery capabilities.

[0074] The specific process of step 102 includes: extracting the coordinates of the center of the geographic grid and the depth of the commodity classification path, determining the data ownership through the product threshold, generating a segment identification code and mapping it to the hash ring, and finally selecting the primary and secondary storage nodes according to the virtual node position.

[0075] 103. Generate a distributed routing strategy based on the energy integration result of the environmental network associated parameters, combined with the coverage density of the geographic grid and the update rate of the commodity classification level;

[0076] Among them, the energy integration result of the environmental network correlation parameters is the cumulative energy value obtained by numerically integrating the spectrum energy in the parameter matrix in the time and space dimensions, which reflects the comprehensive strength of the network environment stability in a specific area; the coverage density of the geographic grid refers to the number of data nodes contained in a unit geographic area, which represents the density of data distribution; the update rate of the commodity classification level is the frequency of data updates under a certain commodity classification node within a statistical unit time; the distributed routing strategy is a set of dynamic path selection rules generated by comprehensively integrating the network environment status, data distribution characteristics and business needs, which is used to optimize data transmission efficiency and reliability.

[0077] Step 103 generates a dynamic routing strategy by integrating the environmental network energy integral, geographic grid density, and product category update rate. The specific process includes calculating the routing weight of the geographic grid, combining the product update rate to obtain a dynamic weight coefficient and classify the routing, allocating candidate routing paths and determining the transmission direction, and finally associating and storing the generated strategy.

[0078] 104. Coordinately calculate the transmission path selection priority of each independent data block based on the communication weight distribution in the distributed routing strategy, in combination with the boundary expansion direction of the geographic grid and the data volume growth trend of the commodity classification level;

[0079] Among them, the communication weight distribution is a set of values ​​defined in the distributed routing strategy that reflects the transmission priority of the geographic grid, and is obtained by comprehensively calculating the environmental network status and business needs; the boundary expansion direction of the geographic grid is a vector direction generated by analyzing the historical expansion trajectory, which represents the main trend of geographic area expansion; the data volume growth trend of the commodity classification level is the rate of change of data increment per unit time calculated based on time series analysis; the transmission path selection priority is a path optimization level sequence generated by collaboratively calculating the network status, geographic expansion direction and business growth needs.

[0080] Step 104 dynamically adjusts weights and generates transmission path priorities by integrating communication weight distribution, geographic expansion direction, and data growth trends. The specific process includes decomposing the geographic expansion direction into vector components, calculating the data growth slope, generating a dynamic overlay coefficient to adjust communication weights, and finally generating priorities based on the integration of multiple parameters.

[0081] 105. Establish a path state evolution model based on the node failure probability of the redundant storage topology and the spectrum energy attenuation rate of the environmental network-related parameters, generate a path correction factor according to the path state evolution model, and when the path correction factor exceeds a preset threshold, push the independent data block to multiple parallel transmission channels in a preset geographical direction according to the transmission path selection priority.

[0082] Among them, the node failure probability of the redundant storage topology is the possibility of storage node service interruption predicted by the Markov chain model, which is calculated based on the hardware failure rate and historical network fluctuation data; the spectrum energy attenuation rate is an indicator of the environmental network correlation parameter that reflects the rate at which the signal strength of the transmission path decays over time, and is calculated through frequency domain energy difference; the path state evolution model is a mathematical model that describes the change of transmission path stability over time, mapping the path parameters into discrete state intervals; the path correction factor is a dynamic adjustment coefficient generated according to the degree to which the path state deviates from the stable interval, which is used to trigger path switching decisions; the segmented channel mapping table is a metadata table that records the binding relationship between the data segment number and the transmission channel, realizing the coordinated control of multi-channel parallel transmission.

[0083] Step 105 analyzes storage node failure probabilities and spectrum energy attenuation to construct a path state evolution model and generate correction factors. When the correction factors exceed a limit, data is pushed in segments according to priority to parallel channels in a preset direction. The specific process includes collecting node failure probability and spectrum attenuation data, dynamically combining them to generate path evolution parameters, dividing state intervals, calculating correction factors, and monitoring thresholds to trigger segmented transmission.

[0084] To address the problem of low data storage and retrieval efficiency in a distributed quotation system, in some embodiments, step 102 divides the real-time updated quotation data into independent data blocks according to the boundaries of the geographic grid and the ownership relationship of the commodity classification level nodes, and dynamically binds the independent data blocks to storage nodes through a scalable hash ring to generate a redundant storage topology, including:

[0085] 1021. Extracting the center point coordinates of each geographic grid and the path depth of the commodity classification level node based on the rectangular boundary coordinates of the geographic grid;

[0086] 1022. When the geographic coordinates of the real-time updated quotation data fall within the rectangular boundary of any geographic grid, the product value of the center point coordinates of the geographic grid and the path depth of the commodity classification level node is calculated. If the product value is less than a preset threshold, the quotation data is assigned to the independent data block corresponding to the current geographic grid and commodity classification level node.

[0087] 1023. Generate a segment identification code for each independent data block, construct a data structure of the scalable hash ring, and input the segment identification code into a key-value mapping function of the scalable hash ring to calculate a mapping angle value of the segment identification code on the scalable hash ring;

[0088] 1024. Select the storage node corresponding to the first virtual mapping point containing the mapping angle value in the clockwise direction as the primary storage node, and select at least two storage nodes separated from the primary storage node by a fixed angle on the scalable hash ring as secondary storage nodes to generate a redundant storage topology in which the primary and secondary storage nodes are bound.

[0089] Among them, the center point coordinates of the geographic grid are the geometric center coordinates determined by calculating the intersection of the diagonals of the rectangular grid; the path depth of the commodity classification level node is the number of levels obtained by recursively traversing the classification tree; the segment identification code is a unique code generated based on the attributes of the data block; the virtual mapping point is the logical storage node identifier distributed at a fixed angle on the scalable hash ring.

[0090] In step 1021, first, the rectangular boundary coordinates are generated based on the geographic grid division algorithm (such as GeoHash), and the center point coordinates of each grid are calculated using the geometric center formula. At the same time, the product classification tree structure is traversed, and the depth-first search algorithm is used to count the path depth from each classification node to the root node to form a mapping table of grid center coordinates and classification path depth.

[0091] After entering step 1022, when new quotation data arrives, the geographic grid to which it belongs is determined through the geographic coordinate matching algorithm (such as R-tree spatial index), the corresponding center point coordinates and product classification path depth are obtained, and the product value of the two is calculated; if the product value is less than the preset threshold (obtained through historical data training), the data is classified into the independent data block corresponding to the current geographic grid and product classification level based on the K-means clustering algorithm, otherwise the exception handling process is triggered.

[0092] In step 1023, the SHA-256 hash algorithm is used to generate a segment identification code for the metadata of the independent data block (including the grid ID and the classification path hash value), and then the data structure of the scalable hash ring is constructed: the virtual nodes are evenly distributed on the ring at an angle of 2^N, each virtual node corresponds to the logical mapping of the physical storage node, and the segment identification code is mapped to a specific angle value on the ring through the consistent hash algorithm.

[0093] Finally, execute step 1024. According to the mapping angle value of the segment identification code, search the hash ring clockwise for the first virtual mapping point that is larger than the angle, and set the corresponding physical storage node as the primary node; then, according to the redundancy strategy (such as N=3 copies), select the physical nodes corresponding to the two virtual mapping points 120 degrees apart in the clockwise direction as secondary nodes, and complete the binding of the primary and secondary nodes through the distributed lock mechanism to form a redundant storage topology structure that includes the data copy distribution relationship.

[0094] In order to solve the problem of insufficient optimization of data transmission paths in a distributed bidding system, in some embodiments, the energy integration result based on the environmental network-related parameters in step 103 is combined with the coverage density of the geographic grid and the update rate of the commodity classification level to generate a distributed routing strategy, including:

[0095] 1031. For each geographic grid, cumulatively calculate the energy integration results of the environmental network associated parameters, and superimpose the coverage density values ​​of the geographic grids to generate a routing weight for each geographic grid;

[0096] 1032. Multiply the routing weight by the update rate of the same commodity classification level to obtain a dynamic weight coefficient for each commodity classification level, and divide the routing levels according to the numerical range of the dynamic weight coefficient;

[0097] 1033. Allocate a preset number of candidate routing paths to each routing level, and determine a set of transmission directions of the candidate routing paths based on the geographical distribution direction of the storage nodes in the redundant storage topology and the boundary extension direction of the geographical grid;

[0098] 1034. Associate and store the routing level, transmission direction set, and corresponding dynamic weight coefficient to generate a distributed routing strategy.

[0099] Among them, the routing weight is the quantitative value of the path selection priority obtained by weighted calculation of the environmental energy integral and coverage density within the geographic grid; the dynamic weight coefficient is the product of the routing weight and the commodity classification update rate, which is used to dynamically adjust the routing priority; the routing level is the priority level divided according to the range of the dynamic weight coefficient; the candidate routing path is the set of transmission paths pre-configured for each routing level; the transmission direction set is the path direction constraint condition generated based on the storage node distribution and the geographic expansion direction.

[0100] In step 1031, a discrete integral calculation is first performed on the environmental network correlation parameter matrix within each geographic grid, and the time-frequency domain energy value is accumulated using the trapezoidal integration algorithm. At the same time, the coverage density of the data nodes in the grid is calculated using the kernel density estimation algorithm. The energy integration result and the coverage density are then weighted and superimposed according to a preset ratio (such as 6:4), and a routing weight value in the range of 0-1 is generated through normalization processing. The higher the weight, the higher the transmission priority of the grid.

[0101] After entering step 1032, the real-time update rate (such as the number of updates per second) of the commodity classification level node is extracted, and a scalar multiplication operation is performed on it with the routing weight of the corresponding geographic grid to obtain a dynamic weight coefficient; based on the preset grading threshold (such as 0.3, 0.6, 0.9), the interval partitioning algorithm is used to map the coefficient to three routing levels of high, medium, and low, and a priority label is assigned to each level (such as level 1 is the highest priority).

[0102] When executing step 1033, based on the latitude and longitude coordinates of the storage nodes in the redundant storage topology, a vector direction analysis algorithm is used to calculate their distribution direction angles relative to the center point of the geographic grid. At the same time, combined with the expansion direction of the geographic grid boundary (predicted by fitting the historical expansion trajectory), a direction similarity matching algorithm is used to screen out storage nodes whose angles with the expansion direction are less than 15 degrees; a preset number of candidate routing paths (such as 5 for level 1) are allocated to each routing level, and the path direction set is composed of the matching storage node direction angle set.

[0103] Finally, in step 1034, the routing level, transmission direction set and dynamic weight coefficient are associated and stored through a key-value database (such as Redis), and the Bloom filter is used to optimize the fast retrieval capability to generate a distributed routing strategy including a priority mapping table, direction constraint rules and weight update mechanism. The strategy is synchronized to all routing nodes through ZooKeeper to complete the deployment.

[0104] In order to solve the problems of transmission path decision lag and communication weight adaptation mismatch caused by the decoupling of the geographic grid expansion direction and the growth trend of commodity classification data, in some embodiments, step 104 collaboratively calculates the transmission path selection priority of each independent data block based on the communication weight distribution in the distributed routing strategy, combined with the boundary expansion direction of the geographic grid and the data volume growth trend of the commodity classification level, including:

[0105] 1041. Extract a communication weight distribution set from the storage nodes in the distributed routing strategy, perform vector decomposition on the boundary extension direction of the geographic grid to generate a main direction component set, and calculate a trend slope parameter based on an incremental sequence of adjacent time window data at the commodity classification level;

[0106] 1042. Input the amplitude of the main direction component set and the trend slope parameter into a distributed collaborative computing node, generate a dynamic superposition coefficient set through a product operation, and superimpose the dynamic superposition coefficient set onto the communication weight distribution set according to the cell identifier of the geographic grid to form a dynamically adjusted communication weight distribution;

[0107] 1043. Based on the dynamically adjusted communication weight distribution, combined with the coverage density, the dynamic superposition coefficient set, the depth of the commodity classification level node and the real-time update frequency of the independent data block, the transmission path selection priority is generated through the distributed collaborative computing node.

[0108] Among them, the main direction component set is a set of orthogonal vector components that decomposes the geographic grid expansion direction into multiple components, which is used to quantify the spatial distribution of the expansion trend; the trend slope parameter is the rate of change of the data increment in adjacent time windows calculated by linear regression; the dynamic overlay coefficient is the product of the main direction component amplitude and the trend slope, which is used to dynamically correct the communication weight; the communication density parameter is the product of the communication weight and the coverage density, which characterizes the matching degree between the transmission resources and the data distribution; the hierarchical depth parameter is a value calculated by inversely proportionally calculating the depth of the commodity classification node and the dynamic overlay coefficient, which reflects the influence weight of the classification hierarchy on the routing.

[0109] In step 1041, the communication weight distribution set is first extracted from the storage node of the distributed routing strategy. A principal component analysis algorithm is used to perform vector decomposition on the boundary extension direction of the geographic grid, generating a set of principal direction components containing east-west and south-north components. Simultaneously, based on the incremental sequence of time-series data from the commodity classification level nodes, the least squares method is used to fit the data volume change curves of adjacent time windows, calculate the trend slope parameter, and form a mapping table between the slope and classification level.

[0110] After entering step 1042, the amplitude of the main direction component set is calculated using the vector modulus and input into the distributed collaborative computing framework along with the trend slope parameter. A scalar product operation is performed in the computing node to generate a dynamic overlay coefficient set. A mapping relationship table is then established between the geographic grid cell identifier and the communication weight distribution set. The weight adjustment ratio is determined by the inverse relationship between the coverage density and the depth of the commodity classification node. The specific formula is that the merger ratio is equal to 1 divided by 1 plus the product of the coverage density and the depth. Finally, the cell identifier is matched using a distributed hash table, and the dynamic overlay coefficient is weightedly superimposed with the original communication weight according to the merger ratio. For example, the original weight is multiplied by 0.7 and the dynamic coefficient is multiplied by 0.3. This generates a dynamically adjusted communication weight distribution that includes the characteristics associated with geographic expansion and data growth.

[0111] In step 1043, the communication density parameter is first calculated based on the dynamically adjusted communication weight distribution, specifically the communication weight multiplied by the coverage density. At the same time, the hierarchical depth parameter is calculated, which is the dynamic superposition coefficient divided by the value of the commodity classification node depth plus 1. In addition, the update intensity parameter is generated by multiplying the real-time update frequency of the independent data block by the preset gain coefficient. Subsequently, in the distributed collaborative computing node, the communication density parameter, the hierarchical depth parameter, and the update intensity parameter are fused according to a preset ratio, for example, with a weight ratio of 4:3:3, to generate a dynamic composite parameter. Finally, based on the composite ranking result of the dynamic composite parameter, the transmission paths of all independent data blocks in the geographic grid are prioritized using a multi-objective decision algorithm, and a global routing decision table containing path priority labels is output and pushed to the edge routing node for path selection.

[0112] In order to solve the problem of weight distribution imbalance caused by the static coupling of geographic grid expansion and commodity classification data growth trend in traditional communication weight adjustment, in some embodiments, the step 1042 of superimposing the dynamic superposition coefficient set onto the communication weight distribution set according to the cell identifier of the geographic grid to form a dynamically adjusted communication weight distribution includes:

[0113] 201. Obtain a mapping relationship table between the unit identifiers of the geographic grid and the communication weight distribution set, and extract original communication weight values ​​corresponding to the unit identifiers of the geographic grid based on the mapping relationship table;

[0114] 202. Determine a combined ratio of the dynamic superposition coefficient and the original communication weight value based on an inverse relationship between the coverage density of the geographic grid and the depth of the commodity classification level node;

[0115] 203. Match the unit identifiers in the dynamic superposition coefficient set with the unit identifiers of the geographic grid in the mapping relationship table, and for the successfully matched unit identifiers, proportionally superimpose the corresponding dynamic superposition coefficients and the original communication weight values ​​according to the merging ratio;

[0116] 204. Rewrite the communication weight values ​​after the superposition processing into the communication weight distribution set according to the cell identifiers of the geographic grid to generate a dynamically adjusted communication weight distribution including correlation characteristics between the geographic expansion direction and the data growth trend.

[0117] Among them, the unit identifier is the unique code of the geographic grid, which is used to accurately associate data with geographic units; the mapping relationship table is a distributed data structure that stores the correspondence between unit identifiers and communication weight values; the merger ratio is a weight adjustment coefficient dynamically calculated based on the inverse relationship between coverage density and the depth of commodity classification level nodes; the dynamic superposition coefficient is a correction parameter generated by combining the geographic expansion direction component and the data growth trend, which is used to optimize the communication weight distribution.

[0118] In step 201, a pre-stored mapping relationship table between geographic grid unit identifiers and communication weight distribution sets is obtained by querying a distributed key-value database, and the corresponding original communication weight value is quickly located and extracted based on the hash index of the unit identifier.

[0119] After entering step 202, the merging ratio is calculated using a formula based on the inverse relationship between coverage density and the depth of the commodity classification level node. The specific formula is merging ratio = 1 / (1 + coverage density × depth). This ratio is used to dynamically adjust the fusion weight of the superposition coefficient and the original weight.

[0120] When executing step 203, the unit identifiers in the dynamic superposition coefficient set are matched with the unit identifiers in the mapping relationship table, and the identifier consistency is quickly compared through the hash table matching algorithm. For the successfully matched unit identifiers, the dynamic superposition coefficients and the original communication weight values ​​are linearly superimposed according to the merging ratio. For example, the adjusted communication weight value = 0.7W_original + 0.3C_dynamic (where W_original is the original communication weight value and C_dynamic is the dynamic superposition coefficient) to generate the adjusted communication weight value.

[0121] Finally, in step 204, the adjusted communication weight value is rewritten into the distributed storage communication weight distribution set according to the geographic grid unit identifier, and the data consistency is ensured through the distributed transaction mechanism. Finally, a dynamically adjusted communication weight distribution containing the correlation characteristics of the geographic expansion direction and the data growth trend is generated to provide input for subsequent priority calculations.

[0122] In order to solve the problem of decision distortion caused by the linear superposition of parameters such as communication weight, geographic density, and commodity hierarchy depth in traditional transmission path priority calculation, in some embodiments, the transmission path selection priority is generated by the distributed collaborative computing node based on the dynamically adjusted communication weight distribution, the coverage density, the dynamic superposition coefficient set, the depth of the commodity classification hierarchy node, and the real-time update frequency of the independent data block in step 1043, including:

[0123] 301. Generate a communication density parameter by multiplying the communication weight value of the geographic grid unit by the coverage density, and generate a hierarchy depth parameter by inversely converting the dynamic stacking coefficient by the depth of the commodity classification hierarchy node;

[0124] 302. Fusion the communication density parameter, the layer depth parameter, and the update intensity parameter generated by the real-time update frequency of the independent data block according to a preset ratio to generate a dynamic synthesis parameter;

[0125] 303. Generate, in the distributed collaborative computing node, a transmission path selection priority for independent data blocks within a geographic grid unit based on the composite sorting of the dynamic synthesis parameters.

[0126] Among them, the update intensity parameter is a dynamic factor generated by multiplying the real-time update frequency of independent data blocks and the preset gain coefficient, which is used to quantify the real-time requirements of data update for path selection; the dynamic synthesis parameter is a comprehensive decision-making indicator generated by the nonlinear fusion of communication density parameter, hierarchical depth parameter and update intensity parameter, which is used to avoid the priority distortion problem caused by traditional linear superposition.

[0127] In step 301, the communication weight value of each geographic grid cell is first extracted based on the dynamically adjusted communication weight distribution. This value is then multiplied by the corresponding grid's coverage density (communication density parameter = communication weight × coverage density). This generates a parameter reflecting the compatibility of transmission resources with data distribution. Simultaneously, a depth-attenuation correction is applied to the dynamic overlay coefficient using the inverse-proportional conversion formula (layer depth parameter = dynamic overlay coefficient / (product classification node depth + 1)) to mitigate the interference of deep classification nodes on routing decisions.

[0128] After entering step 302, the real-time update frequency of the independent data block is obtained from the distributed storage and multiplied by the preset gain coefficient (set to 1.2 based on business requirements) to generate an update intensity parameter = update frequency × 1.2, which is used to amplify the priority of high-frequency updated data. Subsequently, a nonlinear fusion strategy is adopted to dynamically combine the communication density parameter, the layer depth parameter, and the update intensity parameter in a preset ratio (e.g., 4:3:3) through a weighted summation formula to form a composite parameter = 0.4 × communication density + 0.3 × layer depth + 0.3 × update intensity. This avoids decision bias caused by linear superposition of single parameters.

[0129] In step 303, the dynamic synthesis parameters are input into a distributed collaborative computing node (such as the Apache Flink stream processing engine), and the transmission paths of all independent data blocks within the same geographic grid are prioritized based on a multi-objective composite ranking algorithm (such as the TOPSIS distance method). By calculating the Euclidean distance between the dynamic synthesis parameters of each data block and the ideal solution, a priority ranking result is generated. Finally, a global routing decision table containing path priority labels (such as P0-P3 levels) is output and pushed to the edge routing node via a message queue (such as Kafka) for real-time path selection.

[0130] In order to solve the problem of transmission path stability caused by node failure and network attenuation in a distributed storage environment, in some embodiments, a path state evolution model is established based on the node failure probability of the redundant storage topology and the spectrum energy attenuation rate of the environmental network-related parameters in step 105, and a path correction factor is generated according to the path state evolution model. When the path correction factor exceeds a preset threshold, the independent data block is pushed to multiple parallel transmission channels in a preset geographical direction according to the transmission path selection priority, including:

[0131] 1051. Obtain a real-time failure probability value of each storage node in the redundant storage topology, and a spectrum energy attenuation rate value of a corresponding transmission path in the environmental network association parameter;

[0132] 1052. Dynamically assign a superposition weight of the failure probability value according to a ratio of the historical online time of the storage node to the current spectrum energy value;

[0133] 1053. Dynamically combine the failure probability value and the spectrum energy attenuation rate value according to the assigned superposition weight to generate a path evolution parameter sequence for each transmission path;

[0134] 1054. Based on the numerical distribution characteristics of the path evolution parameter sequence, a path state evolution model is established using a preset evolution parameter interval division rule, wherein the path state evolution model divides the numerical interval into a stable state evolution region, a fluctuating state evolution region, and a failure state evolution region;

[0135] 1055. When the value of the path evolution parameter sequence falls into the wave state evolution zone or the failure state evolution zone, a path correction factor is generated according to the deviation amplitude between the value and the critical value of the corresponding evolution zone.

[0136] 1056. Monitor a current value of the path correction factor. When the current value is greater than a preset correction threshold, determine a push order of the independent data blocks to be pushed according to the ranking in the transmission path selection priority list.

[0137] 1057. Extract, from the redundant storage topology, a set of storage nodes associated with the preset geographical direction based on the preset coverage range of the geographical direction, and allocate at least one transmission channel to each storage node in the set of storage nodes.

[0138] 1058. Segment the independent data blocks to be pushed to generate segmented data blocks equal to the number of transmission channels, and mark the segmented data blocks with corresponding segment numbers in sequence according to the pushing order;

[0139] 1059. Dynamically bind the segment number with the channel identifier of the transmission channel to generate a segment channel mapping table, and according to the segment channel mapping table, synchronously push multiple segmented data blocks of the same independent data block to multiple parallel transmission channels in a preset geographical direction through corresponding transmission channels.

[0140] Among them, the real-time failure probability value is a quantitative indicator of the storage node service availability monitored in real time; the spectrum energy attenuation rate value is the attenuation of the transmission path signal energy per unit time; the superposition weight is the weight coefficient allocated according to the dynamic ratio of the node's historical online time and the current spectrum energy; the path evolution parameter sequence is a set of timing parameters generated by the weighted combination of the failure probability and the spectrum attenuation rate; the stable state evolution zone, the wave state evolution zone and the failure state evolution zone are the stability level intervals defined in the path state model; the deviation amplitude is the difference between the path evolution parameter value and the critical value of the evolution zone; the segmented data block is an equal data unit that splits the independent data block according to the number of transmission channels; the channel identifier is the unique logical identifier of the transmission channel; the segmented channel mapping table is a distributed metadata table that records the binding relationship between segmented data and channels.

[0141] In step 1051, the hardware status log (such as disk health and CPU load) and network packet loss rate of each storage node in the redundant storage topology are collected in real time through the distributed monitoring system, and the node failure probability value is calculated using the exponential smoothing algorithm; at the same time, the spectrum energy time series data of each transmission path is extracted from the environmental network association parameters, and the energy decay rate value per unit time is calculated by first-order difference to form a key-value pair set of failure probability and decay rate.

[0142] Step 1052 dynamically allocates the superposition weight of the failure probability value based on the historical online time (continuous service time within the statistical period) of the storage node and the current spectrum energy value (normalized to the range of 0-1) using the proportional allocation formula: superposition weight = historical online time / (historical online time + current spectrum energy). Nodes with long online time and high energy are assigned low weights to reduce their negative impact on path stability.

[0143] When executing step 1053, the failure probability value is dynamically combined with the spectrum energy attenuation rate according to the superposition weight. The calculation formula is path evolution parameter = α × failure probability + (1-α) × attenuation rate (where α is the superposition weight). The evolution parameter sequence of each transmission path is generated and stored in a time series database (such as InfluxDB).

[0144] Step 1054 divides the evolution interval into three categories based on the numerical distribution of the evolution parameter sequence using the tertiary statistical method: stable state (e.g., path evolution parameter ≤ Q1), fluctuating state (e.g., Q1 < path evolution parameter ≤ Q3), and failure state (e.g., path evolution parameter > Q3), where Q1 and Q3 are the first and third quartile values ​​of the sequence, respectively. This constructs a three-state path evolution model.

[0145] In step 1055, when the path evolution parameter falls into the fluctuating state or failure state, the absolute deviation between the parameter value and the nearest critical value is calculated as deviation amplitude = |parameter value - critical value|, and is mapped to a correction factor in the range of 0-1 through the S-type function path correction factor = 1 / (1+e^(-k×deviation amplitude)) (k is the slope coefficient). The greater the deviation, the closer the correction factor is to 1.

[0146] Step 1056 continuously monitors the correction factor. When the value exceeds a preset threshold (e.g., 0.7), the independent data blocks to be pushed are extracted from the transmission path selection priority list in order, and a push order queue is generated (e.g., data blocks with higher priority are placed at the front of the queue).

[0147] Step 1057 selects matching storage nodes from the redundant storage topology according to the latitude and longitude range of the preset geographical direction (such as the eastern region), and uses a load balancing algorithm (such as a polling strategy) to allocate at least one transmission channel to each node to form a channel-node binding list.

[0148] In step 1058, the independent data blocks to be pushed are divided into equal segments according to the number of transmission channels. For example, if there are three channels, they are divided into three segmented data blocks. Each segment is assigned an increasing segment number (such as Block_001 to Block_003) and marked in the sending order according to the push sequence queue.

[0149] Finally, in step 1059, a mapping relationship between segment numbers and channel identifiers is established, for example, Block_001→Channel_A, Block_002→Channel_B, to generate a segment channel mapping table; the mapping table is broadcast to all relevant nodes through a distributed message queue (such as RabbitMQ), and each channel synchronously transmits the corresponding segment data block according to the mapping table to ensure parallel push to the storage node in the target geographical direction.

[0150] Here are some practical use cases for steps 101 to 105:

[0151] For example, after deploying this solution at an e-commerce logistics center, a cluster of intelligent probes, based on the Prometheus monitoring tool, was deployed at the edge of the warehouse network. These probes collected bandwidth fluctuations along the transmission path in real time, sampling every second and calculating the standard deviation as a fluctuation indicator. Link delay jitter data was also measured, using the ICMP protocol to obtain end-to-end delays and calculating the jitter variance within a 10-second window. Wavelet packet decomposition was used to extract the third-order moment characteristics of bandwidth fluctuations, and a pre-trained LSTM model was used to calculate the network congestion index. This model was trained based on three months of historical traffic data, with the previous hour's traffic load rate and queue buffer overflow rate as input parameters. The model output range was set to 0 to 1, with a value exceeding 0.8 indicating severe congestion.

[0152] Real-time logistics quote data is then partitioned into 10km by 10km geographic grids based on longitude and latitude. This grid is encoded using GeoHash. The coordinates of each grid center are calculated by taking the mean longitude and latitude of the grid's diagonal intersections and multiplying this value by the product classification depth, defined as the path length from the root node "All Categories" to the final category. A preset threshold of 500 is determined by fitting historical order distributions. For example, a clothing product with a classification depth of four levels has a grid center coordinate product of 450, which falls below the threshold of 500. This triggers the classification decision logic and assigns the data to the corresponding block. Dynamic binding is achieved through a scalable hash ring configured with 2 to the 16th power of virtual nodes. Using the MurmurHash3 algorithm, data blocks are bound to three storage nodes in the eastern region. These nodes are built on AWS S3 storage instances, forming a three-copy redundant topology.

[0153] A dynamic routing strategy is then generated based on the energy integration results of the environmental network-related parameters. The energy integration is calculated by integrating the spectral energy of the parameter matrix over a 0.5-second time window, taking into account the grid coverage density and product update rate. Grid coverage density is defined as the number of active devices per square kilometer, and the product update rate is the number of price updates per minute. For example, home appliances, with an update rate of up to 50 times per minute, are assigned a high-priority routing strategy, prioritizing dedicated lines with latency below 50 milliseconds. When the failure probability of a storage node exceeds a threshold of 0.65, the system automatically triggers a path correction mechanism. The failure probability is based on the node's historical failure rate and the current CPU load forecast. The correction mechanism segments data into four 4-megabyte blocks, totaling 16 megabytes. These blocks are transmitted simultaneously via four parallel transmission channels in the southeast direction, each with a bandwidth of 1 gigabit per second. This ensures that order data push latency remains stable within 20 milliseconds during the promotion period.

[0154] For example, in urban traffic monitoring scenarios, the system collects physical interference data using RS-EMF-01 electromagnetic sensors deployed at intersections with a sampling rate of 1 kHz. These sensors measure electromagnetic noise intensity and record it in decibel milliwatts. Network probes also capture end-to-end latency fluctuations from the camera to the central node using the NetFlow protocol, calculating jitter every 5 seconds. A dynamic time warping algorithm is used to align physical interference peaks and latency spike sequences, with a configured window width of 2 seconds and a step size of 0.1 seconds. Physical interference peaks are identified using a threshold detection method, with latency spikes defined as fluctuations exceeding 30% of the baseline value. The calculated time matching degree is 0.78, meaning that the matched peaks account for 78% of the total peaks. The interference amplitude and latency spike were then normalized. The interference amplitude was normalized to a range of 0 to 1 by calculating the difference between the maximum and minimum values. The latency spike was expressed as a percentage of the baseline. Scaling was performed based on a matching degree of 0.78, resulting in an interference amplitude of 0.663 and a latency spike of 0.484. The environmental network correlation parameters were then fused with a weight of 6 to 4, resulting in a final value of 0.581. This weighting was determined based on historical data analysis, with the optimal solution being a 60% impact of physical interference on latency.

[0155] The real-time video stream is then divided into 500-meter by 500-meter geographic grids based on intersection latitude and longitude. The video stream is encoded using H.264 at a bitrate of 4 Mbps. Taking into account the depth of the transportation hub classification hierarchy, main roads are defined as Level 1, secondary roads as Level 2, and branch roads as Level 3. Data blocks are dynamically bound to five edge nodes on a scalable hash ring, using Huawei Atlas 500 intelligent edge servers. The node distribution density is determined by the number of cameras within the grid. A three-level routing strategy is generated based on the energy integral of environmental network-related parameters and grid density. The energy integral is calculated by integrating the parameter values ​​over a 10-second window. For example, a key road surveillance video with an energy integral of 7.1 and an average grid density of 6 cameras per grid is assigned the highest routing level. If the spectral energy decay rate at a node exceeds a threshold of 0.2 decibels per millisecond, the path correction factor is increased to 0.75. This correction factor is calculated based on a sigmoid function with a preset threshold of 0.7. The system immediately split the video stream into six data blocks of 150 megabytes each, for a total of 900 megabytes. These were then pushed in parallel via six pre-defined northwest transmission channels, each providing 200 megabits per second (Mbps) bandwidth, built on a 5G slicing network. Using the TOPSIS algorithm to dynamically optimize path selection, the ideal solution was defined as a combination with latency below 50 milliseconds and bandwidth above 1 gigabit per second. Actual solutions were ranked by Euclidean distance, ultimately reducing transmission latency for critical road condition videos from an average of 120 milliseconds to 45 milliseconds, and packet loss from 5% to 0.3%.

[0156] This application collects and dynamically correlates environmental fluctuation data with network congestion index in real time to accurately perceive the changing trend of network transmission quality, divides data blocks into blocks based on geographic grids and commodity classification levels, and constructs redundant storage topologies, significantly improving data storage reliability and partition management efficiency; generates dynamic routing strategies based on multi-dimensional parameter fusion, and uses communication weight, coverage density and update rate to collaboratively calculate transmission priority, effectively optimizing the real-time and accuracy of path selection decisions; monitors node failure risks and spectrum attenuation in real time through path state evolution models, and dynamically corrects abnormal paths in combination with segmented parallel transmission mechanisms, achieving high fault tolerance and low-latency data push in complex network environments, ultimately achieving the comprehensive optimization effects of improving global resource utilization, reducing transmission interruption rates, and achieving millisecond-level real-time response in multiple business scenarios.

[0157] Figure 2 A structural diagram of a distributed quotation data collaborative push system is provided for the embodiment of the present application. Figure 2 As shown, the system includes:

[0158] The acquisition module 21 collects environmental fluctuation data of the transmission path, dynamically associates the environmental fluctuation data with the change trend of the network congestion index, and generates environmental network correlation parameters;

[0159] The partitioning module 22 divides the real-time updated quotation data into independent data blocks according to the boundaries of the geographic grid and the ownership relationship of the commodity classification level nodes, and dynamically binds them to the storage nodes through a scalable hash ring to generate a redundant storage topology;

[0160] A generation module 23 generates a distributed routing strategy based on the energy integration result of the environmental network associated parameters, combined with the coverage density of the geographic grid and the update rate of the commodity classification level;

[0161] The calculation module 24 calculates the transmission path selection priority of each independent data block in a distributed collaborative manner based on the communication weight distribution in the distributed routing strategy, the boundary expansion direction of the geographic grid, and the data volume growth trend of the commodity classification level;

[0162] The push module 25 establishes a path state evolution model based on the node failure probability of the redundant storage topology and the spectrum energy attenuation rate of the environmental network-related parameters, generates a path correction factor according to the path state evolution model, and when the path correction factor exceeds a preset threshold, pushes the independent data block to multiple parallel transmission channels in a preset geographical direction according to the transmission path selection priority.

[0163] Figure 2 The distributed quotation data collaborative push system can execute Figure 1 The implementation principle and technical effects of the distributed quotation data collaborative push method described in the illustrated embodiment will not be elaborated on here. The specific manner in which each module and unit performs operations in the distributed quotation data collaborative push system in the above embodiment has been described in detail in the embodiment of the method and will not be elaborated on here.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A distributed quotation data collaborative push method, characterized in that: include: Collecting environmental fluctuation data of a data transmission path, and dynamically correlating the environmental fluctuation data with a changing trend of a network congestion index to generate an environmental network correlation parameter; Divide the real-time updated quotation data into independent data blocks according to the boundaries of the geographic grid and the ownership relationship of the commodity classification level nodes, and dynamically bind the independent data blocks to storage nodes through a scalable hash ring to generate a redundant storage topology; Generate a distributed routing strategy based on the energy integration result of the environmental network associated parameters, combined with the coverage density of the geographic grid and the update rate of the commodity classification level; collaboratively calculating the transmission path selection priority of each of the independent data blocks based on the communication weight distribution in the distributed routing strategy, combined with the boundary expansion direction of the geographic grid and the data volume growth trend of the commodity classification level; A path state evolution model is established based on the node failure probability of the redundant storage topology and the spectrum energy attenuation rate of the environmental network-related parameters, and a path correction factor is generated according to the path state evolution model. When the path correction factor exceeds a preset threshold, the independent data block is pushed to multiple parallel transmission channels in a preset geographical direction according to the transmission path selection priority.

2. The method according to claim 1, characterized in that Based on the communication weight distribution in the distributed routing strategy, combined with the boundary expansion direction of the geographic grid and the data volume growth trend of the commodity classification level, the transmission path selection priority of each independent data block is collaboratively calculated, including: Extracting a communication weight distribution set from a storage node in the distributed routing strategy, performing vector decomposition on the boundary extension direction of the geographic grid to generate a main direction component set, and calculating a trend slope parameter based on an incremental sequence of adjacent time window data at the commodity classification level; Inputting the amplitude of the main direction component set and the trend slope parameter into a distributed collaborative computing node, generating a dynamic superposition coefficient set through a product operation, and superimposing the dynamic superposition coefficient set onto the communication weight distribution set according to the cell identifier of the geographic grid to form a dynamically adjusted communication weight distribution; Based on the dynamically adjusted communication weight distribution, combined with the coverage density, the dynamic superposition coefficient set, the depth of the commodity classification level node and the real-time update frequency of the independent data block, the transmission path selection priority is generated through the distributed collaborative computing node.

3. The method according to claim 2, characterized in that Superimposing the dynamic superposition coefficient set onto the communication weight distribution set according to the cell identifier of the geographic grid to form a dynamically adjusted communication weight distribution, comprising: Obtaining a mapping relationship table between the unit identifiers of the geographic grid and the communication weight distribution set, and extracting original communication weight values ​​corresponding to the unit identifiers of the geographic grid based on the mapping relationship table; Determining a combined ratio of the dynamic superposition coefficient and the original communication weight value based on an inverse relationship between the coverage density of the geographic grid and the depth of the commodity classification level node; Matching the unit identifiers in the dynamic superposition coefficient set with the unit identifiers of the geographic grid in the mapping relationship table, and for the successfully matched unit identifiers, proportionally superimposing the corresponding dynamic superposition coefficients and the original communication weight values ​​according to the merging ratio; The communication weight values ​​after the superposition processing are rewritten into the communication weight distribution set according to the cell identifiers of the geographic grid, so as to generate a dynamically adjusted communication weight distribution including the correlation characteristics of the geographic expansion direction and the data growth trend.

4. The method according to claim 2, characterized in that Based on the dynamically adjusted communication weight distribution, combined with the coverage density, the dynamic superposition coefficient set, the depth of the commodity classification level node and the real-time update frequency of the independent data block, and through the distributed collaborative computing node, a transmission path selection priority is generated, including: A communication density parameter is generated by performing a product operation on the communication weight value of the geographic grid unit and the coverage density, and a hierarchical depth parameter is generated by performing an inverse proportional conversion between the dynamic superposition coefficient and the depth of the commodity classification hierarchical node; fusing the communication density parameter, the layer depth parameter, and the update intensity parameter generated by the real-time update frequency of the independent data block according to a preset ratio to generate a dynamic synthesis parameter; In the distributed collaborative computing node, a transmission path selection priority of an independent data block in a geographic grid unit is generated based on the composite ranking of the dynamic synthesis parameters.

5. The method according to claim 1, characterized in that A path state evolution model is established based on the node failure probability of the redundant storage topology and the spectrum energy attenuation rate of the environmental network-related parameters, and a path correction factor is generated according to the path state evolution model, including: Obtaining a failure probability value of each storage node in the redundant storage topology and a spectrum energy attenuation rate value of a corresponding transmission path in the environmental network association parameters; Dynamically assigning a superposition weight of the failure probability value according to the ratio of the historical online time of the storage node to the current spectrum energy value; Dynamically combining the failure probability value with the spectrum energy decay rate value according to the assigned superposition weight to generate a path evolution parameter sequence for each transmission path; Based on the numerical distribution characteristics of the path evolution parameter sequence, a path state evolution model is established by using a preset evolution parameter interval division rule, wherein the path state evolution model divides the numerical interval into a stable state evolution region, a fluctuating state evolution region, and a failure state evolution region; When the value of the path evolution parameter sequence falls into the wave state evolution zone or the failure state evolution zone, a path correction factor is generated according to the deviation amplitude of the value from the critical value of the corresponding evolution zone.

6. The method according to claim 1, characterized in that When the path correction factor exceeds a preset threshold, the independent data block is pushed to a plurality of parallel transmission channels in a preset geographical direction according to the transmission path selection priority, including: monitoring a current value of the path correction factor, and when the current value is greater than a preset correction threshold, determining a push order of the independent data blocks to be pushed according to the ranking in the transmission path selection priority; Extracting a set of storage nodes associated with the preset geographical direction from the redundant storage topology according to the coverage range of the preset geographical direction, and allocating at least one transmission channel to each storage node in the set of storage nodes; Segmenting the independent data blocks to be pushed to generate segmented data blocks equal to the number of transmission channels, and sequentially marking the segmented data blocks with corresponding segment numbers according to the pushing order; The segment number is dynamically bound to the channel identifier of the transmission channel to generate a segment channel mapping table, and according to the segment channel mapping table, multiple segment data blocks of the same independent data block are synchronously pushed to multiple parallel transmission channels in a preset geographical direction through corresponding transmission channels.

7. The method according to claim 1, characterized in that The real-time updated quotation data is divided into independent data blocks according to the boundaries of the geographic grid and the ownership relationship of the commodity classification level nodes. The independent data blocks are dynamically bound to storage nodes through a scalable hash ring to generate a redundant storage topology, including: Extracting the center point coordinates of each geographic grid and the path depth of the commodity classification level node according to the rectangular boundary coordinates of the geographic grid; When the geographic coordinates of the real-time updated quotation data fall within the rectangular boundary of any geographic grid, the product value of the center point coordinates of the geographic grid and the path depth of the commodity classification level node is calculated. If the product value is less than a preset threshold, the quotation data is attributed to the independent data block corresponding to the current geographic grid and commodity classification level node; generating a segment identification code for each of the independent data blocks, constructing a data structure of the scalable hash ring, and inputting the segment identification code into a key-value mapping function of the scalable hash ring to calculate a mapping angle value of the segment identification code on the scalable hash ring; The storage node corresponding to the first virtual mapping point containing the mapping angle value in the clockwise direction is selected as the primary storage node, and at least two storage nodes separated from the primary storage node by a fixed angle on the scalable hash ring are selected as secondary storage nodes to generate a redundant storage topology bound to the primary and secondary storage nodes.

8. The method according to claim 1, characterized in that Based on the energy integration result of the environmental network associated parameters, combined with the coverage density of the geographic grid and the update rate of the commodity classification level, a distributed routing strategy is generated, including: For each of the geographic grids, cumulatively calculating energy integration results of the environmental network associated parameters and superimposing coverage density values ​​of the geographic grids to generate a routing weight for each of the geographic grids; Multiplying the routing weight by the update rate of the same commodity classification level to obtain a dynamic weight coefficient for each commodity classification level, and dividing the routing levels according to the numerical range of the dynamic weight coefficient; Allocating a preset number of candidate routing paths to each routing level, and determining a set of transmission directions of the candidate routing paths based on the geographical distribution direction of the storage nodes in the redundant storage topology and the boundary extension direction of the geographical grid; The routing level, transmission direction set and corresponding dynamic weight coefficient are associated and stored to generate a distributed routing strategy.

9. The method according to claim 1, characterized in that Dynamically correlating the environmental fluctuation data with the changing trend of the network congestion index to generate environmental network correlation parameters, including: Collecting physical interference intensity data and network delay fluctuation data from the environmental fluctuation data, extracting fluctuation peak moments of the physical interference intensity data and the network delay fluctuation data according to the same time window and calculating time matching; Proportionally adjusting the fluctuation amplitude of the physical interference intensity data and the sudden increase amplitude of the network delay fluctuation data based on the time matching degree, so that the fluctuation magnitudes of the fluctuation amplitude and the sudden increase amplitude are mapped to a unified numerical range; The adjusted fluctuation amplitude and sudden increase amplitude are input into a dynamic weight allocator, and the fusion ratio weight of the two is automatically adjusted according to the density distribution of matching events in the time window to generate environmental network correlation parameters.

10. A distributed quotation data collaborative push system, characterized in that: include: A collection module collects environmental fluctuation data of the transmission path, dynamically associates the environmental fluctuation data with a change trend of the network congestion index, and generates an environmental network correlation parameter; The partitioning module divides the real-time updated quotation data into independent data blocks according to the boundaries of the geographic grid and the ownership relationship of the commodity classification level nodes, and dynamically binds them to the storage nodes through a scalable hash ring to generate a redundant storage topology; A generation module generates a distributed routing strategy based on the energy integration result of the environmental network associated parameters, combined with the coverage density of the geographic grid and the update rate of the commodity classification level; a calculation module for collaboratively calculating, based on the communication weight distribution in the distributed routing strategy and in combination with the boundary expansion direction of the geographic grid and the data volume growth trend of the commodity classification level, a transmission path selection priority for each of the independent data blocks; A push module establishes a path state evolution model based on the node failure probability of the redundant storage topology and the spectrum energy attenuation rate of the environmental network-related parameters, generates a path correction factor according to the path state evolution model, and when the path correction factor exceeds a preset threshold, pushes the independent data block to multiple parallel transmission channels in a preset geographical direction according to the transmission path selection priority.

Images