An offline data sharding synchronization method and system for a takeout system

Abstract

The application provides an offline data slice synchronization method and system for a point ordering system, which comprises the following steps: dividing point ordering data into multiple data slices on the offline terminal side, maintaining a state checkpoint containing volatility fingerprints and slice dependency relationship pointers for each data slice, and calculating a synchronization tendency vector quantum of a changed slice based on an associated weight matrix; after networking, the offline terminal initiates synchronization negotiation by carrying the state checkpoint and the synchronization tendency vector quantum, and the central server returns a synchronization starting vector and a heuristic guidance strategy; the offline terminal generates a subsequent synchronization sequence according to the synchronization starting vector and the heuristic guidance strategy, and uploads the data slices after resource verification is up to standard; when the central server detects a data merging conflict, a derived record is generated or an asynchronous merging queue is written according to a comprehensive available processing capacity score and a conflict resolution strategy table, so that the conflict processing is completed.

Description

Technical Field

[0001] This application belongs to the field of ordering systems, and in particular relates to a method and system for offline data sharding and synchronization in ordering systems. Background Technology

[0002] As the digital transformation of industries such as catering, retail, and local services continues to deepen, intelligent ordering systems have become a crucial infrastructure in offline business scenarios. In high-frequency transaction environments, store networks are susceptible to factors such as network congestion during peak hours, base station signal blind spots, and occasional broadband failures. To ensure business continuity and user experience, ordering systems typically need offline operation capabilities, enabling terminals to handle customer browsing, ordering, and temporary settlement operations even in environments with offline or weak networks. However, with extended offline time and sustained peak business hours, offline terminals accumulate a large amount of complex business data, often with logical connections between orders, tables, payments, and operational statuses, posing significant challenges to subsequent data synchronization and consistency maintenance.

[0003] When offline terminals re-establish network connections with the central server, a crucial issue that ordering systems need to address is how to reliably and orderly synchronize backlogged local data to the cloud. Traditional disordered upload methods struggle to balance data dependencies and terminal resource status, easily leading to improper data synchronization order and potentially consuming computing resources and network bandwidth due to high-concurrency uploads, thus affecting the stability of front-end ordering operations. In multi-terminal collaborative scenarios, delayed offline data uploads may also cause merging conflicts with existing data in the cloud. If the conflict handling mechanism is too simplistic, it can easily lead to data overwriting, synchronization blockage, or decreased processing efficiency. Therefore, a data synchronization mechanism that comprehensively considers data sharding status, business dependencies, terminal resource conditions, and server operating environment is needed to improve the consistency, stability, and business continuity of offline ordering data synchronization. Summary of the Invention

[0004] To address the problems in existing technologies where offline ordering data backlogs and is uploaded in an unordered manner, easily disrupting data dependencies, consuming terminal resources, and being difficult to adapt to cloud-based conflict handling environments, this invention proposes a method and system for offline data fragmentation and synchronization in ordering systems.

[0005] In a first aspect, this invention proposes an offline data sharding and synchronization method for an order system, comprising the following steps:

[0006] On the offline terminal side, the order data is divided into multiple data segments, and a status checkpoint is maintained for each data segment; the offline terminal calculates the synchronization tendency quantum for each data segment that has changed based on the correlation weight matrix between the multiple data segments. When the offline terminal establishes a network connection with the central server, the offline terminal sends a synchronization negotiation request containing the state checkpoint and the synchronization tendency quantum. After receiving the synchronization negotiation request, the central server determines the optimal starting synchronization fragment based on the fragment state and dependency relationship represented by the state checkpoint and in combination with the synchronization tendency quantum. It then returns to the offline terminal a synchronization start vector composed of the optimal starting synchronization fragment and a heuristic guidance strategy containing the minimum synchronization tendency threshold. The offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy, and the local association weight matrix. Before uploading each data fragment, it performs resource verification, evaluates the available resource quantity, and adjusts the resource consumption threshold according to the synchronization tendency quantum. When the available resource quantity is not lower than the resource consumption threshold, it initiates the data upload of the data fragment. If the central server detects a data merging conflict, it queries the conflict resolution strategy table based on the comprehensive available processing capacity score, and selects and executes the corresponding conflict resolution operation between generating derived data records and triggering the asynchronous merging queue.

[0007] Optionally, dividing the order data into multiple data segments on the offline terminal side includes: Extract the store identifier, table identifier, and order timestamp from the order data; The order data is allocated to a basic hash bucket using the combination of the store identifier and the table identifier as the partition key; Within the same basic hash bucket, data with adjacent timestamps are clustered and spliced ​​according to a preset fixed time window to obtain a single data fragment.

[0008] Optionally, the maintenance status checkpoint for each data shard includes: Record the timestamp sequence of each data change operation generated by the data shards since the last successful synchronization, calculate the time difference of all adjacent timestamps in the timestamp sequence to form a time interval vector, and calculate the variance of the time interval vector. The obtained variance values ​​are normalized to obtain a dimensionless fluctuation fingerprint. Based on the business identifier and operation type in the order data, determine the pre- and post-dependencies between each data segment, and generate segment dependency pointers according to the pre- and post-dependencies. Save the volatility fingerprint and the fragmentation dependency pointer to the corresponding state checkpoint.

[0009] Optionally, the offline terminal calculates a synchronization tendency quantum for each changed data shard based on the association weight matrix among the multiple data shards, including: The predicted frequency value of the changed data fragment is output through the data change frequency model, and the predicted frequency value is normalized according to the preset reference change frequency to obtain the frequency evaluation value. Find the dependency weight values ​​from the association weight matrix that point from the changed data shard to other data shards, and sum the dependency weight values ​​to obtain the association score; Extract the fixed peak-valley weighting coefficient from the parameters of the business operation period; Multiplying the frequency evaluation value, the correlation score, and the fixed peak-valley weight coefficient yields the synchronization tendency quantum of the changed data shard.

[0010] Optionally, the offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy, and the local association weight matrix, including: Extract all unsynchronized shards to form a candidate set; Unsynchronized fragments with a synchronization tendency quantum lower than the minimum synchronization tendency threshold and unsynchronized fragments that hit the prohibition synchronization rule are removed from the candidate set; Based on the shard dependency pointers, perform pre-dependency checks on the remaining candidate set and retain data shards without unsynchronized pre-dependencies. For each unsynchronized fragment in the remaining candidate set, calculate the sum of its weights relative to the optimal starting synchronization fragment in the local association weight matrix. Under the condition of satisfying the prerequisite dependency, arrange the fragments in descending order of the sum of their weights to generate the subsequent synchronization sequence.

[0011] Optionally, the step of performing resource verification and assessing the amount of available resources before uploading each of the data fragments includes: The first indicator is the average uplink rate of data from the device over the past minute, which is obtained by calling the network interface in real time. The battery management module is called to obtain the remaining power percentage value normalized to the range of 0 to 1 as the second indicator; The bandwidth index value is obtained by dividing the first index by the set reference rate and then multiplying it by a fixed bandwidth weight. The power index value is obtained by multiplying the second index by a fixed power weight. The available resource quantity under the current environment is obtained by adding the bandwidth index value and the power index value.

[0012] Optionally, adjusting the resource consumption threshold according to the synchronization tendency quantum includes: Divide the preset constant by the synchronization tendency quantum of the data slice to obtain the scaling parameter; The resource consumption threshold for the data shard is obtained by multiplying the system's preset basic resource consumption threshold by the reduction ratio parameter, so that the resource consumption threshold corresponding to the data shard with a higher synchronization tendency quantum is lower.

[0013] Optionally, the step of selecting and executing the corresponding conflict resolution operation between generating derived data records and triggering the asynchronous merge queue includes: Read the overall available processing capacity score; When the overall available processing power score is greater than the set fixed computing power threshold, a derived data record with an incrementing version number suffix is ​​created to store the changed data. If the overall available processing capacity score is not greater than the set fixed computing power threshold, the changed data is written into the asynchronous merging queue to wait for the central server to be in a period of idle load for delayed merging processing.

[0014] On the other hand, the present invention also proposes an offline data sharding and synchronization system for an order system, comprising the following modules: The calculation module is used to divide the order data into multiple data segments on the offline terminal side and maintain a status checkpoint for each data segment; the offline terminal calculates the synchronization tendency quantum for each changed data segment based on the correlation weight matrix between the multiple data segments. An adjustment module is used to: when the offline terminal establishes a network connection with the central server, the offline terminal sends a synchronization negotiation request containing the state checkpoint and the synchronization tendency quantum; after receiving the synchronization negotiation request, the central server determines the optimal starting synchronization fragment based on the fragment state and dependency relationship represented by the state checkpoint and in combination with the synchronization tendency quantum, and returns to the offline terminal a synchronization start vector composed of the optimal starting synchronization fragment and a heuristic guidance strategy containing the minimum synchronization tendency threshold; the offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy, and the local association weight matrix, and performs resource verification before uploading each data fragment, evaluates the available resource quantity, and adjusts the resource consumption threshold according to the synchronization tendency quantum; when the available resource quantity is not lower than the resource consumption threshold, the data upload of the data fragment is initiated; The generation module is used to, if the central server detects a data merging conflict, query the conflict resolution strategy table based on the comprehensive available processing capacity score, select between generating derived data records and triggering an asynchronous merging queue, and execute the corresponding conflict resolution operation.

[0015] Preferably, the step of dividing the order data into multiple data segments on the offline terminal side includes: Extract the store identifier, table identifier, and order timestamp from the order data; The order data is allocated to a basic hash bucket using the combination of the store identifier and the table identifier as the partition key; Within the same basic hash bucket, data with adjacent timestamps are clustered and spliced ​​according to a preset fixed time window to obtain a single data fragment.

[0016] Preferably, the maintenance status checkpoint for each data shard includes: Record the timestamp sequence of each data change operation generated by the data shards since the last successful synchronization, calculate the time difference of all adjacent timestamps in the timestamp sequence to form a time interval vector, and calculate the variance of the time interval vector. The obtained variance values ​​are normalized to obtain a dimensionless fluctuation fingerprint. Based on the business identifier and operation type in the order data, determine the pre- and post-dependencies between each data segment, and generate segment dependency pointers according to the pre- and post-dependencies. Save the volatility fingerprint and the fragmentation dependency pointer to the corresponding state checkpoint.

[0017] Preferably, the offline terminal calculates a synchronization tendency quantum for each changed data shard based on the correlation weight matrix among the multiple data shards, including: The predicted frequency value of the changed data fragment is output through the data change frequency model, and the predicted frequency value is normalized according to the preset reference change frequency to obtain the frequency evaluation value. Find the dependency weight values ​​from the association weight matrix that point from the changed data shard to other data shards, and sum the dependency weight values ​​to obtain the association score; Extract the fixed peak-valley weighting coefficient from the parameters of the business operation period; Multiplying the frequency evaluation value, the correlation score, and the fixed peak-valley weight coefficient yields the synchronization tendency quantum of the changed data shard.

[0018] Preferably, the offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy, and the local association weight matrix, including: Extract all unsynchronized shards to form a candidate set; Unsynchronized fragments with a synchronization tendency quantum lower than the minimum synchronization tendency threshold and unsynchronized fragments that hit the prohibition synchronization rule are removed from the candidate set; Based on the shard dependency pointers, perform pre-dependency checks on the remaining candidate set and retain data shards without unsynchronized pre-dependencies. For each unsynchronized fragment in the remaining candidate set, calculate the sum of its weights relative to the optimal starting synchronization fragment in the local association weight matrix. Under the condition of satisfying the prerequisite dependency, arrange the fragments in descending order of the sum of their weights to generate the subsequent synchronization sequence.

[0019] Preferably, the step of performing resource verification and assessing the amount of available resources before uploading each of the data fragments includes: The first indicator is the average uplink rate of data from the device over the past minute, which is obtained by calling the network interface in real time. The battery management module is called to obtain the remaining power percentage value normalized to the range of 0 to 1 as the second indicator; The bandwidth index value is obtained by dividing the first index by the set reference rate and then multiplying it by a fixed bandwidth weight. The power index value is obtained by multiplying the second index by a fixed power weight. The available resource quantity under the current environment is obtained by adding the bandwidth index value and the power index value.

[0020] Preferably, adjusting the resource consumption threshold according to the synchronization tendency quantum includes: Divide the preset constant by the synchronization tendency quantum of the data slice to obtain the scaling parameter; The resource consumption threshold for the data shard is obtained by multiplying the system's preset basic resource consumption threshold by the reduction ratio parameter, so that the resource consumption threshold corresponding to the data shard with a higher synchronization tendency quantum is lower.

[0021] Preferably, the step of selecting and executing the corresponding conflict resolution operation between generating derived data records and triggering the asynchronous merge queue includes: Read the overall available processing capacity score; When the overall available processing power score is greater than the set fixed computing power threshold, a derived data record with an incrementing version number suffix is ​​created to store the changed data. If the overall available processing capacity score is not greater than the set fixed computing power threshold, the changed data is written into the asynchronous merging queue to wait for the central server to be in a period of idle load for delayed merging processing.

[0022] This invention improves the targeting of offline data synchronization assessment by fragmenting offline order data on the terminal side, maintaining state checkpoints to characterize fragment changes and dependencies, and calculating synchronization tendency quantum using an association weight matrix. After network recovery, a synchronization start vector and heuristic guidance strategy are obtained through a synchronization negotiation mechanism to deduce the subsequent synchronization sequence, reducing data dependency errors and resource consumption risks caused by disordered uploads. Before uploading, resource consumption thresholds are dynamically adjusted based on available terminal resources to reduce lag or transmission interruptions caused by insufficient resources. When data merging conflicts occur, the central server selects either a derived record or an asynchronous merging queue based on the comprehensive available processing capacity score, improving the consistency of multi-terminal data processing and system stability. Attached Figure Description

[0023] Figure 1 A flowchart of the first embodiment; Figure 2 A schematic diagram showing the correlation between prediction frequency and synchronization tendency quantum under different business periods; Figure 3 This is a schematic diagram illustrating the correlation between the average uplink speed and available resource score for different terminals with remaining battery power. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In the first embodiment, the present invention proposes an offline data fragmentation and synchronization method for an order system, such as... Figure 1 As shown, it includes the following steps: S1, on the offline terminal side, the order data is divided into multiple data segments, and a status checkpoint is maintained for each data segment; the offline terminal calculates the synchronization tendency quantum for each data segment that has changed based on the correlation weight matrix between the multiple data segments.

[0026] On the offline terminal side, locally generated order data is divided into multiple data shards with fixed granularity according to order timestamps and business categories. When generating each data shard, the offline terminal configures a shard version number and maintains this version number as metadata along with the shard content. When a data shard is added, modified, deleted, or its status is updated, the corresponding shard version number is incremented. The offline terminal periodically scans each data shard, creates a status checkpoint for each shard, and stores it in a Redis cache, which serves as a unified location for reading and updating status checkpoints. When generating status checkpoints, the offline terminal generates a volatility fingerprint based on the fluctuation of the change time interval since the last successful synchronization of the data shard. Simultaneously, based on the business identifier and operation type in the order data, it identifies the pre- and post-dependencies between data shards and converts these dependencies into shard dependency pointers. The volatility fingerprint and the shard dependency pointers are then saved to the corresponding status checkpoint.

[0027] The offline terminal further combines the logical dependencies of the data shards and the historical concurrent update intensity to construct a non-negative directed association weight matrix. When a data shard is detected to have an addition, modification or deletion operation, the terminal calculates the synchronization tendency quantum of the data shard based on the normalized prediction frequency value, association score and peak and valley weight coefficient of the current business period, which serves as a dimensionless synchronization priority score.

[0028] In an optional embodiment, dividing the order data into multiple data fragments on the offline terminal side includes: Extract the store identifier, table identifier, and order timestamp from the order data; The order data is allocated to a basic hash bucket using the combination of the store identifier and the table identifier as the partition key; Within the same basic hash bucket, data with adjacent timestamps are clustered and spliced ​​according to a preset fixed time window to obtain a single data fragment.

[0029] The system monitors real-time order data write events generated by the ordering application, extracting the store identifier (e.g., Store_ID="S1002"), table identifier (e.g., Table_ID="T015"), and Unix order timestamp (e.g., Timestamp=1683012500000) from each JSON-formatted order data. A string concatenation function is used to concatenate the store identifier and table identifier into a partition key in the format "Store_ID:Table_ID", such as "S1002:T015". The MurmurHash3 algorithm is then applied to this partition key to calculate a 32-bit hash value. This 32-bit hash value is moduloed by the total number of basic hash buckets, for example, set between 128 and 512 (e.g., 256). The order data is then routed and assigned to the memory queue of the corresponding basic hash bucket, such as bucket number 47.

[0030] Within the same basic hash bucket, the data in the memory queue is periodically scanned, with a preset fixed time window size, for example, between 5 and 15 minutes. In this example, 10 minutes (600,000 milliseconds) is used as the baseline to divide the time interval. By traversing and extracting order data records with adjacent timestamps, all offline order events belonging to the same time window, such as order addition and cancellation records within the time interval [1683012000000, 1683012600000], are clustered and concatenated in chronological order. This is serialized into a single Protobuf format data fragment containing a complete transaction block. A unique fragment identifier is generated for each fragment and stored in the local SQLite database for synchronization.

[0031] In an optional embodiment, the maintenance status checkpoint for each data shard includes: Record the timestamp sequence of each data change operation generated by the data shards since the last successful synchronization, calculate the time difference of all adjacent timestamps in the timestamp sequence to form a time interval vector, and calculate the variance of the time interval vector. The obtained variance values ​​are normalized to obtain a dimensionless fluctuation fingerprint. Based on the business identifier and operation type in the order data, determine the pre- and post-dependencies between each data segment, and generate segment dependency pointers according to the pre- and post-dependencies. Save the volatility fingerprint and the fragmentation dependency pointer to the corresponding state checkpoint.

[0032] A change log table is maintained for each data shard in the local database, recording the Unix timestamps of every add, modify, or delete operation that occurred within that data shard since the last synchronization confirmation point. These timestamps are arranged chronologically to form a timestamp sequence, for example, T=[1683012000,1683012015,1683012045,1683012060]. All adjacent timestamp elements in this sequence are iterated through, and the absolute time difference is calculated by subtracting the previous one from the next. These differences are then combined sequentially to form a time interval vector, for example, ΔT=[15,30,15] based on the aforementioned sequence. The average value of this time interval vector is calculated (in this example, the average is 20 seconds), and the variance is calculated by summing the squares of the deviations of each time interval value from the average. For example, the variance is calculated to be 50 squared seconds.

[0033] The variance value is divided by a pre-set maximum extreme value parameter with the dimension of time squared. This parameter is obtained by training with historical 30-day point-interval data, for example, set between 300 and 1000 seconds squared, such as 500 seconds squared, and then normalized to obtain a dimensionless volatility fingerprint, such as 50 / 500=0.10. The volatility fingerprint is then updated in real time to a specific field of the corresponding shard status checkpoint in the Redis cache using a single-precision floating-point data format. The volatility fingerprint is used to characterize the degree of volatility in the data shard change rhythm.

[0034] Simultaneously, the offline terminal identifies the business dependencies between data shards based on the main order identifier, sub-order identifier, table status identifier, payment status identifier, and operation type in the order data. When the operation result recorded in one data shard constitutes a prerequisite for the continued processing of another data shard, the former is identified as the preceding shard, and the latter as the following shard. The corresponding preceding shard identifier, following shard identifier, and dependency type are saved as shard dependency relationship pointers. These shard dependency relationship pointers are recorded using logical identifier references and are used to identify suitable data shards as synchronization starting points during subsequent synchronization negotiation.

[0035] In an optional embodiment, the offline terminal calculates a synchronization tendency quantum for each changed data shard based on the association weight matrix among the plurality of data shards, including: The predicted frequency value of the changed data fragment is output through the data change frequency model, and the predicted frequency value is normalized according to the preset reference change frequency to obtain the frequency evaluation value. Find the dependency weight values ​​from the association weight matrix that point from the changed data shard to other data shards, and sum the dependency weight values ​​to obtain the association score; Extract the fixed peak-valley weighting coefficient from the parameters of the business operation period; Multiplying the frequency evaluation value, the correlation score, and the fixed peak-valley weight coefficient yields the synchronization tendency quantum of the changed data shard.

[0036] A data change frequency model based on an exponential smoothing algorithm is deployed on the terminal side. The model reads the number of changes to the changed data fragment within the most recent 10 preset statistical periods and converts the number of changes within each statistical period into a change frequency per unit time. Then, iterative smoothing calculations are performed on the change frequency per unit time according to a preset smoothing coefficient, outputting the predicted frequency value for the data fragment. The smoothing coefficient can be set between 0.2 and 0.5, for example, 0.3. For example, if the predicted frequency value output by the model is 2.5 times per minute, before participating in synchronous tendency quantum computing, the terminal normalizes the predicted frequency value according to a preset reference change frequency to obtain a dimensionless frequency evaluation value; for example, when the preset reference change frequency is 1 time per minute, the frequency evaluation value is 2.5.

[0037] A directed graph association weight matrix, constructed from the logical dependencies between data shards and historical concurrent update intensities, is maintained in memory. When maintaining this matrix, the offline terminal first determines the direction of the directed edges between data shards based on shard dependency pointers, using the preceding shard as the starting point and the following shard as the ending point, and assigns basic dependency weights according to dependency types. Then, historical order records are read, and the number of times each data shard undergoes a change within the same or adjacent statistical windows is counted. The historical concurrent update intensity is calculated based on the ratio of the number of common changes to the corresponding shard's historical change count. Finally, the basic dependency weights and the historical concurrent update intensities are weighted and fused using a preset fusion coefficient to obtain the association weight values ​​of the corresponding directed edges, and each association weight value is written into the directed graph association weight matrix.

[0038] When calculating the target data shard, the row vectors corresponding to the changed data shards in the matrix are scanned, and the dependency weight values ​​pointing from the data shard to other data shards are extracted. The weight values ​​are preset to range from 0.1 to 1.0. These weight values ​​are then summed as a scalar, and the association score representing the degree to which the data shard is dependent on other shards is output. For example, if two dependencies are found with weights of 0.6 and 0.8 respectively, the association score is 1.4. During the integration phase, the terminal accesses the system clock and refers to a pre-configured business time period mapping table to extract fixed peak and valley weight coefficients from the business operation time period parameters of the current system time. For example, the configuration is as follows: the coefficient is 1.5 for peak dining periods such as 12:00-13:30, 1.0 for off-peak periods, and 0.5 for off-peak periods such as the early morning. If the current time is 12:15 noon, the coefficient is 1.5. The obtained frequency evaluation value of 2.5, correlation score of 1.4, and fixed peak-valley weight coefficient of 1.5 are multiplied together (2.5 × 1.4 × 1.5) to calculate a result of 5.25. This value is used as the synchronization tendency quantum, representing the urgency of uploading the changed data fragment, and is temporarily stored in the synchronization scheduling task pool. The correlation curves between predicted frequency and synchronization tendency quantum under different business periods are shown below. Figure 2 As shown.

[0039] S2, when the offline terminal establishes a network connection with the central server, the offline terminal sends a synchronization negotiation request containing the state checkpoint and the synchronization tendency quantum; after receiving the synchronization negotiation request, the central server determines the optimal starting synchronization fragment based on the fragment state and dependency relationship represented by the state checkpoint and in combination with the synchronization tendency quantum, and returns to the offline terminal a synchronization start vector composed of the optimal starting synchronization fragment and a heuristic guidance strategy containing the minimum synchronization tendency threshold; the offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy and the local association weight matrix, and performs resource verification before uploading each data fragment, evaluates the available resource quantity, and adjusts the resource consumption threshold according to the synchronization tendency quantum; when the available resource quantity is not lower than the resource consumption threshold, the data upload of the data fragment is started.

[0040] The offline terminal continuously monitors network connectivity. When it detects that the device has switched from offline to successfully establishing a TCP long connection with the central server, it serializes the state checkpoints and synchronization tendency extracted from the Redis cache into a JSON-formatted payload and sends a synchronization negotiation request to the central server via HTTPS.

[0041] After receiving a synchronization negotiation request, the central server extracts the state checkpoints and synchronization tendency quantum. Based on the volatility fingerprint in the state checkpoints, it determines the degree of volatility in data shard changes and identifies the preceding dependency states of each data shard based on the shard dependency pointers. The central server first excludes data shards with unsynchronized preceding shards from the candidate set, resulting in a set of candidate data shards without unsynchronized preceding dependencies. Then, from this set of candidate data shards, it selects the data shard with the highest synchronization tendency quantum as the optimal starting synchronization shard. If multiple data shards have the same synchronization tendency quantum, the data shard with the higher volatility fingerprint value is selected first. If they are still the same, one data shard is determined as the optimal starting synchronization shard according to a preset shard identifier order. The central server encapsulates the identifier of the optimal starting synchronization shard into a synchronization start vector.

[0042] Simultaneously, the central server reads historical synchronization records, extracts the synchronization tendency quantum and the corresponding actual synchronization delay from each historical record, forming sample pairs. The actual synchronization delay is calculated based on the upload start time and synchronization confirmation time of data fragments during the historical synchronization process. The central server first standardizes the sample pairs, then uses the K-means clustering algorithm to cluster them, obtaining multiple synchronization performance sample clusters. Each sample cluster contains historical samples with similar synchronization tendency quantum values ​​and similar actual synchronization delay performance. The central server calculates the average actual synchronization delay within each sample cluster and determines the sample clusters whose average delay is not higher than a preset delay target as target sample clusters. If multiple target sample clusters exist, the sample cluster with the smallest average synchronization delay is selected as the target sample cluster. Subsequently, the central server extracts the minimum value of the synchronization tendency quantum within the target sample cluster as the minimum synchronization tendency threshold. The central server further reads the rate limiting control conditions configured in the rule engine Drools. The rate limiting control conditions include the maximum number of concurrent uploads, the upper limit of the server receiving queue length, the network types allowed for synchronization, and the data types prohibited from synchronization. The minimum synchronization tendency threshold is combined with the rate limiting control conditions to generate a heuristic guidance strategy.

[0043] The central server encapsulates the synchronization start vector and heuristic bootstrapping strategy into an HTTP response message and returns it to the offline terminal. Upon receiving the message, the offline terminal uses the data fragments in the synchronization start vector as the synchronization starting point, combines them with the local association weight matrix, performs pre-dependency checks on the candidate fragments, and, under the condition of satisfying the constraints of the heuristic bootstrapping strategy, generates a sequentially arranged subsequent synchronization sequence according to the association weight between the candidate fragments and the optimal starting synchronization fragment.

[0044] Before traversing the subsequent synchronization sequence and preparing to upload each data fragment, the offline terminal obtains the average uplink rate and the current remaining battery percentage over the past minute. After dimensionless and weighted processing, the available resources of the device are obtained. Simultaneously, based on the synchronization tendency quantum of the data fragment to be uploaded, a preset baseline resource threshold is inversely scaled to obtain the corresponding resource consumption threshold. When the available resources are not lower than the resource consumption threshold, the offline terminal starts a background transmission thread and uploads the data fragment to the central server through a streaming interface.

[0045] In an optional embodiment, the offline terminal derives the subsequent synchronization sequence based on the synchronization initiation vector, the heuristic guidance strategy, and the local association weight matrix, including: Extract all unsynchronized shards to form a candidate set; Unsynchronized fragments with a synchronization tendency quantum lower than the minimum synchronization tendency threshold and unsynchronized fragments that hit the prohibition synchronization rule are removed from the candidate set; Based on the shard dependency pointers, perform pre-dependency checks on the remaining candidate set and retain data shards without unsynchronized pre-dependencies. For each unsynchronized fragment in the remaining candidate set, calculate the sum of its weights relative to the optimal starting synchronization fragment in the local association weight matrix. Under the condition of satisfying the prerequisite dependency, arrange the fragments in descending order of the sum of their weights to generate the subsequent synchronization sequence.

[0046] Scan the local data synchronization status table, extract all unsynchronized shard objects marked as pending synchronization, and load their unique identifiers and attribute pointers into a memory list to form an initial candidate set, such as shards A, B, C, D, and E. Parse the minimum synchronization tendency threshold from the heuristic guidance policy issued by the server (set to 2.0 in this example). Compare the synchronization tendency quantum of each unsynchronized shard in the candidate set, and simultaneously query the local policy library to verify whether the shard has violated a synchronization prohibition rule. For example, if the current network is in a restricted subnet, or the shard contains undo operation records requiring manual review, shards with a quantum value below 2.0 (e.g., shard D with a quantum value of 1.5) or that have violated a synchronization prohibition rule are removed from the candidate set through a dequeue operation. This achieves filtering and noise reduction, leaving shards A, B, and C in the filtered set.

[0047] Read the shard dependency pointers to determine if any of the remaining candidate shards have unsynchronized predecessor shards. If an unsynchronized predecessor shard exists, postpone the sorting of the candidate shard or place it after the corresponding predecessor shard. Extract the optimal starting synchronization shard carried in the synchronization start vector, for example, the basic master order shard O. Using a breadth-first search algorithm, traverse the candidate data shards along the directed edges in the locally stored association weight matrix, with the optimal starting synchronization shard as the source node. Calculate the path association weights from the optimal starting synchronization shard to each candidate data shard. For example, calculate the weight of shard A as 3.5, shard B as 4.2, and shard C as 2.1. The quicksort algorithm is invoked to sort the unsynchronized fragments in descending order from largest to smallest. During the sorting process, the dependency constraint that the preceding fragments take precedence over the following fragments is maintained, generating a queue of subsequent synchronization sequences that is ordered. That is, the sorting result is [fragment B, fragment A, fragment C]. This ensures that the fragments of point-to-point data that are strongly related to and highly dependent on the business of the starting node enter the network transmission channel first, and reduces the risk that subsequent business data is uploaded before preceding business data.

[0048] In an optional embodiment, the step of performing resource verification and assessing the amount of available resources before uploading each of the data fragments includes: The first indicator is the average uplink rate of data from the device over the past minute, which is obtained by calling the network interface in real time. The battery management module is called to obtain the remaining power percentage value normalized to the range of 0 to 1 as the second indicator; The bandwidth index value is obtained by dividing the first index by the set reference rate and then multiplying it by a fixed bandwidth weight. The power index value is obtained by multiplying the second index by a fixed power weight. The available resource quantity under the current environment is obtained by adding the bandwidth index value and the power index value.

[0049] Before each fragmented upload is triggered, the total number of bytes sent by Tx within a 60-second time window is statistically analyzed in real time using the underlying TrafficStats API of the operating system, and the average uplink rate is calculated. This average rate is recorded as the first indicator; for example, the actual measured average uplink rate is 1.5Mbps. The device's built-in battery management module interface is called via the system bus to read the absolute remaining battery percentage of the current device's battery pack. For example, if the battery level is 45%, it is 0.45. This value is stored in a double-precision floating-point variable as the second indicator.

[0050] During the resource measurement calculation phase, standard reference rate parameters are pre-configured in the configuration file, for example, set between 2.0Mbps and 10.0Mbps (5.0Mbps in this example), along with two environmental weight coefficients: a fixed bandwidth weight (0.7 in this example, between 0.6 and 0.8) and a fixed power weight (0.3 in this example, between 0.2 and 0.4). Dividing the first indicator 1.5Mbps by the reference rate 5.0Mbps yields a dimensionless network health score of 0.3. This network health score is then multiplied by the fixed bandwidth weight of 0.7 to calculate a bandwidth index value of 0.21. Multiplying the second indicator 0.45 by the fixed power weight of 0.3 yields a power index value of 0.135. The bandwidth and power index values ​​are then summed to obtain a scalar value of available resources with a score of 0.345, used for subsequent resource consumption threshold matching. The correlation curve between the average uplink rate and available resource score for different terminals with remaining power is shown below. Figure 3 As shown.

[0051] In an optional embodiment, adjusting the resource consumption threshold according to the synchronization tendency quantum includes: Divide the preset constant by the synchronization tendency quantum of the data slice to obtain the scaling parameter; The resource consumption threshold for the data shard is obtained by multiplying the system's preset basic resource consumption threshold by the reduction ratio parameter, so that the resource consumption threshold corresponding to the data shard with a higher synchronization tendency quantum is lower.

[0052] A preset constant is loaded from the environment configuration file. This constant is used to smooth the impact of quantum changes on the threshold and is in the same dimensionless scoring system as the synchronization tendency quantum, for example, set between 1.0 and 5.0, such as 4.0. The preset basic resource consumption threshold is set between 0.3 and 0.6, such as 0.5. The synchronization tendency quantum variable value currently allocated to the data shard is read. If the synchronization tendency quantum is lower than the preset quantum lower limit, the preset quantum lower limit is used in the calculation. Assuming it is a high-priority, high-propensity shard with a quantum value of 8.0, the preset constant 4.0 is divided by the synchronization tendency quantum 8.0 through division, and the resulting scaling parameter is 0.5.

[0053] After obtaining the scaling factor, this parameter is multiplied by a preset base resource consumption threshold of 0.5, i.e., 0.5 × 0.5 = 0.25. This derives and outputs the resource consumption threshold specific to the currently pending data fragment. This design mechanism, which maps through an inverse proportional relationship, ensures that when an offline order fragment exhibits a high synchronization tendency due to frequent modifications or numerous core dependencies (e.g., a backlog of table order records), the calculated resource consumption threshold will automatically decrease proportionally. Therefore, when the device is in a resource-scarce state with weak network or low power, this high-priority order data fragment is more likely to reach the judgment threshold and obtain priority transmission opportunities.

[0054] S3. If the central server detects a data merging conflict, the central server queries the conflict resolution strategy table based on the comprehensive available processing capacity score, selects and executes the corresponding conflict resolution operation between generating derived data records and triggering the asynchronous merging queue.

[0055] When the central server receives data shards and persists them to the cloud-based MySQL database, it reads the shard version number carried in the shard metadata and compares this shard version number with the existing version number corresponding to the same shard identifier in the database using an optimistic locking mechanism combined with the database's multi-version concurrency control mechanism. If the version number of the currently uploaded data shard is lower than an existing version number, or if the version numbers are the same but the business data content is inconsistent, a data merge conflict is determined to have occurred, and a data merge conflict alarm is triggered.

[0056] After triggering a data merging conflict alarm, the central server collects the current server CPU idle rate, available memory percentage, number of concurrent requests, and network latency, and normalizes these parameters. The server CPU idle rate and available memory percentage are used as positive parameters, while the number of concurrent requests and network latency are used as negative parameters. The central server then performs a weighted calculation on the normalized parameters according to preset weights to obtain a comprehensive available processing capability score. This comprehensive available processing capability score characterizes the sufficiency of resources currently available for real-time conflict processing on the server; a higher score indicates a stronger real-time conflict processing capability.

[0057] The central server reads a pre-configured conflict resolution strategy table from memory. The conflict resolution strategy table is arranged in order according to the comprehensive available processing capability score interval, and each comprehensive available processing capability score interval corresponds to a conflict resolution action instruction.

[0058] The real-time calculated comprehensive available processing capacity score is input into the binary search algorithm to quickly locate the corresponding interval. If the system is matched to an interval with low available processing capacity, the central server uses the chain of responsibility pattern to trigger the asynchronous merging queue. The RabbitMQ message queue middleware is called to encapsulate the conflicting data shards and snapshots into messages and deliver them to the background delayed consumption queue, awaiting manual intervention during off-peak hours or delayed retries. If the system is matched to an interval with high available processing capacity, the central server uses the abstract factory design pattern to generate derived data records. A three-way merging algorithm is called to compare the differences, updating the non-conflicting fields to the main record, and appending the conflicting fields as independent key-value pairs to a dedicated sub-table to generate derived data records. An internal message containing the difference comparison results is sent to the administrator.

[0059] In an optional embodiment, selecting and executing the corresponding conflict resolution operation between generating derived data records and triggering the asynchronous merge queue includes: Read the overall available processing capacity score; When the overall available processing power score is greater than the set fixed computing power threshold, a derived data record with an incrementing version number suffix is ​​created to store the changed data. If the overall available processing capacity score is not greater than the set fixed computing power threshold, the changed data is written into the asynchronous merging queue to wait for the central server to be in a period of idle load for delayed merging processing.

[0060] After receiving the fragments uploaded by the offline ordering terminal, the central server reads the comprehensive available processing capacity score in real time through the cloud resource scheduler. This score is obtained by weighting the server's CPU idle rate, available memory percentage, number of concurrent requests, and network latency after normalization, with a value range of [0,1]. For example, the current score is 0.75. A higher server CPU idle rate and available memory percentage contribute more positively to the comprehensive available processing capacity score; conversely, a higher number of concurrent requests and network latency have a greater negative impact on the comprehensive available processing capacity score.

[0061] The background conflict resolution module's judgment logic compares the score with a pre-configured fixed computing power threshold parameter in the database table. This parameter is established based on the server's benchmark load test performance, for example, set between 0.6 and 0.8, with an example setting of 0.7. If the judgment condition is met, i.e., the current comprehensive available processing capacity score of 0.75 is detected to be greater than the set fixed computing power threshold of 0.70, it indicates that the central server has sufficient real-time processing capacity at this moment. It immediately clones the main record row of the conflict, and generates a new unique identifier by concatenating the original record's primary key, such as OrderID, with an incrementing version number suffix, such as "_V2", for example, OrderID_V2. A parallel derived data record is created and persisted in the relational database to fully save the conflicted change order data for subsequent manual review and comparison. Conversely, if the overall available processing capacity score drops to 0.45, which is not greater than the fixed computing power threshold of 0.7, it indicates that the cloud is currently experiencing a high-concurrency resource bottleneck. To reduce the risk of conflict handling disrupting core business operations, the intercepted data will be immediately written into the operation. The conflicted changed data will be encapsulated into a transaction body with a standard message format and sequentially pushed into the asynchronous merging queue of the message middleware. A timed listening task will be started, waiting for a period of idle load when the central server's CPU utilization drops below 30%, before pulling messages from this queue for latency comparison and merging processing.

[0062] In the second embodiment, the present invention also proposes an offline data sharding and synchronization system for an order system, comprising the following modules: The calculation module is used to divide the order data into multiple data segments on the offline terminal side and maintain a status checkpoint for each data segment; the offline terminal calculates the synchronization tendency quantum for each changed data segment based on the correlation weight matrix between the multiple data segments. An adjustment module is used to: when the offline terminal establishes a network connection with the central server, the offline terminal sends a synchronization negotiation request containing the state checkpoint and the synchronization tendency quantum; after receiving the synchronization negotiation request, the central server determines the optimal starting synchronization fragment based on the fragment state and dependency relationship represented by the state checkpoint and in combination with the synchronization tendency quantum, and returns to the offline terminal a synchronization start vector composed of the optimal starting synchronization fragment and a heuristic guidance strategy containing the minimum synchronization tendency threshold; the offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy, and the local association weight matrix, and performs resource verification before uploading each data fragment, evaluates the available resource quantity, and adjusts the resource consumption threshold according to the synchronization tendency quantum; when the available resource quantity is not lower than the resource consumption threshold, the data upload of the data fragment is initiated; The generation module is used to, if the central server detects a data merging conflict, query the conflict resolution strategy table based on the comprehensive available processing capacity score, select between generating derived data records and triggering an asynchronous merging queue, and execute the corresponding conflict resolution operation.

[0063] The above description represents the preferred embodiments of the present invention. It should be noted that, for those skilled in the art, various improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for offline data sharding and synchronization in an order-taking system, characterized in that, include: On the offline terminal side, the order data is divided into multiple data segments, and a status checkpoint is maintained for each data segment. The offline terminal calculates the synchronization tendency quantum for each changed data shard based on the correlation weight matrix between the multiple data shards. When the offline terminal establishes a network connection with the central server, the offline terminal sends a synchronization negotiation request containing the state checkpoint and the synchronization tendency quantum. After receiving the synchronization negotiation request, the central server determines the optimal starting synchronization fragment based on the fragment state and dependency relationship represented by the state checkpoint and in combination with the synchronization tendency quantum. It then returns to the offline terminal a synchronization start vector composed of the optimal starting synchronization fragment and a heuristic guidance strategy containing the minimum synchronization tendency threshold. The offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy, and the local association weight matrix. Before uploading each data fragment, it performs resource verification, evaluates the available resource quantity, and adjusts the resource consumption threshold according to the synchronization tendency quantum. When the available resource quantity is not lower than the resource consumption threshold, it initiates the data upload of the data fragment. If the central server detects a data merging conflict, it queries the conflict resolution strategy table based on the comprehensive available processing capacity score, and selects and executes the corresponding conflict resolution operation between generating derived data records and triggering the asynchronous merging queue.

2. The method according to claim 1, characterized in that, The process of dividing order data into multiple data segments on the offline terminal side includes: Extract the store identifier, table identifier, and order timestamp from the order data; The order data is allocated to a basic hash bucket using the combination of the store identifier and the table identifier as the partition key; Within the same basic hash bucket, data with adjacent timestamps are clustered and spliced ​​according to a preset fixed time window to obtain a single data fragment.

3. The method according to claim 2, characterized in that, The aforementioned checkpoints for maintaining the status of each data shard include: Record the timestamp sequence of each data change operation generated by the data shards since the last successful synchronization, calculate the time difference of all adjacent timestamps in the timestamp sequence to form a time interval vector, and calculate the variance of the time interval vector. The obtained variance values ​​are normalized to obtain a dimensionless fluctuation fingerprint. Based on the business identifier and operation type in the order data, determine the pre- and post-dependencies between each data segment, and generate segment dependency pointers according to the pre- and post-dependencies. Save the volatility fingerprint and the fragmentation dependency pointer to the corresponding state checkpoint.

4. The method according to claim 1, characterized in that, The offline terminal calculates a synchronization tendency quantum for each changed data shard based on the correlation weight matrix between the multiple data shards, including: The predicted frequency value of the changed data fragment is output through the data change frequency model, and the predicted frequency value is normalized according to the preset reference change frequency to obtain the frequency evaluation value. Find the dependency weight values ​​from the changed data shard to other data shards in the association weight matrix, and sum the dependency weight values ​​to obtain the association score; Extract the fixed peak-valley weighting coefficient from the parameters of the business operation period; Multiplying the frequency evaluation value, the correlation score, and the fixed peak-valley weight coefficient yields the synchronization tendency quantum of the changed data shard.

5. The method according to claim 4, characterized in that, The offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy, and the local association weight matrix, including: Extract all unsynchronized shards to form a candidate set; Unsynchronized fragments with a synchronization tendency quantum lower than the minimum synchronization tendency threshold and unsynchronized fragments that hit the prohibition synchronization rule are removed from the candidate set; Based on the shard dependency pointers, perform pre-dependency checks on the remaining candidate set and retain data shards without unsynchronized pre-dependencies. For each unsynchronized fragment in the remaining candidate set, calculate the sum of its weights relative to the optimal starting synchronization fragment in the local association weight matrix. Under the condition of satisfying the prerequisite dependency, arrange the fragments in descending order of the sum of their weights to generate the subsequent synchronization sequence.

6. The method according to claim 1, characterized in that, The process of performing resource verification and assessing available resource quantity before uploading each of the data fragments includes: The first indicator is the average uplink rate of data from the device over the past minute, which is obtained by calling the network interface in real time. The battery management module is called to obtain the remaining power percentage value normalized to the range of 0 to 1 as the second indicator; The bandwidth index value is obtained by dividing the first index by the set reference rate and then multiplying it by a fixed bandwidth weight. The power index value is obtained by multiplying the second index by a fixed power weight. The available resource quantity under the current environment is obtained by adding the bandwidth index value and the power index value.

7. The method according to claim 1, characterized in that, The step of adjusting the resource consumption threshold according to the synchronization tendency quantum includes: Divide the preset constant by the synchronization tendency quantum of the data slice to obtain the scaling parameter; The resource consumption threshold for the data shard is obtained by multiplying the system's preset basic resource consumption threshold by the reduction ratio parameter, so that the resource consumption threshold corresponding to the data shard with a higher synchronization tendency quantum is lower.

8. The method according to claim 1, characterized in that, The step of selecting and executing the corresponding conflict resolution operation between generating derived data records and triggering the asynchronous merge queue includes: Read the overall available processing capacity score; When the overall available processing power score is greater than the set fixed computing power threshold, a derived data record with an incrementing version number suffix is ​​created to store the changed data. If the overall available processing capacity score is not greater than the set fixed computing power threshold, the changed data is written into the asynchronous merging queue to wait for the central server to be in a period of idle load for delayed merging processing.

9. An offline data sharding and synchronization system for an order-taking system, characterized in that, include: The calculation module is used to divide the order data into multiple data segments on the offline terminal side and maintain a status checkpoint for each data segment. The offline terminal calculates the synchronization tendency quantum for each changed data shard based on the correlation weight matrix between the multiple data shards. An adjustment module is used to: when the offline terminal establishes a network connection with the central server, the offline terminal sends a synchronization negotiation request containing the state checkpoint and the synchronization tendency quantum; after receiving the synchronization negotiation request, the central server determines the optimal starting synchronization fragment based on the fragment state and dependency relationship represented by the state checkpoint and in combination with the synchronization tendency quantum, and returns to the offline terminal a synchronization start vector composed of the optimal starting synchronization fragment and a heuristic guidance strategy containing the minimum synchronization tendency threshold; the offline terminal derives the subsequent synchronization sequence based on the synchronization start vector, the heuristic guidance strategy, and the local association weight matrix, and performs resource verification before uploading each data fragment, evaluates the available resource quantity, and adjusts the resource consumption threshold according to the synchronization tendency quantum; when the available resource quantity is not lower than the resource consumption threshold, the data upload of the data fragment is initiated; The generation module is used so that if the central server detects a data merging conflict, the central server queries the conflict resolution strategy table based on the comprehensive available processing capacity score, selects between generating derived data records and triggering the asynchronous merging queue, and executes the corresponding conflict resolution operation.

10. The system according to claim 9, characterized in that, The process of dividing order data into multiple data segments on the offline terminal side includes: Extract the store identifier, table identifier, and order timestamp from the order data; The order data is allocated to a basic hash bucket using the combination of the store identifier and the table identifier as the partition key; Within the same basic hash bucket, data with adjacent timestamps are clustered and spliced ​​according to a preset fixed time window to obtain a single data fragment.

Images