A forest resource field investigation data synchronization method based on an offline priority architecture

Abstract

The application belongs to the technical field of mobile terminal data acquisition and synchronization, and discloses a forest resource field investigation data synchronization method based on an offline priority architecture, which comprises the following steps: establishing a hierarchical database architecture, and dividing the investigation data into three levels of user layer, company layer and project layer; constructing a three-level cache system of memory LRU cache, disk cache and browser local storage; adopting a replication protocol of a local document database to realize bidirectional incremental synchronization of hierarchical filtering, using a server filtering function for a pull synchronization channel and a local filtering function for a push synchronization channel; adopting a staggered start and idempotent switching mechanism for project layer synchronization; and detecting data conflicts in real time and automatically solving the conflicts by adopting a Last-Write-Wins strategy. The application solves the problems of poor network environment, multi-person collaboration conflict and large data transmission in forest field investigation, realizes data acquisition and intelligent synchronization in a completely offline environment, and is suitable for forest resource field investigation data synchronization.

Description

Technical Field

[0001] This invention belongs to the field of mobile data acquisition and synchronization technology, and relates to a method for synchronizing forest resource field survey data based on an offline-first architecture. Background Technology

[0002] Forest resource surveys are fundamental to forestry management, carbon sequestration measurement, and ecological protection. These surveys typically require field personnel to conduct in-depth measurements of sample plots, subplots, and individual trees within forest areas, collecting multi-dimensional data including tree species, diameter at breast height (DBH), tree height, canopy closure, and carbon sequestration parameters. However, forest field surveys have several significant characteristics: First, forest areas are often located in remote mountainous regions with extremely unstable mobile network coverage, frequently leaving equipment completely offline. Second, the types of data collected are complex, involving various formats such as tables, text, photographs, and spatial vectors. Third, surveys are often conducted in teams, including project leaders, chief engineers, technical leaders, and multiple surveyors, requiring team members to collaborate across different time periods and areas, ultimately compiling the collected data into a unified database. Fourth, forest resource data concerns state-owned assets, necessitating extremely high standards for data integrity, consistency, and security.

[0003] To address the aforementioned challenges, various mobile data collection and synchronization solutions have been proposed in existing technologies.

[0004] For example, traditional client / server (C / S) architecture requires mobile terminals to maintain a continuous network connection with the server, with data submitted directly to the server. This solution works well in stable network environments, but when applied to weak or offline environments in forest areas, data submission frequently fails, leading to a significant decrease in survey efficiency and even making it impossible to complete the task.

[0005] For example, some simple local caching solutions temporarily store data in the mobile device's local file system or a lightweight relational database (such as SQLite), and then upload it in batches after the network is restored. While such solutions support offline data entry, they typically only achieve one-way data flow and cannot provide real-time bidirectional synchronization between the local device and the server. When multiple people are working collaboratively, data uploaded from different devices is prone to version conflicts, and there is a lack of effective detection and automatic resolution mechanisms. Manual intervention to merge data is often required, increasing management costs and the risk of errors.

[0006] For example, message queue-based synchronization solutions decouple data production and consumption processes by introducing message middleware. However, this approach has a complex architecture and consumes significant computing, storage, and power resources from mobile terminals. Furthermore, it lacks a data tiering and filtering strategy tailored to forestry survey scenarios. Clients often need to synchronize large amounts of data from the server that are irrelevant to their current project or role, resulting in unnecessary network transmission and storage overhead, which is particularly inefficient in bandwidth-constrained forest environments.

[0007] Furthermore, in existing solutions, mobile data access performance often depends on network conditions. In offline or weak network environments, access latency to frequently used reference data increases significantly, impacting the user experience and work efficiency of field personnel. For sensitive data stored locally, existing solutions also generally lack targeted encryption protection measures, posing a risk of data leakage.

[0008] In summary, there is an urgent need for an offline-priority data synchronization method that can adapt to the field survey scenarios of forest resources, so as to achieve reliable data collection in weak network or even completely offline environments, efficient incremental synchronization after network recovery, automatic conflict resolution in multi-person collaborative operations, and optimize local data access performance and network transmission efficiency through multi-level caching and intelligent filtering strategies. Summary of the Invention

[0009] The purpose of this invention is to provide a method for synchronizing forest resource field survey data based on an offline-first architecture, in order to solve the technical problems of achieving multi-person collaborative data collection, offline storage, incremental synchronization, automatic conflict resolution, and multi-level cache management in forest field survey scenarios with extremely poor network environments.

[0010] To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0011] A method for synchronizing forest resource field survey data based on an offline-first architecture includes the following steps:

[0012] S1. Establish a local document database on the mobile terminal and deploy a remote document database on the server. On the server, divide the survey data into three synchronous layers according to business relevance: user layer, company layer, and project layer. The design document of the local document database contains local filtering functions, and the design document of the remote document database contains three server-side filtering functions, which correspond one-to-one with the user layer, company layer, and project layer, respectively.

[0013] S2. Construct a three-level caching system on the mobile terminal. The first level cache is a memory LRU cache, the second level cache is a disk cache of the embedded key-value storage engine, and the third level cache is the browser's local storage. When querying the cache, the first level cache is searched in order to the third level cache. If a match is found, the cache is returned. If all three levels of cache are not found, a network request is initiated to access the remote document database, and the response data is asynchronously populated back into each level of cache.

[0014] S3. A replication protocol for the local document database is adopted to achieve bidirectional incremental synchronization between the local document database and the remote document database. An independent pull synchronization channel and push synchronization channel are established for each local document database instance. The pull synchronization channel uses three server-side filtering functions of the remote document database during synchronization to filter data according to the corresponding synchronization level, and only sends matching document changes to the local document database. The push synchronization channel uses local filtering functions during synchronization to filter and skip data fields marked as stored locally only and whose data length exceeds the threshold by project identifier, and only sends matching document changes to the remote document database.

[0015] S4. The project layer synchronization adopts a staggered start strategy and ensures the correctness of project switching through idempotency checks and race condition protection. User layer and company layer data synchronization starts immediately after the user logs into the application.

[0016] S5. Real-time detection of data conflicts, and automatic resolution of conflicts using the Last-Write-Wins strategy based on update timestamps.

[0017] As a limitation, in step S1, the user layer is configured to filter project data according to user roles, specifically including: receiving documents and request parameters through the server-side filtering function corresponding to the user layer in the remote document database; firstly checking the deletion mark of the document and directly allowing documents with deletion marks to pass; for documents without deletion marks, further checking whether the company identifier of the document matches the company identifier in the request parameters; finally checking whether the user identifier in the request parameters matches any one of the following arrays in the document: creator identifier, project manager identifier, chief engineer identifier, technical manager identifier, or investigator identifier.

[0018] The company layer is configured to filter shared data by company identifier. The shared data includes: factor dictionary, tile image source, and forest tenure attribute information.

[0019] The project layer is configured to filter project-level data tables by project identifier. The project-level data tables include: zoning sub-compartment data table, forest tenure sub-compartment data table, sample plot data table, photo data table, actual diameter at breast height (DBH) data table, diameter class DBH data table, survey report data table, tree species summary data table, as well as carbon sink-related soil survey data table, shrub survey data table, herb survey data table, litter survey data table, fallen tree survey data table, carbon sink baseline survey data table, and bamboo forest survey data table.

[0020] As a second limitation, in step S2, the first-level cache is a fixed-capacity memory LRU cache maintained in the local application backend, with a capacity of N records, where N is a preset positive integer, used to store HTTP response data and image data, and the read / write latency is in the microsecond range;

[0021] In the second-level cache, data is encoded in MessagePack format and stored as a byte array. Each record is accompanied by a write timestamp, and the read / write latency is in the millisecond range.

[0022] In the third-level cache, data is serialized in JSON format. Each record contains three fields: value, timestamp, and expiration timestamp. Sensitive data is stored after being encrypted using the AES-GCM algorithm. The encryption key is derived from the device hardware's unique identifier using the PBKDF2 algorithm.

[0023] As a third limitation, in step S3, the bidirectional incremental synchronization further includes:

[0024] After the pull synchronization channel is established, a checkpoint document is maintained in the local document database to record the change sequence number reached during the last synchronization. Each time synchronization occurs, the pull starts from the changes after that sequence number. If the checkpoint document does not exist during the first synchronization, the full pull starts from the beginning of the change sequence.

[0025] Both the push synchronization channel and the pull synchronization channel use a long polling mode to maintain a persistent connection, configure a heartbeat interval to prevent intermediate network devices from disconnecting idle connections, and enable an automatic reconnection mechanism.

[0026] As a further limitation, in step S4, the staggered start strategy for project-level synchronization specifically includes: registering each project-level data table in the project layer to the synchronization registry; starting synchronization by sequentially starting the synchronization channel of each data table in the project-level data table at a preset time interval; and checking whether the currently active project identifier is still the target project identifier before each start, and canceling the start if it is not.

[0027] As a further limitation, in step S4, the project layer also adopts an automatic recovery mechanism: when the application starts, it reads the last active project identifier from the browser's local storage in the third-level cache and sets it as the current active project identifier. When each data model is initialized and registered, it detects that the identifier has been set and automatically starts background synchronization according to the off-peak strategy.

[0028] As the fourth type of limitation, the specific process of step S5 is as follows:

[0029] S51. Check the conflict markers for each change document in the data change listener;

[0030] S52. Obtain the current winning version and all conflicting versions of the conflicting document. Sort all versions in descending order of update timestamp. Select the version with the latest timestamp as the winning version. Mark the remaining versions as deleted and submit them to the local document database through batch operation. If the winning version is not the current active version, overwrite the current active version with the winning version.

[0031] As a fifth limitation, the forest resource field survey data synchronization method based on the offline priority architecture also includes: the first-level cache adopts the LRU eviction policy for lifecycle management;

[0032] The second-level cache uses a TTL expiration strategy for lifecycle management; the TTL expiration strategy includes configuring independent TTL durations for HTTP response data and image data in the second-level cache;

[0033] The third-level cache uses an on-demand expiration strategy for lifecycle management.

[0034] As a further limitation, the first-level cache, the second-level cache, and the third-level cache all adopt an active cleanup mechanism: providing a timed cleanup interface to batch clean up expired data in each level of cache according to the maximum lifespan, and returning the number of entries cleaned up.

[0035] As a sixth limitation, the forest resource field survey data synchronization method based on the offline priority architecture also includes resetting the synchronization operation: stopping the synchronization channel of the current local document database instance, destroying the local document database instance, recreating an empty local document database, and then using the server-side filtering function to pull all the data of the current project from the remote document database at once. After the pull is completed, the real-time synchronization mode is restored.

[0036] The advantages achieved by this invention compared to the prior art, due to the adoption of the above-described technical solution, are as follows:

[0037] (1) This invention achieves offline availability. Investigators can collect data normally in a field survey environment with no network. All operations are written to the local document database. After the network is restored, it will automatically be incrementally synchronized without losing any data.

[0038] (2) This invention achieves efficient data transmission. Through a three-layer synchronization hierarchy and a server-side filtering function, the mobile terminal only synchronizes data related to the current user, the current company, and the current project, which greatly reduces the amount of network transmission.

[0039] (3) The present invention achieves fast data access. The three-level caching system reduces the access latency of frequently used data from hundreds of milliseconds of network requests to microseconds of memory reads, significantly improving the response speed of user operations in field survey environments.

[0040] (4) The present invention achieves reliable conflict handling. The Last-Write-Wins strategy based on the update timestamp automatically resolves data conflicts generated by multi-person collaboration without manual intervention; the data change listener detects data conflicts in real time to ensure no omissions.

[0041] (5) This invention achieves resource utilization optimization. The project layer adopts a staggered start strategy to avoid network congestion caused by multiple project-level data tables initiating connections at the same time. In addition, the first-level cache adopts the LRU eviction policy for lifecycle management, the second-level cache adopts the TTL expiration policy for lifecycle management, and the third-level cache adopts the on-demand expiration policy for lifecycle management. The LRU eviction policy, TTL expiration policy and on-demand expiration policy automatically manage the cache space to avoid exhaustion of storage resources.

[0042] (6) This invention achieves security assurance. Sensitive data is encrypted using the AES-GCM algorithm when stored locally in the browser. The encryption key is derived from the device hardware unique identifier through the PBKDF2 algorithm and is not hardcoded in the source code. The server-side filtering function implements data permission isolation, and mobile terminals cannot access data without authorization.

[0043] In summary, this invention achieves reliable data collection and efficient synchronization in weak network or even completely offline environments through the coordinated use of hierarchical database architecture, three-level caching system, bidirectional incremental synchronization, and off-peak startup. This invention is applicable to the synchronization of forest resource field survey data. Attached Figure Description

[0044] Figure 1 The diagram shown is an overall flowchart of the forest resource field survey data synchronization method based on an offline priority architecture according to Embodiment 1 of the present invention.

[0045] Figure 2The diagram shown is an illustration of the architecture and data flow of the three synchronization layers—user layer, company layer, and project layer—in Embodiment 1 of this invention.

[0046] Figure 3 The diagram shown is a data flow diagram of the three-level cache in Embodiment 1 of the present invention;

[0047] Figure 4 The flowchart shown is a process for conflict detection and automatic resolution in Embodiment 1 of the present invention. Detailed Implementation

[0048] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Example 1

[0050] like Figure 1 As shown, this embodiment is a method for synchronizing forest resource field survey data based on an offline-first architecture, including the following steps:

[0051] S1. Establish a hierarchical database architecture

[0052] A local document database (PouchDB) is established on the mobile terminal, and a remote document database (CouchDB) is deployed on the server. On the server, the survey data is divided into three synchronization layers according to business relevance: user layer, company layer, and project layer. The design document of the local document database includes local filtering functions, and the design document of the remote document database includes three server-side filtering functions, which correspond one-to-one with the user layer, company layer, and project layer, respectively.

[0053] In this step, the survey data is entered into the survey form, and the system writes the data to the local document database after verifying the data format through the Schema data verification module.

[0054] The three server-side filtering functions are user-level filtering, company-level filtering, and project-level filtering; among them, the user-level filtering function filters data according to user roles, the company-level filtering function filters shared data according to company identifiers, and the project-level filtering function filters project-level data according to project identifiers.

[0055] In this step, the User layer is configured to filter project data by user role. Specifically, it involves receiving documents and request parameters through the user layer filtering function corresponding to the User layer in the remote document database. First, it checks the deletion mark of the document and allows documents with deletion marks to pass directly. For documents without deletion marks, it checks whether the company identifier (companyId) of the document matches the company identifier in the request parameters. Finally, it checks whether the user identifier (uid) in the request parameters matches any one of the following arrays in the document: creator identifier, project manager identifier (managerId), chief engineer identifier (chiefEngineerId), technical manager identifier (techLeaderId), or investigator identifier (invstIds).

[0056] Company Layer: Configured to filter shared data by company identifier. Specifically, it receives documents and request parameters through the company layer filtering function corresponding to the company layer in the remote document database. First, it checks the deletion mark of the document and allows documents with deletion marks to pass directly. For documents without deletion marks, it further checks whether the company identifier of the document matches the company identifier in the request parameters, and only sends the changes of matching documents to the local document database. The shared data includes: factor dictionary, tile image source, and forest tenure attribute information.

[0057] The factor dictionary contains the standard codes and names of the survey factors, such as land type codes (forested land, sparse forest land, shrubland, etc.), forest type codes (timber forest, protection forest, fuelwood forest, etc.), and source code (natural, artificial). The tile image source is the configuration of the tile data source used for map base map display, including satellite imagery, topographic maps, administrative division maps, etc. The forest tenure attribute information is the attribute data related to forest tenure, such as forest land owner, user, ownership type, ownership period, etc.

[0058] Project Layer: Configured to filter project-level data tables by project identifier (p_id). Specifically, it receives documents and request parameters through the project-level filtering function corresponding to the project layer in the remote document database. First, it checks the deletion mark of the documents and allows documents with deletion marks to pass directly. For documents without deletion marks, it further checks whether the project identifier of the document matches the project identifier in the request parameters, and only sends the changes of matching documents to the local document database. The project-level data tables include: zoning sub-compartment data table, forest tenure sub-compartment data table, sample plot data table, photo data table, actual diameter at breast height (DBH) data table, diameter class DBH data table, survey report data table, tree species summary data table, as well as carbon sink-related soil survey data table, shrub survey data table, herb survey data table, litter survey data table, fallen tree survey data table, carbon sink baseline survey data table, and bamboo forest survey data table.

[0059] like Figure 1 As shown, the three server-side filtering functions deployed in the design document of the remote document database are: by_user, which is the user-level filtering function and corresponds to the user layer; by_company, which is the company-level filtering function and corresponds to the company layer; and by_project, which is the project-level filtering function and corresponds to the project layer.

[0060] like Figure 2 The diagram shows the architecture and data flow of the three synchronization layers: user layer, company layer, and project layer. As shown, the user layer includes one project data table, svy_project. The company layer includes three data tables: svy_dict (corresponding to the factor dictionary), svy_imagery (corresponding to the tile image source), and svy_ownership_properties (corresponding to forest tenure attribute information). The project layer includes 15 project-level data tables: svy_planning (corresponding to zoning plot data), svy_ownership (corresponding to forest tenure plot data), svy_samples (corresponding to sample plot data), svy_photos (corresponding to photo data), svy_dbh_actual (corresponding to actual diameter at breast height (DBH) data), svy_dbh_step (corresponding to DBH by diameter step data), svy_report (corresponding to survey report data), svy_tree_summary (corresponding to tree species summary data), svy_soil_survey (corresponding to soil survey data), svy_shrub_survey (corresponding to shrub survey data), svy_herb_survey (corresponding to herb survey data), svy_litter_survey (corresponding to litter survey data), svy_fallen_wood_survey (corresponding to fallen wood survey data), svy_baseline_survey (corresponding to carbon sink baseline survey data), and svy_bamboo_survey (corresponding to bamboo forest survey data).

[0061] S2. Establish a multi-level caching system

[0062] A three-level caching system is built on the mobile terminal. The first level cache L1 is a memory LRU cache, the second level cache L2 is a disk cache of the embedded key-value storage engine (Redb), and the third level cache L3 is the browser's local storage (localStorage). When querying the cache, the first level cache L1 to the third level cache L3 are searched in order. If a match is found, the data is returned and the previous level cache is populated. If all three levels of cache are not found, a network request is initiated to access the remote document database, and the response data is asynchronously populated into each level cache.

[0063] like Figure 3The diagram shows the data flow of a three-level cache. The first-level cache (L1) is a fixed-capacity in-memory LRU (Least Recently Used) cache maintained in the local application backend (Rust process). Its capacity is set to N records, where N is a preset positive integer (default 100 records). It stores frequently accessed HTTP response data and image data, with read / write latency in the microsecond range. It's important to note that the local application backend refers to the background process of the application running on the mobile terminal. N is generally no less than 50 records and no more than 500 records. If N is too small, the cache hit rate will significantly decrease, failing to effectively leverage the acceleration effect of the first-level cache (L1). If N is too large, it will consume too much mobile terminal memory, potentially increasing application memory pressure or even leading to system garbage collection. Adjustments can be made based on different scenarios.

[0064] The second-level cache (L2) uses an embedded key-value storage engine as a disk persistent cache. Data is encoded in MessagePack format and stored as a byte array. Each record is accompanied by a write timestamp for expiration detection. The read / write latency is in the millisecond range.

[0065] Level 3 caching: Uses browser local storage as a lightweight fallback storage. Data is serialized in JSON format. Each record contains three fields: value, timestamp, and expire timestamp. Sensitive data is stored after being encrypted using the AES-GCM algorithm. The encryption key is derived from the device's unique hardware identifier UUID through the PBKDF2 algorithm with 100,000 iterations.

[0066] Sensitive data includes investigators' identity information, whereabouts, sample plot location coordinates, and forest tenure information.

[0067] S3, a bidirectional incremental synchronization engine for implementing hierarchical filtering.

[0068] A local document database replication protocol is used to achieve bidirectional incremental synchronization between the local document database and the remote document database.

[0069] This step specifically includes:

[0070] S31. Push-Pull Separation Strategy: The local document database replication protocol is used to establish independent pull synchronization channels (Pull) and push synchronization channels (Push) for each local document database instance. The push synchronization channel connects the local document database to the remote document database, while the pull synchronization channel connects the remote document database to the local document database. During synchronization, the pull synchronization channel uses three server-side filtering functions from the remote document database to filter data according to the corresponding synchronization level. The remote document database executes the server-side filtering functions line by line on the changesfeed, sending only matching document changes to the local document database. During synchronization, the push synchronization channel uses local filtering functions to filter and skip data fields marked as "localOnly" and exceeding the data length threshold (1024 bytes) by project identifier, sending only matching document changes to the remote document database. Fields marked as "localOnly" are used to store large amounts of data such as raw image data.

[0071] In this step, on the pull synchronization channel, the server-side filtering function is executed one by one on the change stream through the remote document database, and only the matching document changes are sent to the mobile terminal, reducing network transmission volume and adapting to the weak network environment in forest areas; deleted documents (tombstones) are directly allowed to pass through, ensuring that the deletion operation can be synchronized in real time. Deleted documents refer to those that only contain the identifier and version number, and the business fields have been lost; data permission isolation is implemented, and the mobile terminal cannot obtain unauthorized data;

[0072] S32. Resume interrupted download mechanism: After the pull synchronization channel is established, a checkpoint document is maintained in the local document database to record the change sequence number (seq) reached during the last synchronization. Each time synchronization occurs, the pull starts from the changes after that sequence number. If the checkpoint document does not exist during the first synchronization, the full pull starts from the beginning of the change sequence.

[0073] S33, Long Connection Real-Time Synchronization Mode: Both the push synchronization channel and the pull synchronization channel use long polling mode to maintain a persistent connection. A heartbeat interval (default 5000 milliseconds) is configured to prevent intermediate network devices from disconnecting idle connections, and an automatic retry mechanism is enabled to automatically resume synchronization after a network disconnection. There are upper limits on the number of documents processed per batch and the number of parallel batches (default 50 documents per batch, maximum 5 parallel batches).

[0074] S4. Implement hierarchical synchronous start / stop management

[0075] The project-level synchronization adopts a staggered start strategy and ensures the correctness of project switching through idempotency checks and race condition protection. User-level and company-level data synchronization starts immediately after the user logs into the application.

[0076] In this step, the project-level synchronization adopts a staggered startup strategy, specifically as follows: The project layer contains 15 project-level data tables. Each project-level data table in the project layer is registered in the synchronization registry (projectSyncRegistry). To avoid network congestion (connection storm) caused by a large number of simultaneous connections, the synchronization channels of each project-level data table are started sequentially at a preset time interval (default 150 milliseconds) when synchronization starts. It takes about 2.25 seconds to start all of them. Before each startup, it checks whether the currently active project identifier is still the target project identifier. If not, the startup is canceled.

[0077] Among them, the currently active project identifier refers to the unique identifier information corresponding to the project that the user is currently operating on or the system is currently processing in the scenario of multiple projects operating in parallel and synchronously, which is used to distinguish the project from other inactive projects.

[0078] The project layer also employs an automatic recovery mechanism: when the application starts, it reads the last active project identifier from the third-level cache L3 browser local storage and sets it as the current active project identifier. When subsequent data models are initialized and registered, if the identifier has been set, background synchronization will be automatically started according to the staggered peak strategy.

[0079] Idempotency check: When switching projects, a "stop old, start new" strategy is adopted. First, the synchronization channels of all current project-level data tables are canceled, and then the synchronization of the new project is started according to the off-peak strategy. Duplicate start requests for the same project are automatically skipped to avoid duplicate creation of synchronization channels.

[0080] S5, Conflict Detection and Automatic Resolution Module

[0081] During bidirectional incremental synchronization and when local document database data changes, data conflicts are detected in real time, and a Last-Write-Wins strategy based on update timestamps is used to automatically resolve conflicts.

[0082] like Figure 4 As shown, the specific process of this step is as follows:

[0083] S51. Real-time conflict detection: In the data change listener (onChange), check the conflict flag (_conflicts field) of each change document, and trigger the resolution process immediately when a conflict is found;

[0084] S52. The Last-Write-Wins policy based on the update timestamp automatically resolves conflicts:

[0085] Retrieve the current winning version and all conflicting versions of the conflicting document. Sort all versions in descending order by update timestamp (updatedAt). Select the version with the latest timestamp as the winning version. Mark the remaining versions as deleted and submit them to the local document database in batch operation. If the winning version is not the currently active version, overwrite the currently active version with the winning version.

[0086] S53, Full Conflict Scan: Provides full database conflict scanning capability, traversing all changed documents to check and resolve conflicts one by one, used for system initialization or periodic maintenance.

[0087] S6. Implement automatic TTL cleanup and graded expiration mechanism.

[0088] S61. Tiered Expiration Strategy:

[0089] Level 1 cache (L1): Uses an LRU eviction policy for lifecycle management, evicting the least accessed entry when the capacity is full;

[0090] Level 2 cache (L2): Lifecycle management is performed using a TTL expiration strategy. The TTL expiration strategy includes configuring independent TTL durations (15 days by default) for HTTP response data and image data in the L2 cache. Expired data is removed by background cleanup tasks.

[0091] The third-level cache (L3) uses an on-demand expiration strategy for lifecycle management. Each record carries an optional expiration timestamp. When reading, it checks whether the record has expired. If it has expired, it is automatically deleted and the default value is returned.

[0092] S62. Active cleanup mechanism: The first-level cache L1, the second-level cache L2, and the third-level cache L3 all adopt an active cleanup mechanism, providing a timed cleanup interface to clean up expired data in each level of cache in batches according to the maximum lifespan (maxAgeMs) and return the number of entries cleaned up.

[0093] S7. Reset Synchronization Operation (resetSync): Stops the synchronization channel of the current local document database instance, destroys the local document database instance, recreates an empty local document database, and then uses the server-side filtering function to fetch all the data of the current project from the remote document database at once. After the fetch is complete, the real-time synchronization mode is restored.

[0094] Example 2

[0095] This embodiment presents three specific application scenarios for the implementation of a forest resource field survey data synchronization method based on an offline-first architecture, using the method described in Embodiment 1.

[0096] I. Multi-person collaborative field data collection and synchronization for Class II forest survey projects

[0097] Application scenario: A county-level forestry bureau is conducting a forest resource planning and design survey (Class II survey). The survey area covers approximately 500,000 mu of forest land. The survey team consists of 1 project leader, 1 chief engineer, 1 technical leader, and 8 surveyors. They are using 11 mobile terminals (Android tablets) to conduct the field survey.

[0098] Implementation steps:

[0099] (1) System initialization and hierarchical synchronous startup

[0100] After the project manager logs into the system in an office environment with network access, the system will automatically perform the following operations:

[0101] Parse the user identifier (uid=1001) and company identifier (companyId=5) from the server-side JWT token;

[0102] Start user-level synchronization immediately: svy_project uses the by_user user-level filtering function, with {companyId:'5',uid:'1001'} as parameters, to pull all projects for which the user is the project manager (managerId);

[0103] Simultaneously, company-level synchronization is initiated: the three tables svy_dict, svy_imagery, and svy_ownership_properties use the by_company company-level filtering function with {companyId:'5'} as the parameter to retrieve the company's shared factor dictionary, tile image source, and forest tenure attribute information;

[0104] The system reads the current_project_id (the identifier of the last active project) from the third-level cache L3 browser local storage and automatically resumes project-level background synchronization.

[0105] After the project leader selects the current survey project identifier p_id='proj_2026_001', the system calls the startProjectSync function to perform the following operations:

[0106] Idempotency check: Skip if the project is already synchronized;

[0107] Staggered startup of 15 project-level data tables for synchronization: each at 150ms interval, the first one (svy_planning) starts at 0ms, and the 15th one (svy_bamboo_survey) starts at 2100ms;

[0108] Each project-level data table establishes a push synchronization channel and a pull synchronization channel. The pull synchronization channel uses the server-side by_project project-level filtering function to pull only the document with p_id='proj_2026_001'.

[0109] (2) Offline field data collection

[0110] Eight investigators entered the forest area with tablets and then switched to an offline environment. The investigators opened the small-plot factor input form, and the system validated the data format using the schema data verification module before writing it to the local document database. Taking the small-plot survey as an example:

[0111] Investigator A entered factors such as land type (forested land), forest type (timber forest), origin (natural), dominant tree species (Masson pine), age group (middle-aged forest), and canopy closure (0.65) into small plot 001;

[0112] The data is first validated by the Schema data validation module (field type, required fields, enumeration value checks, range checks). Once the validation is successful, metadata (uid, companyId, p_id, createdAt, updatedAt) is automatically appended and written to the local svy_planning data table. The local document database immediately generates a version number (_rev) for the document to ensure that subsequent conflict detection is traceable.

[0113] Meanwhile, the configuration data, such as the factor dictionary and tree type code table, that investigator A consulted were obtained from the three-level cache system:

[0114] First, query the first-level cache L1 (capacity 100 entries). If a match is found, return the result in microseconds.

[0115] If a cache miss occurs, the second-level cache L2 is queried to check if the data timestamp is within the TTL (15 days). If valid, the data is returned and the first-level cache L1 is populated.

[0116] If the cache still misses, query the third-level cache L3, check the expiration timestamp, and if it is valid, return the result and populate the first-level cache L1 and the second-level cache L2.

[0117] (3) Automatic incremental synchronization after network recovery

[0118] The network was restored after Investigator A returned to base or entered a signal coverage area. The bidirectional incremental synchronization engine executed automatically:

[0119] The local document database's automatic reconnection mechanism detects network availability and automatically restores the push synchronization channel and pull synchronization channel connection;

[0120] Retrieve sync channel: Obtain the change sequence number (seq) of the last sync arrival from the checkpoint document, and request _changes?since=<last_seq> &filter=sync / by_project&p_id=proj_2026_001, only retrieves newly added and changed documents;

[0121] Push synchronization channel: Use a local filtering function to check whether the project identifier p_id of each document to be pushed is the current project, and push it to the server after it passes the check;

[0122] Each batch processes 50 documents, with a maximum of 5 parallel batches, to avoid a large number of concurrent requests in a weak network environment.

[0123] (4) Automatic resolution of conflicts involving multiple people

[0124] Investigator A and Investigator B modified factors for the same sub-class 001 at different times (e.g., Investigator A modified the canopy closure to 0.70, and Investigator B modified the water volume to 85.3 m³ / ha), resulting in a data conflict. The system automatically handles this:

[0125] The data change listener detected that the document's _conflicts field was not empty, triggering the conflict resolution process;

[0126] Obtain Investigator A's version (updatedAt='2026-03-15T14:30:00Z') and Investigator B's version (updatedAt='2026-03-15T16:45:00Z'); according to the Last-Write-Wins policy based on the update timestamp, Investigator B's version was updated later and was selected as the winning version;

[0127] Investigator A's version was marked for deletion and submitted in bulk, while Investigator B's version was retained as the currently active version.

[0128] Implementation results:

[0129] Eight investigators collected data from approximately 200 small groups throughout the day in an offline environment, with no data loss.

[0130] Upon returning to their base, each mobile terminal completed all incremental synchronization within approximately 3-5 minutes.

[0131] The system automatically resolved 12 conflicts without human intervention;

[0132] By using project-level filtering, each mobile device only synchronizes the data for the current project (approximately 2,000 documents), instead of the entire database (approximately 50,000 documents), reducing the amount of data synchronized by 96%.

[0133] II. Data Acquisition and Synchronization of Multiple Carbon Banks in Carbon Sequestration Projects

[0134] Application Scenario: A forestry carbon sink development company is conducting a CCER forestry carbon sink project and needs to conduct carbon storage surveys for both the baseline scenario and the project scenario. The survey involves seven types of survey forms for five major carbon pools (aboveground biomass, belowground biomass, deadwood, litter, and soil). The survey area is located in a high-altitude mountainous region with extremely poor network coverage (only 2G signal is available at the foot of the mountain).

[0135] Implementation steps:

[0136] (1) Definition and synchronous registration of multi-table data models

[0137] The system defines data models and synchronization parameters for seven types of carbon sequestration survey forms using data model declaration functions:

[0138] Shrub survey: svy_shrub_survey(syncMode='project');

[0139] Herb survey: svy_herb_survey(syncMode='project');

[0140] Litter survey: svy_litter_survey(syncMode='project');

[0141] Fallen wood survey: svy_fallen_wood_survey(syncMode='project');

[0142] Soil survey: svy_soil_survey(syncMode='project');

[0143] Carbon sink baseline survey: svy_baseline_survey(syncMode='project');

[0144] Bamboo forest survey: svy_bamboo_survey(syncMode='project').

[0145] Each model definition contains validation rules from the Schema data validation module (field type, required fields, enumeration value checks, range checks). The model is automatically registered in the synchronization registry when it is defined, and it automatically participates in project synchronization when the startProjectSync function is called.

[0146] (2) Intelligent processing of localOnly fields stored only locally

[0147] During carbon sequestration surveys, investigators are required to take photographs on-site as supporting evidence (panoramic views of the sample plot, close-up photos of sample trees, soil profiles, etc.). In the photo data model, fields for locally stored only are marked as `localOnly`.

[0148] constPhotoDB=defineSyncModel('photos',{localData:{type:'string',localOnly:true}, / / Base64 encoded photo data URL:{type:'string'}, / / Server URL after upload p_id:{type:'string',required:true},sampleId:{type:'string'} / / Associated sample plot ID});

[0149] When synchronizing via the push synchronization channel, if the local filtering function detects that the localData field exceeds 1024 bytes, it will skip pushing the original photo data for that document and only synchronize metadata (url, associated plot ID, etc.). The original photo data is uploaded via a separate file upload channel when network conditions are good.

[0150] (3) Caching strategies in weak network environments

[0151] The bandwidth of the 2G signal area at the foot of the mountain is only tens of Kbps. The system adopts the following caching strategy to optimize this:

[0152] The carbon sink calculation formula library (biomass equation, carbon conversion coefficient, etc.) is requested through the useCache interface, and the data is cached in the second-level cache L2 with a TTL of 15 days.

[0153] After the initial request, subsequent calculations can still be performed on-site by retrieving formula parameters from the second-level cache (L2) even when completely offline in mountainous areas.

[0154] Factor dictionary data is cached in the third-level cache (L3), with a 30-day expiration time set: `setLocal('carbon_dict', dictData, 30...` twenty four 60 60 1000).

[0155] (4) Tiered expiration and storage management

[0156] Lifecycle management of cached data during extended fieldwork periods (typically 2-4 weeks):

[0157] The first-level cache (L1) is automatically evicted using the LRU (Least Recently Used) policy, and its 100-entry capacity ensures that high-frequency data remains resident.

[0158] HTTP response data and image data older than 15 days in the second-level cache (L2) are cleaned up by the cache_clean_expired command.

[0159] When each piece of data is read in the L3 cache, the expiration timestamp field is automatically checked. Expired data is immediately deleted and a network request is triggered to retrieve it again.

[0160] When the system starts, the cleanExpiredCache function is called to clean up expired data in the second-level cache (L2) in batches, and the number of data cleaned up is output to the console for operation and maintenance monitoring.

[0161] Implementation results:

[0162] Three investigators worked continuously for five days in high-altitude mountainous areas, collecting a total of about 1,200 records of data from seven carbon pools in 48 sample plots, all of which were securely stored in the local document database PouchDB.

[0163] When descending the mountain each day and passing through areas with 2G or higher network signals, the system automatically performs bidirectional incremental synchronization. Each synchronization takes about 8-15 minutes (limited by 2G bandwidth). Only newly added data for the day is transmitted through incremental transmission.

[0164] The localOnly field for local storage avoids syncing approximately 300MB of raw photo data over a 2G network; photos are uploaded separately via WiFi.

[0165] The three-level cache enables fast access to carbon sink calculation formulas and factor dictionary data even when completely offline (the L2 cache hit latency is about 2ms).

[0166] III. Multi-project switching and data isolation in cross-regional survey projects

[0167] Application Scenario: A provincial forestry survey and design institute is simultaneously undertaking forest resource survey projects in three counties (Project C, Project D, and Project E). Surveyors need to travel to different counties at different times. The system needs to allow for rapid switching between projects on the same mobile terminal, while ensuring strict data isolation between different projects.

[0168] Implementation steps:

[0169] (1) Idempotent synchronization management of project switching

[0170] The investigator is currently synchronizing project C (p_id='proj_C') and needs to switch to project D (p_id='proj_D'). The system call startProjectSync('proj_D') executes the following process:

[0171] Idempotency check: if activeProjectId('proj_C') ≠ 'proj_D', perform a switch;

[0172] Stop old synchronization: Traverse all 15 project-level data table entries in the synchronization registry projectSyncRegistry, call the stopSync() function for each one, cancel 30 long connections for push synchronization channels and pull synchronization channels, and release network resources;

[0173] Update global state: activeProjectId='proj_D', and write localStorage.current_project_id='proj_D';

[0174] Staggered startup of new synchronization: 15 project-level data tables are started sequentially at 150ms intervals. Before each startup, activeProjectId==='proj_D' is checked again (race condition protection to prevent users from switching to project E during the 2.25-second wait).

[0175] The new pull synchronization channel filter parameter is {p_id:'proj_D'}, which only pulls documents for project D.

[0176] (2) Data isolation and query filtering for multiple projects

[0177] The local document database may contain data for both project C and project D simultaneously (because local document data is not deleted when switching projects). The system ensures proper isolation of query results through the following mechanisms:

[0178] All query operations (query, getAll, paginate, aggregate) automatically append the filter condition {companyId:companyId} to ensure that only data from the current company is returned;

[0179] Project-level data is distinguished by the p_id field, and the {p_id:currentProjectId} selector is explicitly added when querying on the front-end page;

[0180] The server-side pull synchronization channel uses the by_project project-level filtering function to ensure that only data from the target project is pulled during synchronization, and new data from other projects is not pulled to the local machine.

[0181] (3) Concurrent synchronization of the same project on multiple terminals

[0182] Three investigators are simultaneously using three mobile devices to work in different areas of Project D. The push synchronization channels of these three mobile devices may simultaneously write to the same database on the server. The Multi-Version Concurrency Control (MVCC) mechanism of the remote document database ensures that:

[0183] Data written to different classes from different terminals will not cause conflicts (because the _id is different);

[0184] If two terminals modify the same class data (for example, investigator F and investigator G modify class 015 one after the other), the remote document database will automatically generate a conflict version tree;

[0185] After each terminal pulls a conflicting version through its synchronization channel, the data change listener triggers a Last-Write-Wins strategy based on the update timestamp: comparing the update timestamp updatedAt and retaining the version with the later update time.

[0186] (4) Reset synchronization and full restore

[0187] When mobile terminal data becomes abnormal (such as storage corruption), investigators can trigger a reset synchronization operation (resetSync):

[0188] Stop the synchronization channel of the current local document database instance;

[0189] Destroy the local document database instance and remove it from the database instance cache (dbInstances);

[0190] Recreate an empty local document database;

[0191] Retrieve all data of the current project from the remote document database in one go (using server-side filtering functions, non-live mode).

[0192] After the fetch is complete, resume real-time synchronization in live mode.

[0193] Implementation results:

[0194] The investigators switched between the three projects a total of more than 20 times, with each switch taking about 2-3 seconds (15 project-level data tables were started in staggered shifts), and no data was lost during the switching process;

[0195] During the concurrent synchronization of three mobile terminals in the same project, the system automatically resolved five conflicts, and the latest version was correctly selected in all cases based on the Last-Write-Wins strategy of updating timestamps.

[0196] Project-level filtering ensures that only the current project data is synchronized on the mobile terminal. When switching to project E, data from project C and project D will not be pulled, saving approximately 80% of the amount of synchronized data and storage space per terminal.

[0197] After a mobile device shuts down abnormally due to a depleted battery, the system automatically restores the last active item identifier from the browser's localStorage after restarting and automatically starts background synchronization without manual operation.

[0198] It should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can still modify the technical solutions described in the above embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A forest resource field investigation data synchronization method based on an offline priority architecture, characterized in that, Includes the following steps: S1. Establish a local document database on the mobile terminal and deploy a remote document database on the server. On the server side, the survey data is divided into three synchronous layers according to business relevance: user layer, company layer, and project layer. The design document of the local document database deploys local filtering functions, and the design document of the remote document database deploys three server-side filtering functions, which correspond one-to-one with the user layer, company layer, and project layer, respectively. S2. Construct a three-level caching system on the mobile terminal. The first level cache is a memory LRU cache, the second level cache is a disk cache of the embedded key-value storage engine, and the third level cache is the browser's local storage. When querying the cache, the first level cache is searched in order to the third level cache. If a match is found, the cache is returned. If all three levels of cache are not found, a network request is initiated to access the remote document database, and the response data is asynchronously populated back into each level of cache. In step S2, the first-level cache is a fixed-capacity memory LRU cache maintained in the local application backend, with a capacity of N records, where N is a preset positive integer, used to store HTTP response data and image data, and the read / write latency is in the microsecond range; In the second-level cache, data is encoded in MessagePack format and stored as a byte array. Each record is accompanied by a write timestamp, and the read / write latency is in the millisecond range. In the third-level cache, the data is serialized in JSON format. Each record contains three fields: value, timestamp, and expiration timestamp. Sensitive data is encrypted using the AES-GCM algorithm before storage. The encryption key is derived from the device hardware's unique identifier through the PBKDF2 algorithm. S3. The replication protocol of the local document database is adopted to realize bidirectional incremental synchronization between the local document database and the remote document database. Independent pull synchronization channels and push synchronization channels are established for the local document database instances corresponding to each synchronization level. The pull synchronization channel uses three server-side filtering functions of the remote document database during synchronization to filter data according to the corresponding synchronization level and only sends the matching document changes to the local document database. The push synchronization channel uses a local filtering function during synchronization to filter and skip data fields marked as locally stored only and whose data length exceeds a threshold, based on project identifiers, and only sends matching document changes to the remote document database; S4. The project layer synchronization adopts a staggered start strategy and ensures the correctness of project switching through idempotency checks and race condition protection. User layer and company layer data synchronization starts immediately after the user logs into the application. S5. Real-time detection of data conflicts, and automatic resolution of conflicts using the Last-Write-Wins strategy based on update timestamps.

2. The forest resource field survey data synchronization method based on an offline-first architecture according to claim 1, characterized in that, In step S1, the user layer is configured to filter project data by user role. Specifically, this includes: receiving documents and request parameters through the server-side filtering function corresponding to the user layer in the remote document database; firstly, checking the deletion mark of the document and allowing documents with deletion marks to pass directly; for documents without deletion marks, further checking whether the company identifier of the document matches the company identifier in the request parameters; finally, checking whether the user identifier in the request parameters matches any one of the following arrays in the document: creator identifier, project manager identifier, chief engineer identifier, technical manager identifier, or investigator identifier. The company layer is configured to filter shared data by company identifier. The shared data includes: factor dictionary, tile image source, and forest tenure attribute information. The project layer is configured to filter project-level data tables by project identifier. The project-level data tables include: zoning sub-compartment data table, forest tenure sub-compartment data table, sample plot data table, photo data table, actual diameter at breast height (DBH) data table, diameter class DBH data table, survey report data table, tree species summary data table, as well as carbon sink-related soil survey data table, shrub survey data table, herb survey data table, litter survey data table, fallen tree survey data table, carbon sink baseline survey data table, and bamboo forest survey data table.

3. The method for synchronizing forest resource field survey data based on an offline-first architecture according to claim 1, characterized in that, In step S3, the bidirectional incremental synchronization further includes: After the pull synchronization channel is established, a checkpoint document is maintained in the local document database to record the change sequence number reached during the last synchronization. Each time synchronization occurs, the pull starts from the changes after that sequence number. If the checkpoint document does not exist during the first synchronization, the full pull starts from the beginning of the change sequence. Both the push synchronization channel and the pull synchronization channel use a long polling mode to maintain a persistent connection, configure a heartbeat interval to prevent intermediate network devices from disconnecting idle connections, and enable an automatic reconnection mechanism.

4. The method for synchronizing forest resource field survey data based on an offline-first architecture according to claim 2, characterized in that, In step S4, the staggered startup strategy for project-level synchronization specifically includes: registering each project-level data table in the project layer to the synchronization registry; starting synchronization by sequentially starting the synchronization channel of each data table in the project-level data table at a preset time interval; and checking whether the currently active project identifier is still the target project identifier before each startup, and canceling the startup if it is not.

5. The method for synchronizing forest resource field survey data based on an offline-first architecture according to claim 4, characterized in that, In step S4, the project layer also adopts an automatic recovery mechanism: when the application starts, it reads the last active project identifier from the browser's local storage in the third-level cache and sets it as the current active project identifier. When each data model is initialized and registered, it detects that the identifier has been set and automatically starts background synchronization according to the staggered peak strategy.

6. The method for synchronizing forest resource field survey data based on an offline-first architecture according to claim 1, characterized in that, The specific process of step S5 is as follows: S51. Check the conflict markers for each change document in the data change listener; S52. Obtain the current winning version and all conflicting versions of the conflicting document. Sort all versions in descending order of update timestamp. Select the version with the latest timestamp as the winning version. Mark the remaining versions as deleted and submit them to the local document database through batch operation. If the winning version is not the current active version, overwrite the current active version with the winning version.

7. The method for synchronizing forest resource field survey data based on an offline-first architecture according to claim 1, characterized in that, Also includes: The first-level cache uses an LRU eviction policy for lifecycle management; The second-level cache uses a TTL expiration strategy for lifecycle management; the TTL expiration strategy includes configuring independent TTL durations for HTTP response data and image data in the second-level cache; The third-level cache uses an on-demand expiration strategy for lifecycle management.

8. The method for synchronizing forest resource field survey data based on an offline-first architecture according to claim 7, characterized in that, The first-level cache, second-level cache, and third-level cache all employ an active cleanup mechanism: providing a timed cleanup interface to batch clean up expired data in each level of cache according to the maximum lifespan, and returning the number of entries cleaned up.

9. The method for synchronizing forest resource field survey data based on an offline-first architecture according to claim 1, characterized in that, It also includes resetting the synchronization operation: stopping the synchronization channel of the current local document database instance, destroying the local document database instance, recreating an empty local document database, and then using the server-side filtering function to fetch all the data of the current project from the remote document database at once. After the fetch is completed, the real-time synchronization mode is restored.

Images