A data offline caching method for mines without network
By combining real-time network monitoring, SQLite database, and edge computing nodes, the problem of data caching in mines without network access was solved, achieving efficient and simplified data caching and synchronization, and ensuring data consistency and system stability.
Patent Information
- Application Number
- CN202411297964.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-18
AI Technical Summary
Existing offline caching technologies are difficult to effectively handle dynamic data in mining environments, are easy to use, and have high development costs, failing to meet the data caching needs of mining scenarios without network access.
This paper proposes an offline data caching method for mines without network access by employing real-time network detection, SQLite database, edge computing nodes, and automated deployment and rollback strategies. The method uses the cordova-plugin-network-information plugin to obtain network metrics, uses an SQLite database to store data, deploys edge computing nodes to handle data synchronization, and configures automated deployment and rollback strategies to ensure data consistency.
It enables large-capacity data caching in harsh network environments, simplifies development, ensures data consistency and system stability, provides an automatic data synchronization mechanism, and reduces data corruption during network anomalies.
Smart Images

Figure CN118869480B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing and data transmission, and mainly to a method for offline data caching in a network-free mining scenario. Background Art
[0002] The application of offline caching in the mining industry is mainly focused on improving work efficiency, ensuring safety, and improving the working environment. Underground mines usually have poor network coverage. Workers can use offline caching technology to collect geological data, equipment status and other information, and upload the data when they return to the ground or to areas with good network coverage. However, the implementation of offline data caching in a mining environment faces a series of technical challenges, especially considering the particularity of the mining environment, such as remote geographical location, unstable or missing network, scattered equipment, and huge data volume. In most cases, the network will be disconnected or weak for a long time, which brings huge challenges to data caching and synchronization.
[0003] The existing offline caching technologies include: 1) using the cache mechanism of the web browser to achieve offline caching. The browser cache mechanism is mainly for static resources (such as HTML, CSS, JavaScript files, etc.), and has limited support for dynamic data, which cannot meet the needs of applications that need to update content frequently. 2) Using third-party offline cache libraries such as IndexedDB and Service Worker to achieve offline caching. The use of these third-party libraries is relatively complex and requires developers to have certain technical knowledge. As the complexity of the application increases, the cost of maintaining these libraries will also increase accordingly. 3) Using local databases (such as SQLite, LevelDB, etc.) to store offline data: Although the local database solution is highly flexible, it requires developers to design a data synchronization mechanism to ensure the consistency of offline data with online data. This increases the complexity and difficulty of development. In summary, although the existing offline caching technologies can meet some needs, they still have certain limitations in processing dynamic data, ease of use, and development costs. These limitations prompt us to find more efficient and flexible offline caching solutions. Summary of the invention
[0004] In view of the technical problems existing in the above-mentioned prior art, the present application proposes an offline caching method for offline scenarios.
[0005] According to one aspect of the present invention, a method for offline data caching in a mine without network is proposed, comprising:
[0006] S1. Real-time detection of the network environment in the mine, identification of the types of data that need to be cached in the mine, including sensor data, equipment status information and production data, initialization and classification storage of the collected data, wherein the real-time detection adopts a network status cycle detection method, uses the network strategy pre-configured in the background to dynamically generate a network configuration process algorithm, and obtains device network indicators regularly through the cordova-plugin-network-information plug-in, obtains the conditional parameters in the network configuration process algorithm, and determines the execution strategy list for the acquired data in the current request;
[0007] S2. Design a cache system architecture, obtain the network index of the current request, and determine the actual upload behavior of the current request in combination with the execution strategy list. If the actual upload behavior is offline caching, save the data in the current request to the local database. If the actual upload behavior is online upload, request the background interface to upload the data. If the actual upload behavior is delayed upload, store the data in the current request in the running memory and delay the upload through a timer.
[0008] S3, synchronize local and server data, deploy edge computing nodes at the mine site, use the edge computing nodes to deploy task schedulers to handle data synchronization tasks, and use data synchronization strategies to automatically synchronize local and server data, the data synchronization strategies include: incremental synchronization, handling data conflicts and ensuring data consistency, wherein the incremental synchronization implementation steps include: when the application is started or triggered regularly, detecting the update status of the server data, obtaining the last update timestamp of the server data, comparing the update timestamps of local data and the server data, obtaining the changed data and synchronizing it to the local, wherein the server sends data update notifications by push or polling;
[0009] S4. When data caching fails due to the harsh network environment of the mine, configure automatic deployment and rollback strategies to ensure that the local data is synchronized with the server data. At the same time, record the operation logs before and after the rollback, analyze the operation logs, adjust the cache strategy and update the cache rules. In particular, grayscale release is used to roll back the traffic to the previous version of data processing.
[0010] Furthermore, the specific network indicators in the network configuration process algorithm include: connection type, signal strength, uplink and downlink speeds, data transmission volume, packet loss rate, analysis cycle, data packet size and actual upload behavior, among which the actual upload behavior includes: offline caching, online uploading, delayed uploading and refusal to upload.
[0011] The analysis period in the network indicator determines the timing period of the network status cyclic detection.
[0012] Before the app sends a request, the actual upload behavior of the current request is determined in the request preprocessing process based on the requested data packet size, message type and other parameters combined with the execution strategy list. If the judgment result is offline, the data is saved in the local database. If it is determined to be uploaded online, the data is requested to the backend interface. If it is determined to be delayed upload, the data packet will be stored in the mobile phone's running memory and the upload will be continuously attempted through the timer.
[0013] Furthermore, the local database is specifically: using SQLite database as the local database to store user information, wherein a data table structure including user ID, business field and last updated timestamp is configured, and data caching in offline state is realized through encapsulated database operation class.
[0014] As a lightweight local database solution, the SQLite database is a serverless database engine when used as a local database, which means it does not require a separate server process or operating system to run. In addition, SQLite has a small memory footprint and disk footprint, making it very suitable for resource-constrained devices. SQLite also supports multiple operating systems, including Windows, macOS, Linux, and mobile operating systems (such as Android and iOS). SQLite supports ACID (atomicity, consistency, isolation, persistence) transactions to ensure the integrity and consistency of data operations.
[0015] Furthermore, the data cache in the offline state is realized by the encapsulated database operation class, which is specifically realized by the offline data module interface. The offline data module interface includes a new data encapsulation class, a query data encapsulation class, a delete data encapsulation class and a download data encapsulation class.
[0016] The offline data module interface allows developers to operate on data without a network connection to ensure that the application can still run normally in an offline state. Among them, the encapsulation method can quickly realize the operation of offline data.
[0017] The offline data module interface also encapsulates other operation classes, including but not limited to: initialization module initialize(), including creating or opening database files, setting default configurations, etc., with no parameters and no return values; listening to data changes addListener(callback:Function), registering a callback function, which is called when the data in the offline data module changes.
[0018] Furthermore, the step of implementing the processing of data conflicts includes: when data conflicts occur, obtaining the data modification time or version number, and the client and the server determine which data in the conflicting data is valid data through the data modification time or the version number.
[0019] When using timestamps or version numbers to handle data conflicts, optimistic locking or pessimistic locking can be used to manage data. The optimistic lock is: do not lock the data before updating it, but detect conflicts through timestamps or version numbers. Add a version number field or timestamp field to the data table. When the client performs an update operation, check whether the local version number is consistent with the server version number. If the version numbers are consistent, update the data and increment the version number; if they are inconsistent, refuse to update.
[0020] The pessimistic lock is: locking data before updating it to ensure that the data will not be changed by other clients during the update process, adding a lock field in the data table, and obtaining the lock when the client needs to update the data, and releasing the lock after the update is completed. If other clients try to update the locked data, they wait or refuse to update.
[0021] Both optimistic locking and pessimistic locking can ensure that when data conflicts occur, new conflicts will not arise again during the conflict resolution process, causing data management chaos, thereby improving data management efficiency.
[0022] Furthermore, the steps for ensuring data consistency include: using transactions in local and server-side data operations to ensure data atomicity and consistency, and using checksum or hash verification mechanisms during data transmission and storage to ensure that data is not tampered with during transmission.
[0023] The atomicity of data ensures that all operations in a transaction are executed as a whole. If any operation in a transaction fails, the entire transaction is rolled back, ensuring that the integrity of the data is not compromised. After the transaction is completed, the database must be in a consistent state. This means that the data must conform to predefined constraints and rules before and after the transaction is executed.
[0024] Furthermore, the offline, caching and delay in the actual uploading behavior are caused by the network connection status, and the method of checking the network connection status includes: checking using HTTP request, checking using socket library, checking using ping command and checking using network status listener.
[0025] Use HTTP request check: Use an HTTP library (such as the Requests library) to send a GET request to a known URL (such as Google). If the status code 200 is returned, the network connection is normal; otherwise, a connection timeout or error may occur.
[0026] Using the socket library: Create a TCP connection to a known IP address and port (such as Google's IP address and port). If the connection is successful, the network connection is normal; if the connection times out or fails, there may be a network problem.
[0027] Using the ping command: In some operating systems, you can use the ping command to detect the network connection status by sending ICMP Echo requests and waiting for responses to determine whether the network is reachable.
[0028] Use the network status listener: monitor the status changes of the network interface through the system or third-party library to determine the network connection status.
[0029] Furthermore, a version control system is used to manage software code to ensure that each update has a version record, CI / CD tools are used to achieve automated building and testing, and Docker containers are used to encapsulate applications and Kubernetes container orchestration tools to achieve cluster automated management. When an update fails, it can be quickly rolled back to the previous stable version to reduce downtime. The automated deployment data model is specifically represented as follows:
[0030]
[0031] Among them, p(D) represents the average metric value of the deployment strategy model, p( ) represents the deployment metric value of the i-th deployment unit, and N represents the number of deployment units;
[0032] The mathematical model of the rollback strategy is specifically expressed as follows:
[0033]
[0034] Among them, R(S) represents the metric value of the entire model, M represents the number of rollbacks, and R( ) represents the metric value of the jth rollback unit, Represents the jth rollback unit.
[0035] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, the above method is implemented.
[0036] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:
[0037] 1) Use the database to implement large-capacity data caching, provide a unified data reading and writing interface, simplify the development work of developers, implement an automatic data synchronization mechanism, and ensure that the UI display is consistent with the database data.
[0038] 2) Achieve data consistency in the event of a severe network disconnection or weak network;
[0039] 3) In the event of network anomalies or update failures during data version update, the system data version can be rolled back to the previous version in a timely manner to avoid confusion caused by halfway data update. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.
[0041] Figure 1 A schematic diagram of a framework of a data offline caching method in a mine without network scenario according to an embodiment of the present invention is shown;
[0042] Figure 2 A schematic diagram of a specific process of offline data caching in a network-free mine scenario according to a specific embodiment of the present invention is shown. DETAILED DESCRIPTION
[0043] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0044] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0045] Figure 1 FIG. 4 shows a schematic diagram of a data offline caching method framework for a mine without network according to an embodiment of the present invention. Figure 1 As shown:
[0046] S1. Real-time detection of the network environment in the mine, identification of the types of data that need to be cached in the mine, including sensor data, equipment status information and production data, initialization and classification storage of the collected data, wherein the real-time detection adopts a network status cycle detection method, uses the network strategy pre-configured in the background to dynamically generate a network configuration process algorithm, and obtains device network indicators regularly through the cordova-plugin-network-information plug-in, obtains the conditional parameters in the network configuration process algorithm, and determines the execution strategy list for the acquired data in the current request;
[0047] The data of mines basically comes from the data collected by sensors or other hardware devices near mines in different locations. The data types and data processing protocols collected by different sensors or other hardware devices are very different. Therefore, processing different types of data formats and protocols in the mining environment is an important prerequisite for data synchronization.
[0048] Use ETL tools to convert data from different sources into a unified format, such as CSV, JSON, XML, etc., remove invalid or duplicate data, ensure data quality, define a set of standardized data models, ensure that all data conforms to the same set of specifications, create a data dictionary, and record each data type and its field definition for unified management and query.
[0049] Use middleware (such as MQTT, AMQP, CoAP, etc.) to adapt different communication protocols and realize conversion between protocols.
[0050] The specific network indicators in the network configuration process algorithm include: connection type, signal strength, uplink and downlink speeds, data transmission volume, packet loss rate, analysis cycle, data packet size and actual upload behavior, among which the actual upload behavior includes: offline caching, online uploading, delayed uploading and rejected uploading.
[0051] The analysis period in the network indicator determines the timing period of the network status cyclic detection.
[0052] Before the client sends a request, the actual upload behavior of the current request is determined in the request preprocessing process based on the request data packet size, message type and other parameters combined with the execution strategy list. If the judgment result is offline, the data is saved in the local database. If it is determined to be uploaded online, the data is requested to the backend interface. If it is determined to be delayed upload, the data packet will be stored in the mobile phone's running memory and the upload will be continuously attempted through the timer.
[0053] S2. Design a cache system architecture, obtain the network index of the current request, and determine the actual upload behavior of the current request in combination with the execution strategy list. If the actual upload behavior is offline caching, save the data in the current request to the local database. If the actual upload behavior is online upload, request the background interface to upload the data. If the actual upload behavior is delayed upload, store the data in the current request in the running memory and delay the upload through a timer.
[0054] The local database is specifically: using SQLite database as the local database to store user information, in which a data table structure including user ID, business field and last updated timestamp is configured, and data cache in offline state is realized through encapsulated database operation class. In the application, data is loaded from the local database first according to the network status to ensure that basic user functions can still be provided in offline state, such as displaying user information, filling in simple business forms and other functions.
[0055] The data cache in the offline state is realized by the encapsulated database operation class, which is specifically realized by the offline data module interface. The offline data module interface includes a new data encapsulation class, a query data encapsulation class, a delete data encapsulation class and a download data encapsulation class.
[0056] In this embodiment, the core code of the newly added data encapsulation class is as follows:
[0057] Name: saveData
[0058] Input parameters:
[0059] dbName: String, namespace, corresponding to the database name. It is recommended to use the current login user id + appid;
[0060] type: String, business data type identifier;
[0061] key: String, business primary key name, for example, for device, it is its devId;
[0062] data: Array, business data list;
[0063] Example:
[0064] cordova.exec(function (success) {}, function (error) {}, "OfflinePlugin", "saveData", [{
[0065] dbName: 'userId_appId',
[0066] type: 'devices',
[0067] key: 'devId',
[0068] data: [{key1: value1, key2: value2}]}]);
[0069] The core code of the query data encapsulation class is as follows:
[0070] Name: queryDataByCondition
[0071] Input parameters:
[0072] dbName: String, namespace, corresponding to the database name. It is recommended to use the current logged-in user + appid;
[0073] type: String, business data type identifier;
[0074] query: Array, condition query array;
[0075] Example:
[0076] cordova.exec(function (data) {
[0077] / / data: [{id, key, data}]
[0078] }, function (error) {
[0079] }, "OfflinePlugin", "queryDataByCondition", [{
[0080] dbName: 'userId_appId',
[0081] type: 'devices',
[0082] query: [{devFullName: 'Underground Mining Plant', devName: 'Filter Press'}]
[0083] }]);
[0084] The core code for deleting the data encapsulation class is as follows:
[0085] Name: deleteDataById
[0086] Input parameters:
[0087] dbName: String, namespace, corresponding to the database name. It is recommended to use the current logged-in user + appid;
[0088] type: String, business data type identifier;
[0089] ids: Array, id array;
[0090] Example
[0091] cordova.exec(function (success) {
[0092] }, function (error) {
[0093] }, "OfflinePlugin", "deleteDataById", [{
[0094] dbName: 'userId_appId',
[0095] type: 'devices',
[0096] ids: [123, 124, 125]
[0097] }]);
[0098] The core code of the download data encapsulation class is as follows:
[0099] Name: fetchData
[0100] Description: Download data directly to the local database through the app (suitable for scenarios with large amounts of data);
[0101] Input parameters:
[0102] dbName: String, namespace, corresponding to the database name. It is recommended to use the current login user id + appid
[0103] url: String
[0104] type: String, business data type identifier
[0105] key: String, business primary key name, for example, for device it is its devId
[0106] data: Object, body parameter of the request, such as the complete zjinfo input structure
[0107] token: String, token required by the request header
[0108] Example
[0109] cordova.exec(function (success) {
[0110] }, function (error) {
[0111] }, "OfflinePlugin", "fetchData", [{
[0112] dbName: 'userId_appId',
[0113] url: 'http: / / www.baidu.com',
[0114] type: 'devices',
[0115] key: 'devId',
[0116] data: '{key1: value1, key2: value2}',
[0117] token: 'abcd123456',
[0118] }]);
[0119] The offline data module interface allows developers to operate on data without a network connection to ensure that the application can still run normally in an offline state. Among them, the encapsulation method can quickly realize the operation of offline data.
[0120] The offline data module interface also encapsulates other operation classes, including but not limited to: initialization module initialize(), including creating or opening database files, setting default configurations, etc., with no parameters and no return values; listening to data changes addListener(callback:Function), registering a callback function, which is called when the data in the offline data module changes.
[0121] The sample code for monitoring data changes is as follows:
[0122] function addListener(callback) {
[0123] / / Implement the logic of monitoring data changes
[0124] console.log('Listening for data changes...');
[0125] / / Assume that callback is called when data changes
[0126] callback({ key: 'Key1', data: { value: 'Value1'}});
[0127] }
[0128] Function call: addListener(console.log);
[0129] S3, synchronize local and server data, deploy edge computing nodes at the mine site, use the edge computing nodes to deploy task schedulers to handle data synchronization tasks, and use data synchronization strategies to automatically synchronize local and server data, the data synchronization strategies include: incremental synchronization, handling data conflicts and ensuring data consistency, wherein the incremental synchronization implementation steps include: when the application is started or triggered regularly, detecting the update status of the server data, obtaining the last update timestamp of the server data, comparing the update timestamps of local data and the server data, obtaining the changed data and synchronizing it to the local, wherein the server sends data update notifications by push or polling;
[0130] In this embodiment, edge computing nodes are deployed on-site at the mine to ensure that data can be processed locally. The edge nodes are responsible for collecting sensor data, equipment status information, etc., and perform preliminary processing on the collected data on the edge nodes, such as data cleaning, format conversion, and compression.
[0131] Real-time tasks are executed on edge nodes without waiting for network transmission and cloud processing. For example, fault diagnosis algorithms are deployed on edge nodes to process device status information in real time.
[0132] In this embodiment, the incremental synchronization core code is as follows:
[0133] function incrementalSync() {
[0134] const lastSyncTime = getLastSyncTime();
[0135] fetch(`server_endpoint?lastSyncTime=${lastSyncTime}`)
[0136] .then(response => response.json())
[0137] .then(data => {
[0138] updateLocalData(data);
[0139] setLastSyncTime(Date.now());
[0140] })
[0141] .catch(error => console.error('Error during incrementalsync:', error));
[0142] }
[0143] / / Set up scheduled tasks or event listeners to regularly check and synchronize data.
[0144] setInterval(() => {
[0145] incrementalSync();
[0146] }, 60000); / / Synchronize once every minute
[0147] Incremental synchronization only transmits the changed data, reducing the amount of data transmission and processing time, and improving synchronization efficiency.
[0148] The steps for implementing the data conflict processing include: when a data conflict occurs, obtaining the data modification time or version number, and the client and the server determine which data in the conflicting data is valid data through the data modification time or the version number.
[0149] The core code for processing data conflicts in this embodiment is as follows:
[0150] function resolveConflict(localData, serverData) {
[0151] if (localData.version > serverData.version) {
[0152] return localData;
[0153] } else {
[0154] return serverData;
[0155] }
[0156] }
[0157] Determining the latest version of data through timestamp or version number is simple and effective.
[0158] In this embodiment, optimistic locking is used to solve the data conflict problem. In the process of resolving conflicts, no new conflicts will appear again to cause data management confusion, thereby improving data management efficiency.
[0159] In this embodiment, optimistic locking is used for data caching, and the following example is provided: when the client obtains data from the server, it receives a version number or timestamp. When the client modifies the data offline and tries to synchronize the data to the server again, it carries the latest version number or timestamp. When the server receives the synchronization request, it checks whether the version number or timestamp matches the version on the server. If the versions match, the data is updated. If they do not match, it is considered that a conflict has occurred, and the user can choose to reject the update or prompt the user to resolve the conflict.
[0160] In the present invention, using transactions for data processing can also further solve the problem of data consistency defects in high concurrency scenarios caused by optimistic locking.
[0161] The steps for ensuring data consistency include: using transactions in local and server data operations to ensure data atomicity and consistency, and using checksum or hash verification mechanisms in data transmission and storage to ensure that data is not tampered with during transmission.
[0162] Transactions ensure the atomicity and consistency of data operations, preventing data inconsistency caused by partial operation failures. Verification mechanisms such as checksums and hashes ensure the integrity of data during transmission and prevent data tampering.
[0163] The offline, cache and delay in the actual upload behavior are caused by the network connection status. The method of checking the network connection status includes: checking by HTTP request, checking by socket library, checking by ping command and checking by network status listener.
[0164] S4. When data caching fails due to the harsh network environment of the mine, configure automatic deployment and rollback strategies to ensure that the local data is synchronized with the server data. At the same time, record the operation logs before and after the rollback, analyze the operation logs, adjust the cache strategy and update the cache rules. In particular, grayscale release is used to roll back the traffic to the previous version of data processing.
[0165] The configuration automatic deployment and rollback specifically also includes: using a version control system to manage software code to ensure that each update has a version record, using CI / CD tools to achieve automatic construction and testing, using Docker containers to encapsulate applications and Kubernetes container orchestration tools to achieve cluster automatic management, and when an update fails, it can quickly roll back to the previous stable version to reduce downtime. The automatic deployment data model is specifically represented as follows:
[0166]
[0167] Among them, p(D) represents the average metric value of the deployment strategy model, p( ) represents the deployment metric value of the i-th deployment unit, and N represents the number of deployment units;
[0168] The mathematical model of the rollback strategy is specifically expressed as follows:
[0169]
[0170] Among them, R(S) represents the metric value of the entire model, M represents the number of rollbacks, and R( ) represents the metric value of the jth rollback unit, Represents the jth rollback unit.
[0171] The mine network is a harsh environment, and extreme network disconnection often occurs, so a large amount of data will not be synchronized. Data cache synchronization when the network is connected is not completed in a short time. Although the network status cycle detection will start data update and cache once it detects that it is connected to the network, the network environment of the mine sometimes cannot support the update time and directly drops the line. At this time, the update will be interrupted and synchronization problems will often occur. Therefore, the present invention introduces an automatic deployment and rollback mechanism. When the data cache update fails, the system version is automatically pulled back to the previous version and the system is quickly put into operation. There will be no card segment and shutdown due to disconnection, and the mine operation will be maintained continuously. At the same time, data synchronization and local caching continue, waiting for the arrival of the next iteration, and the whole process of automatic caching of offline data is fully guaranteed.
[0172] In a specific embodiment, after each rollback is executed, the system can record the operation logs before and after the rollback in real time, collect user feedback, understand the effect after the rollback, analyze the performance data of the system before and after the rollback, and if the performance is reduced, find the entry point for improvement, such as adjusting the specific network indicators in the network configuration process algorithm, or reconfiguring the delayer. Adjust the cache strategy according to the analysis results, such as increasing the persistence frequency of cached data, optimizing the storage format of cached data, etc. Update the cache rules to ensure the consistency and validity of the cached data. Resynchronize the cached data after the rollback to ensure that the cache is consistent with the system status, introduce data verification, verify the integrity and consistency of the cached data, and ensure that the data is not lost or damaged. Feedback the feedback information after the rollback to the offline cache management, continuously optimize the cache strategy, continuously collect feedback information, and continuously adjust the cache strategy to improve the stability and reliability of the system.
[0173] Furthermore, in some embodiments, Figure 2 As shown, a schematic diagram of a specific process of offline data caching in a mine without network scenario according to a specific embodiment of the present invention is shown, and the specific steps are as follows:
[0174] Establish a local database on the APP side to store offline data.
[0175] When the APP starts, it checks the network connection status. If the network is unobstructed, the application goes online and obtains data directly from the server. If the network is offline, it reads data from the local database to ensure that the application can run normally.
[0176] When the network connection is restored, the local database and server data are automatically synchronized to ensure data consistency.
[0177] Design appropriate data caching strategies for different business scenarios to improve offline usage experience.
[0178] In summary, the technical solution of the present invention realizes:
[0179] Database integration: Choose the lightweight, cross-platform database SQLite and embed it into the APP.
[0180] Data model design: Design a suitable data model based on the business requirements of the APP and persist it in the database.
[0181] Data read and write interface: encapsulates the database read and write logic, provides a unified CRUD interface, and simplifies the operations of developers.
[0182] Data synchronization mechanism:
[0183] When the APP is connected to the Internet, the data in the database will be automatically synchronized and updated with the server.
[0184] When offline, you can still access cached data in the database to provide basic business functions.
[0185] When the network is restored, the local database is automatically merged with the server data to ensure data consistency.
[0186] When data caching fails, the rollback mechanism can be used in time to ensure data consistency.
[0187] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0188] The modules involved in the embodiments of the present application may be implemented by software or by hardware.
[0189] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.
Claims
1. A data offline caching method for mines without network, characterized in that: The following steps are involved: S1. Real-time detection of the network environment in the mine, identification of the types of data that need to be cached in the mine, including: sensor data, equipment status information and production data, initialization and classification storage of the collected data, wherein the real-time detection adopts a network status cycle detection method, uses the network strategy pre-configured in the background to dynamically generate a network configuration process algorithm, and obtains device network indicators regularly through the cordova-plugin-network-information plug-in, obtains the conditional parameters in the network configuration process algorithm, and determines the execution strategy list for the acquired data in the current request, wherein the specific network indicators in the network configuration process algorithm include: connection type, signal strength, uplink and downlink speeds, data transmission volume, packet loss rate, analysis cycle, data packet size and actual upload behavior, wherein the actual upload behavior includes: offline cache, online upload, delayed upload and refusal to upload; S2. Design a cache system architecture, obtain the network index of the current request, and determine the actual upload behavior of the current request in combination with the execution strategy list. If the actual upload behavior is offline caching, save the data in the current request to the local database. If the actual upload behavior is online upload, request the background interface to upload the data. If the actual upload behavior is delayed upload, store the data in the current request in the running memory and delay the upload through a timer. S3, synchronize local and server data, deploy edge computing nodes at the mine site, use the edge computing nodes to deploy task schedulers to process data synchronization tasks, and use data synchronization strategies to automatically synchronize local and server data. The data synchronization strategies include: incremental synchronization, handling data conflicts, and ensuring data consistency. The incremental synchronization implementation steps include: when the application is started or triggered regularly, detecting the update status of the server data, obtaining the last update timestamp of the server data, comparing the update timestamp of the local data and the server data, obtaining the changed data and synchronizing it to the local data, wherein the server sends data update notifications by push or polling, wherein the data conflict processing implementation step includes: when data conflicts occur, obtaining the data modification time or version number, and the client and the server determine which data in the conflicting data is valid data through the data modification time or the version number, and the data consistency assurance implementation step includes: using transactions in local and server data operations to ensure data atomicity and consistency, and using checksum or hash verification mechanisms in data transmission and storage to ensure that data has not been tampered with during transmission; S4. When data caching fails due to the harsh network environment of the mine, configure automatic deployment and rollback strategies to ensure that the local data is synchronized with the server data. At the same time, record the operation logs before and after the rollback, analyze the operation logs, adjust the cache strategy and update the cache rules, wherein the traffic is rolled back to the previous version of the data processing by grayscale release, wherein the configuration of automatic deployment and rollback strategies specifically also includes: using the version control system to manage software code, ensuring that each update has a version record, using CI / CD tools to achieve automated building and testing, using Docker containers to encapsulate applications and Kubernetes container orchestration tools to achieve cluster automation management, and when the update fails, it can quickly roll back to the previous stable version to reduce downtime, wherein the automatic deployment data model is specifically expressed as follows: Among them, p(D) represents the average metric value of the deployment strategy model, p( ) represents the deployment metric value of the i-th deployment unit, and N represents the number of deployment units; The mathematical model of the rollback strategy is specifically expressed as follows: Among them, R(S) represents the metric value of the entire model, M represents the number of rollbacks, and R( ) represents the metric value of the jth rollback unit, Represents the jth rollback unit.
2. The offline caching method according to claim 1, characterized in that: The analysis period in the network indicator determines the timing period of the network status cyclic detection.
3. The offline caching method according to claim 1, characterized in that: The local database specifically uses SQLite database as the local database to store user information, wherein a data table structure including user ID, business field and last updated timestamp is configured, and data caching in offline state is realized through encapsulated database operation class.
4. The offline caching method according to claim 3, characterized in that: The data cache in the offline state is realized by the encapsulated database operation class, which is specifically realized by the offline data module interface. The offline data module interface includes a new data encapsulation class, a query data encapsulation class, a delete data encapsulation class and a download data encapsulation class.
5. The offline caching method according to claim 1, characterized in that: The offline, cache and delay in the actual upload behavior are caused by the network connection status. The method of checking the network connection status includes: checking by HTTP request, checking by socket library, checking by ping command and checking by network status listener.
6. A computer program product, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
