Data storage method, device and system based on file cache and medium

By adopting a file cache-based data storage method in the industrial Internet of Things scenario, using memory to record data accumulation and run time to batch store data, and failing to recover during downtime, the problem of low data storage efficiency and loss risk is solved. Achieve high concurrency performance and low cost data storage solutions.

CN119046364APending Publication Date: 2024-11-29JIANGSU TIANHE ENERGY STORAGE CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411184918.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

In the industrial Internet of Things scenario, when device data is uploaded to the cloud, it is necessary to solve the problem of carrying a large number of concurrent connections and massive data quickly falling into the disk. The existing technology has the defects of low data storage efficiency, risk of data loss and additional processing links to increase request response time. .

Method used

Using a file cache-based data storage method, by recording the accumulated data and running time in memory, batch storage of data to the database when preset conditions are met, and failure recovery process is carried out when abnormal downtime is performed, using the MQTT protocol and the server Communication to ensure data security and reliability.

Benefits of technology

It realizes real-time data drop, improves concurrency performance, reduces the cost of cloud on the device, has the ability to recover and failover, reduces system magnitude and maintenance costs, and avoids the risk of data loss.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046364A_ABST
    Figure CN119046364A_ABST
Patent Text Reader

Abstract

The invention discloses a data storage method, device and system based on file cache and a medium, and relates to the technical field of data storage, the data storage method comprises the following steps: in response to an obtained data request, obtaining data from the data request, writing the data into the file cache, recording the accumulated data volume of the data in the file cache in the memory and / or recording the running time of the file cache in the memory; and when it is judged that a data batch storage condition is met based on the accumulated data volume and / or the running time, storing the data in the file cache into a database in batches. According to the data storage method, a file caching mode is used, the data is synchronously written into the file in the request, which is equivalent to real-time disk dropping of the data, the data is safe and complete, no data loss risk exists during downtime, the data writing speed is high, the data processing links are few, the possible fault points are few, and the fault risk of the whole process is effectively reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a data storage method, device, system and medium based on file cache. Background Art

[0002] In the industrial Internet of Things scenario, when device data is uploaded to the cloud, there are many concurrent connections (large device scale), small data transmission interval, and large data message volume (many measurement points). Therefore, a solution that can solve the problem of carrying a large number of concurrent connections and quickly storing massive data is needed. The most fundamental problem of concurrent carrying capacity is to reduce the response time of data processing, and the data storage process accounts for the largest proportion of time. The existing data storage method is that one data request triggers one data storage. At this time, the processing efficiency is extremely low. Even if the storage process time is 1 millisecond, the concurrency that a single machine can carry per second is only 1000 milliseconds. However, in actual situations, even the most efficient database component in a single machine environment cannot meet the 1 millisecond requirement. For this reason, batch operation is generally used in the prior art for data storage, which improves data storage efficiency by reducing the number of responses. However, data is stored in batches in the memory cache, and there is a risk of data loss during downtime.

[0003] The existing data storage methods also have the following defects: (1) Using memory cache as the cache medium, data is stored in batches in the memory cache and waits for multi-threaded asynchronous batch processing. When the service crashes abnormally, the data in the cache will be lost. (2) Introducing multi-level cache or additional third-party components will increase unnecessary data processing links and increase request response time. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present application proposes a data storage method, device, system and medium based on file caching. On the basis of solving the problems of carrying a large amount of concurrency and quickly dropping data to disk, it has higher data security, more reliable fault transfer recovery capabilities and lower equipment cloud costs.

[0005] In a first aspect, the present application provides a data storage method based on file caching, comprising:

[0006] In response to the acquired data request, acquiring data from the data request, writing the data into the file cache, recording the accumulated data volume of the data in the file cache in the memory, and / or recording the running time of the file cache in the memory;

[0007] When it is determined based on the accumulated data volume and / or the running time that the data batch storage condition is met, the data in the file cache is batch stored in the database.

[0008] In some embodiments, when it is determined based on the accumulated data volume and / or the running time that a data batch storage condition is met, batch storing the data in the file cache to the database includes:

[0009] When there is data in the file cache and the accumulated data volume of the data reaches a preset quantity indicator and / or there is data in the file cache and the running time of the file cache reaches a preset time indicator, it is determined that the data batch storage conditions are met, and the data in the file cache is batch stored in the database.

[0010] In some embodiments, after storing the data in the file cache in batches to the database, the method further includes: resetting the accumulated data volume recorded in the memory, and / or resetting the running time recorded in the memory.

[0011] In some embodiments, storing the data in the file cache in batches to the database includes:

[0012] The data in the file cache is written into the database of the server, and it is determined whether the writing is successful. If so, the data in the file cache is cleared; otherwise, the data in the file cache is written into the disaster recovery file.

[0013] The above-mentioned writing of the data in the file cache into the database of the server includes: the device end where the file cache is located communicates with the server end based on the MQTT protocol to realize writing of the data into the database of the server end.

[0014] In some embodiments, the method further includes: when an abnormal downtime occurs during the data batch storage process, executing a fault recovery process based on device restart, wherein the fault recovery process includes:

[0015] Check if there is any leftover data in the file cache;

[0016] If there is any remaining data, writing the remaining data in the file cache into the database;

[0017] If there is no remaining data, check whether there is data in the disaster recovery file. If yes, write the data in the disaster recovery file into the database. Otherwise, directly terminate the fault recovery process.

[0018] In a third aspect, the present application provides a data storage device based on file cache, comprising a first processing module, a second processing module, a file cache and a memory;

[0019] The first processing module is used to, in response to the acquired data request, acquire data from the data request, write the data into the file cache, record the accumulated data volume of the data in the file cache in the memory, and / or record the running time of the file cache in the memory;

[0020] The second processing module is used to batch store the data in the file cache into the database when it is determined that a data batch storage condition is met based on the accumulated data volume and / or the running time recorded in the memory.

[0021] In a fourth aspect, the present application provides a data storage system, including a device side and a server side, wherein the device side includes the above-mentioned data storage device, and the server side includes the database.

[0022] In a fifth aspect, the present application provides an electronic device, comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, and the program codes are suitable for being loaded and run by the processor to execute the above-mentioned file cache-based data storage method.

[0023] In a sixth aspect, the present application provides a computer-readable storage medium, which stores a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the above-mentioned file cache-based data storage method.

[0024] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0025] In the technical solution of the present application, local files are used as cache media. Different from the asynchronous disk drop of memory cache, data is synchronously written to the file in the request, which is equivalent to real-time disk drop of data. The data is safe and complete, and there is no risk of data loss during downtime. Using the file cache method, the data writing speed is fast (microsecond level), which further improves the concurrency performance. Under the same configuration server, it can carry more devices and reduce the average cloud cost of the device. Further, the present application uses the file cache method, which has the ability of fault recovery and failover. When the storage process is abnormally down, it does not affect the ability to write data, there is no risk of data loss, and it has the fault recovery ability of its own node (such as equipment). When a downstream node (such as a server) fails, the data can be recorded in the disaster recovery file, and when the downstream fault node is restored, the data in the disaster recovery file is restored to the normal process, which has the failover ability and fault recovery ability for the downstream node. The data storage method of the present application does not use any other components except the necessary data entry component and data export component. The system level is small, which reduces the server cost, implementation cost, and maintenance cost. It supports distributed deployment and can be horizontally expanded. There are no centralized components and no single point failure risk. The data storage method of the present application has fewer data processing links and fewer possible failure points, which effectively reduces the risk of overall process failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The disclosure of the present application will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. In addition, similar numbers in the drawings are used to represent similar components, among which:

[0027] Figure 1 This is a schematic diagram of an application scenario of a data storage method based on file cache according to the present application;

[0028] Figure 2 It is a flowchart of the main steps of a data storage method based on file cache provided in an embodiment of the present application;

[0029] Figure 3 This is a schematic diagram of a specific implementation process of a data access process provided according to an embodiment of the present application;

[0030] Figure 4 This is a schematic diagram of a specific implementation process of a data storage process provided according to an embodiment of the present application;

[0031] Figure 5 This is a schematic diagram of a specific implementation process of a fault recovery process provided according to an embodiment of the present application;

[0032] Figure 6 It is a schematic diagram of the main structure block diagram of a data storage device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0034] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0035] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0036] like Figure 1 As shown, it is a schematic diagram of the application scenario of the embodiment of the present application. The application scenario diagram includes a device end 110 and a server end 120. In the embodiment of the present application, the device end 110 includes but is not limited to mobile phones, tablet computers, laptops, desktop computers, vehicle-mounted terminals and other devices; the device end may be installed with a component for accessing the database of the server end 120, and the server end 120 is a background server corresponding to the device end that can be used to implement data on the device end to the cloud, which is not specifically limited in this application. The server end 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, and big data and artificial intelligence platforms. The data storage method of the embodiment of the present application can be applied to Figure 1In the hardware environment shown as consisting of a device end 110 and a server end 120, the method can use the device end 110 as the executor of the data storage method. When the device end responds to a data request and needs to store the data to the server end, the device end uses file caching to synchronously write the data, which is equivalent to real-time data storage. The data writing speed can reach microseconds. Only the current quantity indicator or time indicator is saved in the device end memory. When it is judged based on the quantity indicator or time indicator that the batch processing conditions are met, the data in the batch read file is uploaded to the server end. Furthermore, the device end 110 and the server end 120 communicate based on the MQTT protocol. The MQTT service is used as the data entry on the server end 120, and the time series database is used as the final storage medium. Through a lightweight storage process design proposed in this application, the problem of large concurrent connections and rapid storage of massive data when the device is on the cloud can be solved.

[0037] MQTT: (Message Queuing Telemetry Transport) is a lightweight communication protocol used to transmit messages between devices. It is based on the TCP / IP protocol or other network protocols. Based on the publish / subscribe model, it allows devices to publish messages to specific topics and allow devices subscribed to the same topic to receive these messages. MQTT message headers are relatively small, occupying less bandwidth and resources, making it perform well in restricted network environments.

[0038] In order to improve the efficiency of data storage, the industry generally uses batch operations for data storage methods, and the data request ends directly after entering the memory cache. In the data storage process implemented by the present application based on file caching and batch operations, local files are used as cache media, which is equivalent to directly dropping data to the disk, taking advantage of the high efficiency of file writing, and completing data writing within the data request, which is equivalent to synchronously writing real-time data. In the batch operation mode, a time window or a quantity window can be used to determine when a specified time or a specified quantity is reached, and batch processing of batch data in the memory cache is carried out to reduce the number of connections and operations to the database and compress I / O processing time.

[0039] See attached Figure 2 The present application provides a data storage method based on file caching, which can be applied to Figure 1 The device side shown in the figure, the data storage space of the device side includes memory and file cache, and the data storage method proposed in the embodiment of the present application mainly includes the following steps:

[0040] Step S11: in response to the acquired data request, acquiring data from the data request, writing the data into the file cache, recording the accumulated data volume of the data in the file cache in the memory, and / or recording the running time of the file cache in the memory;

[0041] In actual application scenarios, the device side can be specifically an energy storage device, and the data can be specifically device data related to battery performance collected and obtained by the energy storage device. In actual applications, a component for accessing the database of the server side, namely the MQTT component, is installed on the device side. A large amount of device data is transmitted to the MQTT component of the device side in a high-concurrency manner, triggering the device side to start the data storage process of this application. At this time, the device side needs to process a large number of data requests concurrently. In a single data request, the device side only completes two tasks, accumulating the number of data and writing the data to the file cache, where the accumulated number is completed in the memory, and the time occupied can be ignored. The speed of writing files is not affected by the number of concurrent requests, and the time occupied is about 5 microseconds on average. Theoretically, the upper limit of concurrent requests allowed per second for a single machine is 200,000. The embodiment of the present application achieves a rapid response to high-concurrency data requests through a lightweight data access process.

[0042] In one implementation, whether the conditions for batch data storage are met may be determined based on a data volume index and / or a time index.

[0043] Step S12: When it is determined based on the accumulated data volume and / or the running time that the data batch storage condition is met, the data in the file cache is batch stored in the database.

[0044] Specifically in this embodiment, when there is data in the file cache and the accumulated data volume of the data reaches a preset quantity indicator, and / or there is data in the file cache and the running time of the file cache reaches a preset time indicator, it is determined that the data batch storage condition is met, and the data in the file cache is stored in batches to the database. In an embodiment of the present application, the device side communicates with the server side based on the MQTT protocol to implement batch writing of the data in the file cache of the device side to the database of the server side.

[0045] In a specific implementation manner, the above step S12 may specifically include steps 121 to 123.

[0046] Step 121: Determine whether there is data in the file cache, if yes, execute step 122, otherwise continue to execute step 121;

[0047] Step 122: determine whether the accumulated data volume recorded in the memory and / or the running time recorded in the memory meet the data batch storage condition, if yes, store the data in the file cache in batches to the database and execute step 123; otherwise, continue to execute step 122;

[0048] Specifically, two thresholds are pre-stored in the memory, namely a preset quantity indicator and a preset time indicator. The method of judging whether the cumulative data volume recorded in the memory satisfies the data batch storage condition indicator is specifically to judge whether the cumulative data volume reaches the preset quantity indicator. The method of judging whether the running time recorded in the memory satisfies the data batch storage condition indicator is specifically to judge whether the running time reaches the preset time indicator.

[0049] Step 123: Reset the accumulated data volume recorded in the memory, and / or reset the running time recorded in the memory.

[0050] In this embodiment, the specific implementation method of this step may be: clearing the accumulated data volume and / or running time recorded in the memory.

[0051] In the above step 122, the batch storage of the data in the file cache to the database is specifically to write the data in the file cache to the database of the server, and determine whether the writing is successful. If so, clear the data in the file cache, otherwise write the data in the file cache to the disaster recovery file.

[0052] Based on the above step S12, when an abnormal downtime occurs during the data batch storage process, a fault recovery process can be executed based on device restart, and the fault recovery process includes: checking whether there is any remaining data in the file cache; if there is any remaining data, writing the remaining data in the file cache into the database; if there is no remaining data, checking whether there is data in the disaster recovery file, and if so, writing the data in the disaster recovery file into the database; otherwise, directly ending the fault recovery process.

[0053] For example, Figures 3 to 5 This is the main implementation process of a data storage method based on file caching provided by the embodiment of the present application, including data access process, data storage process and fault recovery process. In this example, the device data in the energy storage device is written into the database of the cloud server based on the MQTT component system. It should be understood that the MQTT component system here can be understood as a system composed of components used to implement MQTT communication between the energy storage device and the cloud server. Figures 3 to 5 In the process shown, some operations performed by the energy storage device can be understood as functions implemented by the device side based on the MQTT component, such as receiving data, writing file cache, reading file cache, batch writing to the database, etc.

[0054] like Figure 3In the data access process shown, a large amount of device data enters the system through the data entry in a high-concurrency manner. In a single data request, the system only completes two tasks, accumulating the data and writing the data to the file cache. The accumulation is completed in the memory, and the time occupied is negligible. The speed of writing files is not affected by the number of concurrent requests, and the average time occupied is about 5 microseconds. In theory, the upper limit of concurrent requests per second allowed by this system is 200,000. Through the lightweight access process, a fast response to high-concurrency data requests is achieved, and choosing to write files in real time in the request can maximize the security of the data.

[0055] like Figure 4 In the data storage process shown, the system uses two indicators to determine whether to process data in batches to reduce the pressure on the database. The two indicators are time indicator and quantity indicator. The batch storage logic starts only when the time exceeds the specified interval or the number of data exceeds the specified number. First, determine whether the file cache is an empty file. When the file cache is not empty, determine the batch processing threshold. When the threshold condition is met, read the file and write it to the database. The data reading speed is based on the size of the file cache. It takes 1 millisecond per 1,000 items on average. Compared with the tens to hundreds of milliseconds of time occupied when writing to the database, the overall occupancy rate can be ignored, and the efficiency is very high. In order to further improve the availability of the system, when writing to the database, it will determine whether the write is successful. After the write is successful, the file cache will be cleared to ensure the security of the data. If the write is unsuccessful, the data will be written to the disaster recovery file, and there will be an additional disaster recovery processing thread to be responsible for the recovery of the disaster recovery data. Compared with the existing technology, this system does not use memory as a cache medium, which maximizes the security of the data. No third-party components are used, and the lightweight architecture reduces risk links and average costs.

[0056] like Figure 5 In the fault recovery process shown, this solution will solve the fault recovery problem from two angles. First, when the data storage process crashes abnormally, it will not affect the writing of the file cache by the data access process. At this time, if the data storage process is restarted normally, the system will detect whether there is any remaining data, and the system itself can be restored. Secondly, when a downstream node (database) fails, the data will be written to the disaster recovery file by the storage process, and the failover of the downstream faulty node will be realized without affecting the access process. When there is data in the disaster recovery file, there will be a special processing thread to asynchronously process the data in the disaster recovery file. At this point, the system has the ability to recover from the faults of the system's own nodes, the ability to failover when downstream nodes fail, and the ability to automatically recover from the faulty nodes when they recover.

[0057] Figures 3 to 5 The data access process, data storage process and fault recovery process shown in the figure are mainly implemented in the same way as in this application. Figure 2 The method flow of the illustrated embodiments is consistent and will not be repeated here.

[0058] See attached Figure 6 , a data storage device based on file cache provided in an embodiment of the present application mainly includes a first processing module 201, a second processing module 202, a file cache 203 and a memory 204; wherein:

[0059] The first processing module 201 is used to respond to the acquired data request, acquire data from the data request, write the data into the file cache 203, record the accumulated data volume of the data in the file cache 203 in the memory 204, and / or record the running time of the file cache in the memory 204;

[0060] The second processing module 202 is used to batch store the data in the file cache 203 to the database when it is determined that a data batch storage condition is met based on the accumulated data volume and / or the running time recorded in the memory 204.

[0061] Furthermore, the present application also provides a data storage system, including a device side and a server side, wherein the device side includes the above-mentioned data storage device, and the server side includes the database.

[0062] For ease of explanation, the introduction to the above-mentioned data storage device and data storage system only shows the part related to the embodiments of the present application. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application.

[0063] It should be understood that since the setting of each module is only for illustrating the functional units of the present application, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.

[0064] It is understood by those skilled in the art that each module in the system can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of this application, and therefore, the technical solutions after splitting or merging will fall within the protection scope of this application.

[0065] Furthermore, the present application also provides an electronic device.

[0066] In an embodiment of an electronic device according to the present application, the electronic device mainly includes a processor and a storage device, the storage device can be configured to store a program for executing the data storage method based on file cache of the above method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to the program for executing the data storage method based on file cache of the above method embodiment. For ease of explanation, only the part related to the embodiment of the present application is shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application.

[0067] In the embodiment of the present application, the electronic device may be a control device device formed by various electronic devices. In some possible implementations, the electronic device may include multiple storage devices and multiple processors. The program for executing the data storage method based on file cache of the above method embodiment can be divided into multiple subprograms, and each subprogram can be loaded and run by the processor to execute different steps of the data storage method based on file cache of the above method embodiment. Specifically, each subprogram can be stored in different storage devices, and each processor can be configured to execute programs in one or more storage devices to jointly implement the data storage method based on file cache of the above method embodiment, that is, each processor executes different steps of the data storage method based on file cache of the above method embodiment, to jointly implement the data storage method of the above method embodiment.

[0068] The above-mentioned multiple processors may be processors deployed on the same device. For example, the above-mentioned electronic device may be a high-performance device composed of multiple processors, and the above-mentioned multiple processors may be processors configured on the high-performance device.

[0069] Furthermore, the present application also provides a computer-readable storage medium.

[0070] In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium may be configured to store a program for executing the data storage method based on file caching of the above method embodiment, and the program may be loaded and run by a processor to implement the above data storage method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0071] So far, the technical solutions of the present application have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A data storage method based on file cache, characterized in that: include: In response to the acquired data request, acquiring data from the data request; Writing the data into a file cache; Recording the accumulated data volume of the data in the file cache in the memory and / or recording the running time of the file cache in the memory; When it is determined based on the accumulated data volume and / or the running time that the data batch storage condition is met, the data in the file cache is batch stored in the database.

2. The data storage method according to claim 1, characterized in that: When it is determined based on the accumulated data volume and / or the running time that the data batch storage condition is met, batch storing the data in the file cache to the database includes: When there is data in the file cache and the accumulated data volume of the data reaches a preset quantity indicator and / or there is data in the file cache and the running time of the file cache reaches a preset time indicator, it is determined that the data batch storage conditions are met, and the data in the file cache is batch stored in the database.

3. The data storage method according to claim 2, characterized in that: After storing the data in the file cache in batches into the database, the method further includes: Reset the accumulated data volume recorded in the memory, and / or reset the running time recorded in the memory.

4. The data storage method according to claim 1, characterized in that: The storing of the data in the file cache in batches to the database comprises: Write the data in the file cache to the server's database; It is determined whether the writing is successful, if yes, the data in the file cache is cleared, otherwise the data in the file cache is written into the disaster recovery file.

5. The data storage method according to claim 4, characterized in that: Writing the data in the file cache into the database of the server includes: The device where the file cache is located communicates with the server based on the MQTT protocol to write the data into the database of the server.

6. The data storage method according to claim 4, characterized in that: The method further comprises: When an abnormal downtime occurs during the data batch storage process, a fault recovery process is performed based on device restart, wherein the fault recovery process includes: Check if there is any leftover data in the file cache; If there is any remaining data, writing the remaining data in the file cache into the database; If there is no remaining data, check whether there is data in the disaster recovery file. If yes, write the data in the disaster recovery file into the database. Otherwise, directly terminate the fault recovery process.

7. A data storage device based on file cache, characterized in that: The data storage device includes a first processing module, a second processing module, a file cache and a memory; The first processing module is used to, in response to the acquired data request, acquire data from the data request, write the data into the file cache, record the accumulated data volume of the data in the file cache in the memory, and / or record the running time of the file cache in the memory; The second processing module is used to batch store the data in the file cache into the database when it is determined that a data batch storage condition is met based on the accumulated data volume and / or the running time recorded in the memory.

8. A data storage system, characterized in that: It includes a device side and a server side, wherein the device side includes the data storage device according to claim 7, and the server side includes the database.

9. An electronic device comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the data storage method based on file cache according to any one of claims 1 to 6.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the data storage method based on file cache as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Data storing system and method based on cache

    CN101576855A

  • Data storage method and device

    CN105446893A

  • Data storage management method and device, electronic equipment and readable storage medium

    CN112650807A

  • Data processing method and device

    CN113792074A

  • Time series data management method, device and equipment and readable storage medium

    CN114490802A