Data writing, reading method and system, device, electronic equipment, storage medium

Abstract

The data writing and reading methods, systems, devices, electronic devices, and storage media disclosed herein relate to storage technology. Specific implementation schemes include writing frequently accessed data into a file management module, thereby utilizing the file management module to maintain cached data. This solves the problem in existing technologies where storing cached data in memory leads to the easy loss of cached data after process restarts.

Description

Technical Field

[0001] This disclosure relates to computer technology, and more particularly to storage technology. Background Technology

[0002] A storage system is a system in a computer that consists of various storage devices for storing programs and data, control components, management information scheduling devices (hardware), and algorithms (software). A computer's storage system generally consists of cache, main memory, and secondary storage.

[0003] Caching is an important component of storage systems. Traditional caching systems often utilize memory to store cached data.

[0004] However, this approach will cause all previously accumulated cached data to become invalid after the process restarts, leading to a large number of cache misses and causing a decline in system performance. Summary of the Invention

[0005] This disclosure provides a data writing and reading method, system, device, electronic device, and storage medium to solve the technical problem of easy loss of cached data in the prior art.

[0006] The first aspect of this disclosure is to provide a data writing method, including:

[0007] Obtain the data to be written. If the data to be written is high-frequency data, request management index information from the file management module and request storage space from the file management module.

[0008] The data to be written is written to the storage space, and the correspondence between the information of the data to be written and the management index information is recorded.

[0009] Optionally, the information of the data to be written includes the storage location of the data to be written in the file management module.

[0010] In this optional implementation, by recording the relationship between the address of data in the file management module and the management reference information, the data storage location can be located more quickly when reading data, thereby improving the data reading speed.

[0011] Optionally, if the application for the management-related information fails, and / or the application for the storage space fails, then memory index information corresponding to the data to be written is determined in the memory;

[0012] Record the correspondence between the information of the data to be written and the memory index information.

[0013] In this optional implementation, if management index information or storage space cannot be obtained, the data to be written cannot be written to the file management module. In this case, the data can be written to memory as cached data to avoid the inability to record the index information of the data.

[0014] Optional, also includes:

[0015] If the data to be written is not high-frequency data, then the data to be written will be written to the disk module.

[0016] Optional, also includes:

[0017] The management index information of the data to be written is recorded in the LRU list.

[0018] Optionally, the memory index information of the data to be written is recorded in the LRU list.

[0019] In this implementation, cached data can be evicted using an LRU list, thereby releasing the storage space occupied by cached data in a timely manner.

[0020] The second aspect of this disclosure is to provide a data reading method, including:

[0021] Based on the correspondence between the information of the data to be read and the index information, determine whether the data to be read is cached data;

[0022] If so, then based on the correspondence, determine whether the cached data is the first cached data stored in the file management module or the second cached data stored in memory;

[0023] Read the data to be read based on the determined result.

[0024] A third aspect of this disclosure is to provide a data management system, including:

[0025] The file management module is used to record the relationship between cached data and its corresponding index information; wherein, the cached data includes first cached data stored in the file management module and second cached data stored in memory;

[0026] The disk module is used to record raw data;

[0027] The memory module is used to record the second cache data and the memory data.

[0028] A fourth aspect of this disclosure is to provide a data writing device, comprising:

[0029] The acquisition module is used to acquire the data to be written.

[0030] The application module is used to apply to the file management module for management index information and storage space if the data to be written is high-frequency data.

[0031] The writing module is used to write the data to be written into the storage space and record the correspondence between the information of the data to be written and the management index information.

[0032] The fifth aspect of this disclosure is to provide a data reading device, comprising:

[0033] The first determining module is used to determine whether the data to be read is cached data based on the correspondence between the information of the data to be read and the index information.

[0034] The second determining module is used to determine, if yes, whether the cached data is first cached data stored in the file management module or second cached data stored in memory, based on the correspondence.

[0035] The reading module is used to read the data to be read based on the determined result.

[0036] The sixth aspect of this disclosure is to provide an electronic device, comprising:

[0037] At least one processor; and

[0038] A memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a data writing method as described in the first aspect or a data reading method as described in the second aspect.

[0040] A seventh aspect of this disclosure is to provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform a data writing method as described in the first aspect or a data reading method as described in the second aspect.

[0041] The data writing and reading methods, systems, devices, electronic devices, and storage media disclosed herein write frequently accessed data into a file management module, thereby using the file management module to maintain cached data. This solves the problem in the prior art where cached data is stored in memory, leading to easy loss of cached data after process restart. Attached Figure Description

[0042] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0043] Figure 1 This is an exemplary application scenario diagram of this application;

[0044] Figure 2 A flowchart illustrating a data writing method as shown in an exemplary embodiment of this application;

[0045] Figure 3 A flowchart illustrating a data writing method as shown in another exemplary embodiment of this application;

[0046] Figure 4 A flowchart illustrating a data reading method as shown in an exemplary embodiment of this application;

[0047] Figure 5 A flowchart illustrating a data reading method as another exemplary embodiment of this application;

[0048] Figure 6 This is a structural diagram of a data management system illustrated in an exemplary embodiment of this application;

[0049] Figure 7 This is a structural diagram of a data writing device illustrated in an exemplary embodiment of this application;

[0050] Figure 8 A structural diagram of a data writing apparatus shown in another exemplary embodiment of this application;

[0051] Figure 9 This is a structural diagram of a data reading device shown in an exemplary embodiment of this application;

[0052] Figure 10 A structural diagram of a data reading device shown in another exemplary embodiment of this application;

[0053] Figure 11 This is a block diagram of an electronic device illustrated in an exemplary embodiment of this application. Detailed Implementation

[0054] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0055] A computer system's storage system comprises several components, including hard drives, RAM, and cache. Hard drives offer large storage capacity, but reading data from them is relatively slow. RAM has a smaller storage capacity than hard drives, but its data access speed is faster. Cache has an even smaller storage capacity, but its data access speed is even faster than RAM.

[0056] Caching primarily utilizes a small amount of high-performance storage media to achieve high-performance access at a lower cost. When the processor reads data, it first checks the cache. If a cache hit occurs, the data can be retrieved directly from the cache. If a cache miss occurs, the data can then be read from memory. If a memory miss also occurs, the data can be read from the hard drive.

[0057] Traditional storage systems often use memory to store cached data. However, when a process restarts, the previously accumulated cached data will become invalid, leading to a large number of cache misses and significantly reducing system performance.

[0058] In the solution provided in this embodiment, cached data is managed based on a file management module to solve the problem of easy loss of cached data.

[0059] The method provided in this embodiment can be applied to computer systems, for example, such as... Figure 1 In the application scenario shown, when computer system 11 interacts with external device 12, it may trigger the computer system to read or write data. External device 12 can also be a computer system. For example, user operation of the computer system may also trigger it to read or write data; furthermore, the internal logic of the computer system may also trigger it to read or write data.

[0060] Figure 2 A flowchart illustrating a data writing method as an exemplary embodiment of this application.

[0061] like Figure 2 As shown, the data writing method provided in this embodiment includes:

[0062] Step 101: Obtain the data to be written. If the data to be written is high-frequency data, request management index information from the file management module and request storage space from the file management module.

[0063] The method provided in this embodiment can be applied to a computer system. When the computer system needs to write data, it can execute the method provided in this embodiment.

[0064] Specifically, a file management module can be set up in the computer system to record the correspondence between cached data information and index information. For example, a temporary file system (tmpfs) can be installed in the computer system as a file management module.

[0065] Furthermore, the file management module can also store cached data, specifically for storing data that the processor is more likely to read.

[0066] In practical applications, the method provided in this embodiment can store cached data in both the file management module and memory. The difference lies in that cached data stored in the file management module is more likely to be read by the processor and has a higher frequency of access, while cached data stored in memory is less so. The cached data stored in the file management module is actually also stored in memory, but it is managed by the file management module to solve the problem of data loss after process restart.

[0067] When data needs to be written, it can be determined whether the data is high-frequency data. If so, it is stored in the file management module. If the file management module's storage space is full, the data can also be written to memory. In another implementation, the data with higher frequency among the high-frequency data can be stored in the file management module. When the file management module's storage space is insufficient, the remaining high-frequency data can be stored in memory.

[0068] Specifically, the frequency of data access can be used as an evaluation metric to assess its popularity. For example, when the access frequency reaches a certain threshold, the data is considered to be highly popular.

[0069] Furthermore, if it is determined that the data to be written is high-frequency data, then management index information can be requested from the management module.

[0070] In practical applications, setting index information for cached data allows for better management of the cached data. In the method provided in this embodiment, if the data to be written is frequently accessed data, it needs to be written to the file management module. Therefore, index information for this data can be set in the file management module.

[0071] Users can also request storage space from the file management module to store data to be written. Specifically, the request can be made based on the amount of storage space required by the data to be written.

[0072] Specifically, after successfully applying for management of the referenced information and storage space, step 102 can be executed.

[0073] Step 102: Write the data to be written to the storage space and record the correspondence between the information of the data to be written and the management index information.

[0074] Furthermore, after obtaining storage space, the data to be written can be written to that storage space, thereby writing high-frequency data to the file management module.

[0075] In practical applications, it can also record the correspondence between the information of the data to be written and the management index information.

[0076] The file management module can record information about cached data and its corresponding index. If the cached data is stored in the file management module, its corresponding index information is the management index information.

[0077] Specifically, the information about the data to be written could be, for example, the storage address of the data. For instance, if the data is stored in the file management module, then the information about the data refers to its storage address within the file management module, which is the storage space allocated for high-frequency data.

[0078] Furthermore, cached data stored in the file management module can be stored only within that module and not further stored in memory. For example, data can be read from memory, processed, and then stored. If the data meets the requirements for writing to the file management module, it can be written to the file management module, and the corresponding data in memory can be deleted.

[0079] If data to be written is written to the file management module, when recording the correspondence between the data to be written and the management index information, the correspondence between the data index information and its storage address in the file management module can be recorded. Optionally, since the data is actually recorded in memory, the correspondence between the data index information and its storage address in memory can also be recorded. Through the index information, the storage address of the data in memory can be associated with its storage address in the file management module.

[0080] Specifically, cached data is written to the file management module, and the correspondence between the information of the data to be written and its corresponding index information is recorded for easy management. Furthermore, this data is stored in the file management module so that it is not lost when the process restarts.

[0081] The method provided in this embodiment is used to write data. This method is executed by a device equipped with the method provided in this embodiment. The device is typically implemented in hardware and / or software.

[0082] The data writing method provided in this embodiment includes: acquiring data to be written; if the data to be written is high-frequency data, requesting management index information from the file management module and requesting storage space from the file management module; writing the data to be written to the storage space, and recording the correspondence between the information of the data to be written and the management index information. The data writing method provided in this embodiment writes high-frequency data into the file management module, thereby utilizing the file management module to maintain cached data. This solves the problem in existing technologies where storing cached data in memory leads to easy loss of cached data after process restart.

[0083] Figure 3 A flowchart illustrating a data writing method as another exemplary embodiment of this application.

[0084] like Figure 3 As shown, the data writing method provided in this embodiment includes:

[0085] Step 201: Obtain the data to be written.

[0086] If the data to be written is high-frequency data, proceed to step 202. Otherwise, proceed to step 205.

[0087] Step 202: Request management index information from the file management module and request storage space from the file management module.

[0088] The implementation principle and method of obtaining the data to be written in step 201 are similar to those in step 101, and will not be repeated here.

[0089] The implementation principles and methods of applying for index information and storage space in step 202 are similar to those in step 101, and will not be repeated here.

[0090] If the application for managing index information and storage space is successful, proceed to step 203; otherwise, proceed to step 204.

[0091] Step 203: Write the data to be written to the storage space and record the correspondence between the information of the data to be written and the management index information.

[0092] Step 203 is similar in principle and method to step 102.

[0093] The information about the data to be written includes the storage location of the data in the file management module.

[0094] Specifically, the system records the index information of the data and its storage location in the file management module. When reading data, the system can quickly find the corresponding data based on this correspondence, thereby improving the reading speed.

[0095] When the processor reads data, it can send a data retrieval request to the file management module. The file management module can determine whether the data has been written to it based on the mapping between the storage address and the index information. If so, it can directly read the corresponding data from the file management module and send it back to the processor, achieving fast reading. If the data is not found in the file management module, it can be read from memory.

[0096] Step 204: Determine the memory index information corresponding to the data to be written in memory, and record the correspondence between the information of the data to be written and the memory index information.

[0097] In practical applications, if the request for index information or storage space from the file management module fails, the data to be written cannot be written to the file management module. In this case, it can be written to memory as cached data.

[0098] Specifically, a memory index can be determined in memory as the index information for the data to be written.

[0099] Specifically, it can also record cached data information stored in memory and its corresponding memory index information.

[0100] In this implementation, since the data is not stored in the file management module, the data information can be its storage location in memory. That is, the storage location in memory of the data to be written and its corresponding memory index information can be recorded. Specifically, this can be recorded in the file management module.

[0101] When reading data, the system can determine whether the data is stored in the file management module or in memory based on the data information recorded in the file management module and its corresponding index information, and then read the data from the appropriate location.

[0102] After storing data in memory, it can also be written to disk to prevent data loss. For example, when a user enters text in a document, this change can be stored in memory, and when the user clicks save or the system automatically saves the document, this content can be written to disk.

[0103] Step 205: Write the data to be written to the disk module.

[0104] If the data to be written is not high-frequency data, it is considered not to be cached data. Therefore, it can be written to the disk module to record the data.

[0105] Optionally, step 206 can be performed after step 203. Step 207 can be performed after step 204.

[0106] Step 206: Record the management index information of the data to be written into the LRU list.

[0107] Step 207: Record the memory index information into the LRU list.

[0108] Additionally, an LRU list can be maintained to record index information for cached data. LRU (LatestRecentUsed) means least recently used; if data has been accessed recently, it is more likely to be accessed in the future, so it can be added to the LRU list.

[0109] Generally, the length of an LRU list is limited, and therefore, the index information of cached data that can be recorded in it is also limited. If the index information that needs to be recorded exceeds the limited length of the LRU list, the index information of the lower-value records that will be accessed in the future can be removed. In the method provided in this embodiment, cached data can be evicted based on an LRU list.

[0110] Specifically, if the data to be written is written to the file management module, then the data to be written has management index information. At this time, the management index information of the data can be written to the LRU list.

[0111] Furthermore, if the data to be written is written into memory, then the data to be written has memory index information. In this case, the memory index information of the data can be written into the LRU list.

[0112] In practical applications, cached data can also be adjusted based on updates to the index information in the LRU list. For example, if index A is deleted due to the limited length of the LRU list, the corresponding cached data can be deleted from the file management module or memory, thereby freeing up storage space for this cached data.

[0113] Figure 4 This is a flowchart illustrating a data reading method as an exemplary embodiment of this application.

[0114] Step 301: Determine whether the data to be read is cached data based on the correspondence between the information of the data to be read and the index information.

[0115] The method provided in this embodiment can be applied to a computer system. When the computer system needs to read data, it can execute the method provided in this embodiment.

[0116] Specifically, a file management module can be set up in the computer system to record the correspondence between cached data information and index information. For example, a temporary file system (tmpfs) can be installed in the computer system as a file management module.

[0117] Furthermore, the file management module can also store cached data, specifically for storing data that the processor is more likely to read.

[0118] In practical applications, the method provided in this embodiment can store cached data in both the file management module and memory. The difference is that cached data stored in the file management module is more likely to be read by the processor and has a higher frequency of use, while cached data stored in memory is less likely to be read.

[0119] When data needs to be read, it can be determined whether the data is cached data based on the information of the data and the correspondence between the data information and the index information.

[0120] Specifically, if the mapping relationship stores information about the data to be read, then that data can be considered cached data; otherwise, it is considered not cached data. For example, the mapping relationship stores information about cached data A and its corresponding index 1, information about cached data B and its corresponding index 2, and information about cached data C and its corresponding index 3. When the data to be read is A, then the mapping relationship can be determined to contain information about the data to be read, therefore, that data is cached data.

[0121] If so, proceed to step 302.

[0122] Step 302: Based on the correspondence, determine whether the cached data is the first cached data stored in the file management module or the second cached data stored in memory.

[0123] Furthermore, in the method provided in this embodiment, the cached data can be stored in the file management module or in memory.

[0124] In practical applications, to enable faster retrieval of cached data, it's also possible to determine whether the cached data is stored in the file management module or in memory based on the corresponding relationships.

[0125] If the cached data is stored in the file management module, its corresponding index information can be management index information; if the cached data is stored in memory, its corresponding index information can be memory index information.

[0126] Specifically, the location of cached data storage can be determined based on the category of index information, whether it is stored in the file management module or in memory, and then the data can be read from the corresponding location.

[0127] For ease of distinction, the cached data stored in the file management module is referred to as the first cached data, and the cached data stored in memory is referred to as the second cached data.

[0128] In practical applications, if the cached data is stored in the file management module, the information in the mapping relationship can include the data's storage location in the file management module and its storage location in memory. These two addresses can be linked using index information.

[0129] The memory module can store cached data as well as regular data. When a piece of data becomes frequently accessed, it can be copied to the file management module. In this case, the corresponding data in memory can be deleted, thus writing the frequently accessed data to the file management module.

[0130] Specifically, if the mapping record shows the address of the data to be read in the file management module, then the data can be considered to be stored in the file management module. If the mapping record shows the memory storage address of the data to be read, then the data can be considered to be stored in memory. Optionally, since the data is actually recorded in memory, the mapping between the data index information and its memory storage address can also be recorded. In this case, whether the data is recorded in the file management module can be determined by whether the mapping record shows both storage addresses of the data to be read.

[0131] Step 303: Read the data to be read based on the determined result.

[0132] Furthermore, once the specific storage location of the cached data is determined, the data to be read can be retrieved from the corresponding location based on this determination.

[0133] In practical applications, the information in the mapping relationship for cached data can be the storage location of the cached data. For example, the starting position of cached data in memory, or the starting position of cached data in the file management module.

[0134] Specifically, when reading data, the data can be retrieved based on its storage location within the corresponding relationship, thereby further improving the reading speed.

[0135] Specifically, if the cached data is stored in the file management module, the mapping can record the storage address of that data in the file management module. In this case, the data can be read directly from the file management module.

[0136] Furthermore, if the cached data is stored in memory, the mapping can record the memory address of that data, i.e., it has a storage address. In this case, the data can be read directly from memory.

[0137] In practical applications, when the processor reads data, it can issue the storage address of that data. Specifically, the file management module can determine whether a corresponding address is recorded in the mapping table based on this storage address. If so, the data is considered cached data. At this point, it can further determine whether the data is stored in the file management module based on the mapping table. Specifically, it can determine the index information corresponding to the storage address and further confirm whether the data is stored in the file management module. If so, the data can be read directly from the file management module. Otherwise, the data can be read from memory.

[0138] The method provided in this embodiment is used to read data. This method is executed by a device equipped with the method provided in this embodiment, which is typically implemented in hardware and / or software.

[0139] The data reading method provided in this embodiment includes: determining whether the data to be read is cached data based on the correspondence between the information of the data to be read and the index information; if so, determining whether the cached data is first cached data stored in the file management module or second cached data stored in memory based on the correspondence; and reading the data to be read based on the determination result. In this embodiment, storing frequently accessed cached data in the file management module and maintaining the cached data using the file management module solves the problem in existing technologies where storing cached data in memory leads to easy loss of cached data after process restart.

[0140] Figure 5 A flowchart illustrating a data reading method as another exemplary embodiment of this application.

[0141] like Figure 5 As shown, the data reading method provided in this embodiment includes:

[0142] Step 401: Determine whether the data to be read is cached data based on the correspondence between the information of the data to be read and the index information.

[0143] The implementation principle and method of step 401 are similar to those of step 301, and will not be repeated here.

[0144] If yes, proceed to step 402. Otherwise, proceed to step 404.

[0145] Step 402: Based on the correspondence, determine whether the cached data is the first cached data stored in the file management module or the second cached data stored in memory.

[0146] The implementation principle and method of step 402 are similar to those of step 302, and will not be repeated here.

[0147] Step 403: If the data to be read is the first cached data, then read the data to be read from the file management module; if the data to be read is the second cached data, then read the data to be read from memory.

[0148] If it is determined that the cached data is the first cached data, that is, it is stored in the file management module, then the data can be read from the file management module.

[0149] Specifically, the first cached data can be read from the file management module based on the information of the data to be read in the corresponding relationship. In the method provided in this embodiment, the information of the data to be read may include the storage address of the data, specifically the storage address in memory, and may also include the storage address in the file management module.

[0150] Furthermore, index information can be set in the mapping relationship, which can be used to associate the storage address of the data in memory with its storage address in the file management module. Therefore, the data can be read from the file management module according to the storage address in the file management module based on the mapping relationship of the data to be read.

[0151] In practical applications, the processor can send the storage address of the data to be read to the file management module. The file management module then determines whether the corresponding data exists based on the mapping and the received storage address. If it does, the data is considered cached data. The file management module can further determine whether the data is stored within it. If so, the data is the first cached data, and the file management module can retrieve the corresponding data based on its storage address and send it to the processor.

[0152] Specifically, the first cached data can be read from the file management module based on the information of the data to be read in the corresponding relationship. In the method provided in this embodiment, the information of the data to be read may include the storage address of the data, specifically the storage address in the file management module.

[0153] In practical applications, the processor can send the storage address of the data to be read to the file management module. The file management module then determines whether the corresponding data exists based on the mapping relationship and the received storage address. If it does, the data is considered cached data. The file management module can further determine whether the data is stored in its own memory. If not, the data is considered secondary cached data, and the file management module can send a data retrieval request to the memory, causing the memory to send the data to be read to the processor.

[0154] In one alternative implementation, the processor may send the memory storage address of the data to the file management module. The file management module can then determine whether the data is cached based on the corresponding records. It can further determine whether the data is stored in the file management module or in memory.

[0155] Step 404: Read the data to be read from the disk.

[0156] If the file management module does not store the corresponding relationship for the data to be read, then the data can be considered not to be cached data and can be read from the disk.

[0157] Specifically, when the processor cannot find the relevant information of the data to be read in the file management module, it can send a request to the disk and then read the data from the disk.

[0158] Optionally, data read from the disk can be stored as cached data in the file management module or in memory.

[0159] Step 405: Increase the access frequency of the data to be read.

[0160] Furthermore, after reading the data to be read, which means that it is accessed again, the access frequency of the data to be read can be increased.

[0161] In practical applications, the access frequency of each piece of data can be recorded, such as the access frequency of each piece of data within a preset time period. Alternatively, the access frequency of only cached data can be recorded.

[0162] This allows you to set up an LRU list to record the index information of each cached data item, essentially recording the cached data information and the index information in the corresponding index relationship. You can also record the data access frequency corresponding to each index in the LRU list. The data access frequency in the LRU list can be updated.

[0163] Specifically, if the data read is not the first cached data, that is, not stored in the file management module, then step 406 can also be executed.

[0164] Step 406: Determine whether to write the data to be read into the file management module based on the access frequency.

[0165] Furthermore, the frequency of access after the data update can be used to determine whether to write the data to be read into the file management module.

[0166] For example, the current cached data can be sorted according to the updated access frequency. If the read data ranks high in the preset order, it can be added to the file management module. As another example, if the access frequency of updated data exceeds a preset threshold, the data can be written to the file management module.

[0167] In practical applications, if data is written to the file management module, the management index information of the data, as well as its storage location in the file management module, can also be recorded in the file management module.

[0168] Optionally, the method provided in this embodiment may further include:

[0169] Step 407: Update the index information in the LRU list.

[0170] In the method provided in this embodiment, an LRU list can also be set up, which can store the index information of each cached data.

[0171] Specifically, after data is read from the disk, it may be written to a cache, such as memory. At this point, an index for the data can be created, and the corresponding index information can be written to the LRU list.

[0172] Furthermore, the sorting of index information in the LRU list can be updated based on the access frequency after the data is read and updated. For example, if the index information in the LRU list is sorted according to the access frequency of cached data, then if the access frequency of one piece of data is updated, the sorting result may also be updated. That is, the method provided in this embodiment can update the index information included in the LRU list, and it can also update the sorting of the index information in the LRU list.

[0173] In practical applications, if the index information that needs to be recorded exceeds the limited length of the LRU list, the index information that is sorted to the end can be deleted, and the cached data corresponding to the index information can be deleted in memory or file management module at the same time.

[0174] Figure 6 This is a structural diagram of a data management system illustrated in an exemplary embodiment of this application.

[0175] like Figure 6 As shown, the data management system provided in this embodiment includes a file management module 51, a disk module 52, and a memory module 53.

[0176] The disk module 52 stores the raw data. A portion of this data may be stored in the memory module 53 as in-memory data.

[0177] Specifically, a portion of the data stored in memory or disk module 52 may also be stored as cache data in the file management module 51 and the memory module 53, which are used to store cache data.

[0178] When the processor reads data, it can first read from the cached data. If the query result is not found, that is, the data is not cached data, it can then read from the memory module 53. If the data is also not found in the memory module 53, it can then read from the disk module 52.

[0179] Furthermore, the cached data stored in the file management module 51 is the first cached data. The cached data stored in the memory module 53 is the second cached data.

[0180] The file management module 51 can also store the relationship between cached data and its corresponding index information (Nodes). The cached data includes first cached data and second cached data. Specifically, this mapping relationship can be stored in the Nodes.

[0181] In practical applications, the information for cached data can be its storage location. For example, the first cached data may have both a memory storage location and a storage location in the file management module; therefore, the information for the first cached data can include at least two storage locations. Similarly, the second cached data may have a memory storage location; therefore, its information can include at least one storage location.

[0182] Optionally, an LRU list can be set up in memory to manage the eviction of nodes containing index information for cached data. An in-memory map can quickly locate the position of the cached data's index information within the LRU list.

[0183] Specifically, some data in disk module 52 can be mapped to file management module 51, thereby storing this high-frequency data in file management module 51.

[0184] in, Figure 2-4 The illustrated embodiment can be applied to the data management system provided in this embodiment. This management system can be installed in a computer system, thereby enabling the computer to perform operations such as... Figure 2-4 Any one of the methods described.

[0185] Figure 7 This is a structural diagram of a data writing device illustrated in an exemplary embodiment of this application.

[0186] like Figure 7 As shown, the data writing device provided in this embodiment includes:

[0187] Module 61 is used to acquire the data to be written.

[0188] The application module 62 is used to apply to the file management module for management index information and to apply to the file management module for storage space if the data to be written is high-frequency data.

[0189] The writing module 63 is used to write the data to be written into the storage space and record the correspondence between the information of the data to be written and the management index information.

[0190] The data writing device provided in this embodiment includes: an acquisition module for acquiring data to be written; an application module for requesting management index information and storage space from a file management module if the data to be written is high-frequency data; and a writing module for writing the data to be written into the storage space and recording the correspondence between the information of the data to be written and the management index information. The data writing device provided in this embodiment writes high-frequency data into the file management module, thereby utilizing the file management module to maintain cached data. This solves the problem in existing technologies where cached data is stored in memory, leading to easy loss of cached data after process restart.

[0191] Figure 8 This is a structural diagram of a data writing device shown in another exemplary embodiment of this application.

[0192] like Figure 8 As shown, the data writing device provided in this embodiment, based on the above embodiment, includes the storage location of the data to be written in the file management module.

[0193] Optionally, if the application module 62 fails to apply for the management index information and / or fails to apply for the storage space, the writing module 63 is specifically used to determine the memory index information corresponding to the data to be written in the memory.

[0194] The device further includes a recording module 64, used for:

[0195] Record the correspondence between the information of the data to be written and the memory index information.

[0196] Optionally, if the data to be written is not high-frequency data, the writing module 63 is further configured to determine the memory index information corresponding to the data to be written in the memory;

[0197] The recording module 64 is also used to record the correspondence between the information of the data to be written and the memory index information.

[0198] Optionally, a first LRU module 65 is also included, for:

[0199] The management index information of the data to be written is recorded in the LRU list.

[0200] Optionally, a second LRU module 66 is also included for:

[0201] The memory index information of the data to be written is recorded in the LRU list.

[0202] Figure 9 This is a structural diagram of a data reading device illustrated in an exemplary embodiment of this application.

[0203] like Figure 9 As shown, the data reading device provided in this embodiment includes:

[0204] The first determining module 81 is used to determine whether the data to be read is cached data based on the correspondence between the information of the data to be read and the index information.

[0205] The second determining module 82 is used to determine, if yes, whether the cached data is the first cached data stored in the file management module or the second cached data stored in memory, based on the correspondence.

[0206] The reading module 83 is used to read the data to be read based on the determination result.

[0207] The data reading device provided in this embodiment includes: a first determining module, used to determine whether the data to be read is cached data based on the correspondence between the information of the data to be read and the index information; a second determining module, used to determine, if so, whether the cached data is first cached data stored in the file management module or second cached data stored in memory, based on the correspondence; and a reading module, used to read the data to be read based on the determining result. The data reading device provided in this embodiment stores frequently accessed cached data in the file management module and uses the file management module to maintain the cached data, which solves the problem in the prior art where storing cached data in memory leads to easy loss of cached data after process restart.

[0208] Figure 10 This is a structural diagram of a data reading device shown as another exemplary embodiment of this application.

[0209] like Figure 10 As shown, the data reading device provided in this embodiment, based on the above embodiment, optionally, if the data to be read is not cached data, the reading module 83 is also used to read the data to be read from the disk.

[0210] Optionally, the reading module 83 is specifically used for:

[0211] If the data to be read is first cached data, then the data to be read is read from the file management module;

[0212] If the data to be read is second cached data, then the data to be read is read from the memory.

[0213] Optionally, it also includes a frequency maintenance module 84, used to increase the access frequency of the data to be read;

[0214] If the data to be read is not the first cached data, the device further includes a writing module 85, used for:

[0215] The decision to write the data to be read into the file management module is determined based on the access frequency.

[0216] Optionally, an update module 86 is also included, for:

[0217] Update the index information of the cached data in the LRU list.

[0218] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0219] like Figure 11 The diagram shown is a block diagram of an electronic device according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0220] like Figure 11 As shown, the electronic device includes one or more processors 1001, a memory 1002, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 11 Take processor 1001 as an example.

[0221] The memory 1002 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the data writing method or data reading method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the data writing method or data reading method provided in this application.

[0222] Memory 1002, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the data writing method or data reading method in the embodiments of this application (e.g., attached...). Figure 7 The acquisition module 61, application module 62, and writing module 63 shown are illustrated in the appendix. Figure 9 The first determining module 81, the second determining module 82, and the reading module 83 are shown. The processor 1001 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 1002, that is, it implements the data writing method or data reading method in the above method embodiments.

[0223] The memory 1002 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1002 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1002 may optionally include memory remotely located relative to the processor 1001, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0224] The electronic device may also include an input device 1003 and an output device 1004. The processor 1001, memory 1002, input device 1003, and output device 1004 can be connected via a bus or other means. Figure 11 Taking the example of a connection between China and Israel via a bus.

[0225] Input device 1003 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1004 may include display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, liquid crystal display (LCD), light-emitting diode (LED) display, and plasma display. In some embodiments, the display device may be a touch screen.

[0226] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0227] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0228] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0229] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0230] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0231] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0232] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data writing method, characterized in that, include: Obtain the data to be written. If the data to be written is high-frequency data, request management index information from the file management module and request storage space from the file management module. The data to be written is written to the storage space, and the correspondence between the information of the data to be written and the management index information is recorded. The file management module is implemented based on the temporary file system tmpfs and is used to manage cached data so that the cached data will not be lost after the process restarts. If the application for the management index information fails, and / or the application for the storage space fails, then the memory index information corresponding to the data to be written is determined in memory; Record the correspondence between the information of the data to be written and the memory index information; If the data to be written is not high-frequency data, then the data to be written will be written to the disk module; The information of the data to be written includes the storage location of the data to be written in the file management module; Also includes: The management index information and the memory index information of the data to be written are recorded in the LRU list, and the cached data is adjusted according to the update of the index information in the LRU list.

2. A data reading method, characterized in that, include: Based on the correspondence between the information of the data to be read and the index information, determine whether the data to be read is cached data; If so, then based on the correspondence, it is determined whether the cached data is the first cached data stored in the file management module or the second cached data stored in memory. The file management module is implemented based on the temporary file system tmpfs and is used to manage the first cached data so that the first cached data will not be lost after the process restarts. Read the data to be read based on the determined result; If the data to be read is not cached data, then the data to be read is read from the disk; The step of reading the data to be read based on the determination result includes: If the data to be read is first cached data, then the data to be read is read from the file management module; If the data to be read is second cached data, then the data to be read is read from the memory; The information of the data to be read includes the storage location of the data to be read in the file management module; Also includes: Update the index information of the cached data in the LRU list, and adjust the cached data according to the update of the index information in the LRU list.

3. The method according to claim 2, characterized in that, Increase the access frequency of the data to be read; If the data to be read is not the first cached data, the method further includes: The decision to write the data to be read into the file management module is determined based on the access frequency.

4. A data management system, characterized in that, include: The file management module is used to record the relationship between cached data and its corresponding index information. The cached data includes first cached data stored in the file management module and second cached data stored in memory. The file management module is implemented based on the temporary file system tmpfs and is used to manage the first cached data so that the first cached data is not lost after the process restarts. The disk module is used to record raw data; The memory module is used to record the second cache data and the memory data; The information of the cached data includes the storage location of the cached data in the file management module; Also includes: The index information of the cached data is recorded in the LRU list, and the cached data is adjusted according to the update of the index information in the LRU list.

5. A data writing device, characterized in that, include: The acquisition module is used to acquire the data to be written. The application module is used to apply to the file management module for management index information and storage space if the data to be written is high-frequency data. The writing module is used to write the data to be written into the storage space and record the correspondence between the information of the data to be written and the management index information. The file management module is implemented based on the temporary file system tmpfs and is used to manage the cached data so that the cached data will not be lost after the process restarts. If the application module fails to apply for the management index information, and / or fails to apply for the storage space, the writing module is specifically used to determine the memory index information corresponding to the data to be written in memory; A recording module is used to record the correspondence between the information of the data to be written and the memory index information; If the data to be written is not high-frequency data, the writing module is also used to write the data to be written into the disk module; The information of the data to be written includes the storage location of the data to be written in the file management module; The first LRU module is used to record the management index information of the data to be written into the LRU list; The second LRU module is used to record the memory index information of the data to be written into the LRU list; The device is also used to adjust the cached data based on updates to the index information in the LRU list.

6. A data reading device, characterized in that, include: The first determining module is used to determine whether the data to be read is cached data based on the correspondence between the information of the data to be read and the index information. The second determining module is used to determine, if yes, whether the cached data is first cached data stored in the file management module or second cached data stored in memory, based on the correspondence relationship. The file management module is used to manage the first cached data so that the first cached data is not lost after the process restarts. The reading module is used to read the data to be read based on the determination result; If the data to be read is not cached data, the reading module is also used to read the data to be read from the disk; The reading module is specifically used for: If the data to be read is first cached data, then the data to be read is read from the file management module; If the data to be read is second cached data, then the data to be read is read from the memory; The information of the data to be read includes the storage location of the data to be read in the file management module; The update module is used to update the index information of cached data in the LRU list and adjust the cached data according to the update status of the index information in the LRU list.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 or 2-3.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method of any one of claims 1 or 2-3.

Images