Hash algorithm-based memory structure persistence method, device, equipment and medium

Abstract

The present disclosure provides a memory structure persistence method and device based on a hash algorithm, equipment and medium, relating to the technical field of data processing. The method comprises: dividing a memory structure HashMap based on a hash algorithm into a plurality of sub-HashMaps, each sub-HashMap corresponding to a file for saving key-value pairs; and using memory mapping to map the file corresponding to each sub-HashMap into a page cache memory. According to the embodiment of the present disclosure, the advantages of memory mapping can be fully utilized, and the data of the HashMap can be added to the page cache memory in real time, thereby effectively guaranteeing the security of the data.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, apparatus, device and medium for persistent memory structure based on a hash algorithm. Background Technology

[0002] For today's large-scale, high-performance computer systems, key-value (kv) data structures play an important role, with HashMap being a typical example, providing O(1) level query time complexity.

[0003] In some typical applications, the data in a HashMap cannot be lost and needs to be persistently stored. If the data in the HashMap changes frequently, it also needs to be continuously persisted, and the persistent data should be quickly restored to a HashMap upon system startup to support fast queries from upper-layer business logic.

[0004] In related technologies, Java's native serialization method can be used to serialize HashMap objects to disk. However, if there are a large number of key-value pairs in the HashMap, reaching tens of millions, this (deserialization) method is very slow, exceeding the requirements of general systems.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] This disclosure provides a method, apparatus, device, and medium for persistent memory structures based on hash algorithms, which at least to some extent overcomes the problem that the serialization method in related technologies is very slow when the number of key-value pairs in HashMap reaches tens of millions, exceeding the requirements of general systems.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to one aspect of this disclosure, a method for persisting a memory structure based on a hash algorithm is provided, comprising:

[0009] The HashMap, a memory structure based on a hash algorithm, is divided into multiple sub-HashMaps, each corresponding to a file used to store key-value pairs;

[0010] Memory mapping is used to map the file corresponding to each child HashMap to the page cache.

[0011] In one embodiment of this disclosure, the method further includes:

[0012] When adding a new key-value pair, the corresponding child HashMap is located using a hash algorithm based on the number of child HashMaps.

[0013] Add key-value pairs to the end of the memory-mapped file corresponding to the child HashMap;

[0014] Record the starting position of the key-value pairs in the memory-mapped file;

[0015] The starting position is used as the new value, and it is placed in the file corresponding to the child HashMap along with the original key.

[0016] In one embodiment of this disclosure, the method further includes:

[0017] When querying a key, the location of the memory-mapped file is retrieved from the child HashMap;

[0018] The value is obtained by parsing the location of the memory-mapped file.

[0019] In one embodiment of this disclosure, the method further includes:

[0020] During deserialization, key-value pairs are read concurrently from the memory-mapped file using multithreading;

[0021] Insert the read key-value pairs into the corresponding child HashMap.

[0022] In one embodiment of this disclosure, a HashMap is divided into multiple sub-HashMaps, including:

[0023] Predict the amount of data in the HashMap;

[0024] Plan the number of sub-HashMaps based on the amount of data in the HashMap;

[0025] Based on the number of child HashMaps, a HashMap is divided into child HashMaps.

[0026] According to another aspect of this disclosure, a memory structure persistence device based on a hash algorithm is provided, comprising:

[0027] The splitting module is used to divide the hash-based memory structure HashMap into multiple sub-HashMaps, each sub-HashMap corresponding to a file for storing key-value pairs;

[0028] The memory mapping module is used to map the file corresponding to each child HashMap to the page cache using memory mapping.

[0029] In one embodiment of this disclosure, the apparatus further includes:

[0030] The locator module is used to locate the corresponding child HashMap using a hash algorithm based on the number of child HashMaps when a new key-value pair is added.

[0031] The append module is used to append key-value pairs to the end of the memory-mapped file corresponding to the child HashMap;

[0032] The recording module is used to record the starting position of key-value pairs in the memory-mapped file;

[0033] The storage module is used to store the starting position as the new value and the original key in the file corresponding to the child HashMap.

[0034] In one embodiment of this disclosure, the apparatus further includes:

[0035] The query module is used to retrieve the location of the memory-mapped file from the child HashMap when querying a key;

[0036] The parsing module is used to parse the value from the location of the memory-mapped file.

[0037] In one embodiment of this disclosure, the apparatus further includes:

[0038] The key-value pair reading module is used to read key-value pairs from memory-mapped files concurrently using multiple threads during deserialization;

[0039] The key-value pair insertion module is used to insert the read key-value pairs into the corresponding child HashMap.

[0040] In one embodiment of this disclosure, the split module includes:

[0041] The data volume prediction unit is used to predict the data volume of a HashMap.

[0042] The quantity planning unit is used to plan the number of sub-HashMaps based on the amount of data in the HashMap;

[0043] Splitting Unit: Used to divide a HashMap into sub-HashMaps based on the number of sub-HashMaps.

[0044] According to another aspect of this disclosure, an electronic device is provided, comprising: a memory for storing instructions; and a processor for calling the instructions stored in the memory to implement the above-described hash algorithm-based memory structure persistence method.

[0045] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the above-described method for persistent memory structures based on a hash algorithm.

[0046] According to another aspect of this disclosure, a computer program product is provided, which stores instructions that, when executed by a computer, cause the computer to implement the above-described hash-based memory structure persistence method.

[0047] According to yet another aspect of this disclosure, a chip is provided, including at least one processor and an interface;

[0048] An interface is used to provide program instructions or data to at least one processor;

[0049] At least one processor is used to execute program instructions to implement the above-described hash-based memory structure persistence method.

[0050] The memory structure persistence method based on hash algorithm provided in this disclosure divides the hash algorithm-based memory structure HashMap into multiple sub-HashMaps, each sub-HashMap corresponding to a file for storing key-value pairs. Then, memory mapping is used to map the file corresponding to each sub-HashMap to the page cache. This disclosure fully utilizes the advantages of memory mapping to add HashMap data to the page cache in real time, effectively ensuring data security.

[0051] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0053] Obviously, the accompanying drawings described below are merely some embodiments of this disclosure. Those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0054] Figure 1 A flowchart of a memory structure persistence method based on a hash algorithm is shown in an embodiment of this disclosure;

[0055] Figure 2 This diagram illustrates a flowchart of the addition of a key-value pair in an embodiment of this disclosure;

[0056] Figure 3This document shows flowcharts illustrating the key-value pair addition, query, and update processes in embodiments of this disclosure.

[0057] Figure 4 This diagram illustrates an application scenario of an embodiment of the present disclosure.

[0058] Figure 5 A schematic diagram of a memory structure persistence device based on a hash algorithm is shown in an embodiment of this disclosure;

[0059] Figure 6 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0060] The exemplary implementation will now be described more fully with reference to the accompanying drawings.

[0061] It should be noted that the example implementation can be implemented in many forms and should not be construed as being limited to the examples set forth herein.

[0062] The following detailed description of this exemplary implementation method is provided in conjunction with the accompanying drawings and embodiments.

[0063] Figure 1 This invention discloses a flowchart of a memory structure persistence method based on a hash algorithm, as shown in the embodiment of the present invention. Figure 1 As shown, the memory structure persistence method based on hash algorithm provided in this embodiment includes steps S102-S104.

[0064] In S102, the hash-based memory structure HashMap is divided into multiple sub-HashMaps, and each sub-HashMap corresponds to a file used to store key-value pairs.

[0065] HashMap is an implementation of the Map interface based on a hash table. This implementation provides all optional map operations and allows null values ​​and null keys. It assumes that the hash function distributes elements appropriately across the buckets, providing stable performance for basic operations (get and put). The time required to iterate over a collection view is proportional to the "capacity" of the HashMap instance (the number of buckets) and its size (the number of key-value mappings).

[0066] In this embodiment of the disclosure, the HashMap is divided into several smaller HashMaps, namely sub-HashMaps; each smaller HashMap corresponds to a file used to store the key-value pairs inside.

[0067] In some embodiments, dividing a HashMap into multiple sub-HashMaps may include predicting the amount of data in the HashMap; planning the number of sub-HashMaps based on the amount of data in the HashMap; and dividing the HashMap into sub-HashMaps based on the number of sub-HashMaps.

[0068] In S104, memory mapping is used to map the file corresponding to each child HashMap to the page cache.

[0069] The page cache, also known as pcache or simply page cache, is one page in size, typically 4KB. In Linux, it caches the logical content of files during read / write operations, thereby speeding up access to the disk image and data.

[0070] For a HashMap containing tens of millions of key-value pairs, and whose values ​​are constantly changing, the embodiments of this disclosure can persist the changing values ​​in real time to ensure that the HashMap data is not lost when the system crashes, and can be recovered quickly when the system starts up.

[0071] In some embodiments, when adding a new key-value pair, such as Figure 2 As shown, it may include the following steps:

[0072] S201: When adding a new key-value pair, the corresponding child HashMap is located using a hash algorithm based on the number of child HashMaps.

[0073] S202, append the key-value pairs to the end of the memory-mapped file corresponding to the child HashMap;

[0074] S203, Record the starting position of the key-value pair in the memory-mapped file;

[0075] S204. The starting position is used as the new value, and the original key is placed in the file corresponding to the child HashMap.

[0076] In this embodiment of the disclosure, when putting (adding) a key-value pair, the corresponding sub-HashMap is located using a hash algorithm according to the number of sub-HashMaps; the key-value pair is appended to the end of the corresponding memory-mapped file and the starting position of the key-value pair is recorded; the starting position is used as the new value and put into the sub-HashMap along with the original key.

[0077] In some embodiments, the update process is similar to the put process described above.

[0078] In some embodiments, querying a key may include the following steps: when querying a key, retrieve the location of the memory-mapped file from the child HashMap; parse the value from the location of the memory-mapped file.

[0079] In some embodiments, deserialization may include the following steps: during deserialization, key-value pairs are read concurrently from a memory-mapped file using multiple threads; the read key-value pairs are then inserted into the corresponding child HashMap.

[0080] In this embodiment of the disclosure, the operating system allows an application to map a file to a block of memory for reading and writing. Compared with ordinary file reading and writing, this method has the following advantages: ordinary file reading and writing requires multiple copies until the data reaches the application's memory space before the application can read and write; while memory-mapped files require fewer copies, thus offering higher performance than ordinary file reading and writing.

[0081] Another point is that the mapped memory area is in the page cache, which is not affected by the upper-layer application. When the upper-layer application exits abnormally, as long as the system does not lose power, the dirty data in it will be completely preserved and will not be released or deleted by the operating system when the application exits. Therefore, the embodiments of this disclosure adopt the memory mapping method.

[0082] Splitting the original HashMap into several smaller HashMaps can improve the recovery speed: In order to make full use of the advantages of multi-core concurrency in modern computers, it is undoubtedly faster to use multiple threads to read the memory-mapped file simultaneously to recover the HashMap at system startup; although it is also possible to use multiple threads to read different segments of the same mapped file simultaneously, this method increases the difficulty of engineering implementation.

[0083] Figure 3 The diagram illustrates the flowcharts of the key-value pair apped, query, and update processes in an embodiment of this disclosure. Figure 4 A schematic diagram of an application scenario of this disclosure is shown below. Figure 3 and Figure 4 This disclosure describes a memory structure persistence method based on a hash algorithm provided in the embodiments of this disclosure.

[0084] like Figure 3 As shown in the embodiment of this disclosure, the HashMap is divided into several smaller HashMaps, each corresponding to a file used to store the key-value pairs within it. These smaller HashMaps can also be referred to as sub-HashMaps. Memory mapping is used to map each file into the page cache.

[0085] In some embodiments, the amount of data in the HashMap can be predicted, and the number of smaller HashMaps into which the HashMap can be split can be planned; a disk directory can be planned to store memory-mapped files; and each HashMap corresponds to one memory-mapped file.

[0086] When putting a key-value pair, the corresponding HashMap is located using a hash algorithm based on the number of HashMaps; the key and value are appended to the end of the corresponding memory-mapped file and the starting position of the key and value is recorded; this starting position is used as the new value and put into the HashMap along with the original key.

[0087] In some embodiments, a key-value pair is written, and a target HashMap is selected using a hash(key) operation. The key-value pair is then appended to the memory-mapped file corresponding to the target HashMap, and the position (pos) of the key-value pair is obtained. Finally, the key-pos is written to the HashMap. Upon system restart, the corresponding number of threads are started based on the number of memory-mapped files to concurrently read the memory-mapped files and restore the HashMap.

[0088] When querying a key, the location of the memory-mapped file is retrieved from the HashMap, and the value is parsed from that location in the memory-mapped file.

[0089] The update process is similar to the put process.

[0090] During deserialization, multiple threads can be used to concurrently read key-value pairs from the memory-mapped file and insert them into the corresponding HashMap.

[0091] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result.

[0092] In some embodiments, certain steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be broken down into multiple steps for execution.

[0093] Based on the same inventive concept, this disclosure also provides a memory structure persistence device based on a hash algorithm, as described in the following embodiments. Since the principle by which this device embodiment solves the problem is similar to that of the above method embodiments, the implementation of this device embodiment can refer to the implementation of the above method embodiments, and repeated details will not be elaborated further.

[0094] Figure 5This diagram illustrates a memory structure persistence device based on a hash algorithm according to an embodiment of the present disclosure. Figure 5 As shown, the hash algorithm-based memory structure persistence device 500 includes:

[0095] The splitting module 502 is used to divide the hash-based memory structure HashMap into multiple sub-HashMaps, each sub-HashMap corresponding to a file for storing key-value pairs;

[0096] The memory mapping module 504 is used to map the file corresponding to each child HashMap to the page cache using memory mapping.

[0097] In some embodiments, the hash-based memory structure persistence device 500 may further include:

[0098] The locator module is used to locate the corresponding child HashMap using a hash algorithm based on the number of child HashMaps when a new key-value pair is added.

[0099] The append module is used to append key-value pairs to the end of the memory-mapped file corresponding to the child HashMap;

[0100] The recording module is used to record the starting position of key-value pairs in the memory-mapped file;

[0101] The storage module is used to store the starting position as the new value and the original key in the file corresponding to the child HashMap.

[0102] In some embodiments, the hash-based memory structure persistence device 500 may further include:

[0103] The query module is used to retrieve the location of the memory-mapped file from the child HashMap when querying a key;

[0104] The parsing module is used to parse the value from the location of the memory-mapped file.

[0105] In some embodiments, the hash-based memory structure persistence device 500 may further include:

[0106] The key-value pair reading module is used to read key-value pairs from memory-mapped files concurrently using multiple threads during deserialization;

[0107] The key-value pair insertion module is used to insert the read key-value pairs into the corresponding child HashMap.

[0108] In some embodiments, the split module 502 includes:

[0109] The data volume prediction unit is used to predict the data volume of a HashMap.

[0110] The quantity planning unit is used to plan the number of sub-HashMaps based on the amount of data in the HashMap;

[0111] Splitting Unit: Used to divide a HashMap into sub-HashMaps based on the number of sub-HashMaps.

[0112] The concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to define the order of functions performed by these devices, modules or units or their interdependencies.

[0113] Regarding the hash algorithm-based memory structure persistence device in the above embodiments, the specific methods by which each module performs operations have been described in detail in the embodiments of the hash algorithm-based memory structure persistence method, and will not be elaborated here.

[0114] In summary, the memory structure persistence device based on hash algorithm provided in this application embodiment can persist the changing value in real time to ensure that the HashMap data is not lost when the system crashes, and can be recovered quickly when the system starts up.

[0115] It should be noted that although several modules or units of the device used for action execution are mentioned in the detailed description above, this division is not mandatory.

[0116] In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0117] Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0118] The following reference Figure 6 This describes the electronic device provided in the embodiments of this disclosure. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0119] Figure 6This diagram illustrates the architecture of an electronic device 600 according to an embodiment of the present invention. Figure 6 As shown, the electronic device 600 includes, but is not limited to, at least one processor 610 and at least one memory 620.

[0120] Memory 620 is used to store instructions.

[0121] In some embodiments, memory 620 may include a readable medium in the form of volatile memory cells, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include read-only memory (ROM) 6203.

[0122] In some embodiments, the memory 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0123] In some embodiments, memory 620 may store an operating system. This operating system may be a real-time operating system (RTX), such as Linux, UNIX, Windows, or OS X.

[0124] In some embodiments, the memory 620 may also store data.

[0125] As an example, processor 610 can read data stored in memory 620, which may be stored at the same memory address as the instructions, or the data may be stored at a different memory address than the instructions.

[0126] Processor 610 is configured to invoke instructions stored in memory 620 to implement the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure. For example, processor 610 may execute the following steps of the above method embodiments:

[0127] The HashMap, a memory structure based on a hash algorithm, is divided into multiple sub-HashMaps, each corresponding to a file used to store key-value pairs;

[0128] Memory mapping is used to map the file corresponding to each child HashMap to the page cache.

[0129] It should be noted that the processor 610 described above can be a general-purpose processor or a special-purpose processor. The processor 610 may include one or more processing cores, and the processor 610 executes various functional applications and data processing by running instructions.

[0130] In some embodiments, processor 610 may include a central processing unit (CPU) and / or a baseband processor.

[0131] In some embodiments, the processor 610 may determine an instruction based on the priority identifier and / or function category information carried in each control instruction.

[0132] In this disclosure, the processor 610 and the memory 620 can be configured separately or integrated together.

[0133] As an example, the processor 610 and memory 620 can be integrated on a single board or a system-on-a-chip (SOC).

[0134] like Figure 6 As shown, electronic device 600 is embodied in the form of a general-purpose computing device. Electronic device 600 may also include bus 630.

[0135] Bus 630 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0136] Electronic device 600 can also communicate with one or more external devices 640 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). Such communication can be performed through input / output (I / O) interface 650.

[0137] Furthermore, the electronic device 600 can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via the network adapter 660.

[0138] like Figure 6 As shown, network adapter 660 communicates with other modules of electronic device 600 via bus 630.

[0139] It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0140] It is understood that the structures illustrated in the embodiments of this disclosure do not constitute a specific limitation on the electronic device 600. In other embodiments of this disclosure, the electronic device 600 may include more than Figure 6 This may involve more or fewer components, or combining certain components, or splitting certain components, or different component arrangements. Figure 6 The components shown can be implemented in hardware, software, or a combination of both.

[0141] This disclosure also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the memory structure persistence method based on a hash algorithm described in the above method embodiments.

[0142] In this embodiment of the disclosure, the computer-readable storage medium is a computer instruction that can be sent, propagated, or transmitted for use by or in conjunction with an instruction execution system, apparatus, or device.

[0143] As an example, a computer-readable storage medium is a non-volatile storage medium.

[0144] In some embodiments, more specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, USB flash drives, portable hard drives, or any suitable combination of the foregoing.

[0145] In this embodiment of the disclosure, the computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, wherein computer instructions (readable program code) are carried.

[0146] The transmitted data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.

[0147] In some examples, computational instructions contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0148] This disclosure also provides a computer program product that stores instructions that, when executed by a computer, cause the computer to implement the hash algorithm-based memory structure persistence method described in the above method embodiments.

[0149] The aforementioned instructions can be program code. In practice, the program code can be written using any combination of one or more programming languages.

[0150] Programming languages ​​include object-oriented programming languages—such as Java and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages.

[0151] The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0152] In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0153] This disclosure also provides a chip, including at least one processor and an interface;

[0154] An interface is used to provide program instructions or data to at least one processor;

[0155] At least one processor is used to execute program instructions to implement the hash algorithm-based memory structure persistence method described in the above method embodiments.

[0156] In some embodiments, the chip may further include a memory for storing program instructions and data, the memory being located within or outside the processor.

[0157] Those skilled in the art will understand that all or part of the steps of the above embodiments can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which can be collectively referred to as "circuit", "module" or "system".

[0158] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein.

[0159] This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A hash algorithm based in-memory structure persistence method, characterized in that, include: The HashMap, a memory structure based on a hash algorithm, is divided into multiple sub-HashMaps, each corresponding to a file used to store key-value pairs; Memory mapping is used to map the file corresponding to each child HashMap to the page cache memory; When adding a new key-value pair, the corresponding child HashMap is located using a hash algorithm based on the number of child HashMaps. Add the key-value pairs to the end of the memory-mapped file corresponding to the sub-HashMap; Record the starting position of the key-value pair in the memory-mapped file; The starting position is used as the new value, and the original key is placed in the file corresponding to the child HashMap; When querying a key, the location of the memory-mapped file is retrieved from the child HashMap; the value is then parsed from the location of the memory-mapped file. When the system restarts, the corresponding number of threads are started based on the number of memory-mapped files, and the memory-mapped files are read concurrently to restore the HashMap.

2. The method according to claim 1, characterized in that, The method further includes: During deserialization, key-value pairs are read concurrently from the memory-mapped file using multithreading; The read key-value pairs are then inserted into the corresponding child HashMap.

3. The method according to claim 1, characterized in that, The process of dividing a HashMap into multiple sub-HashMaps includes: Predict the amount of data in the HashMap; Based on the amount of data in the HashMap, plan the number of sub-HashMaps; The HashMap is divided into sub-HashMaps based on the number of sub-HashMaps.

4. A memory structure persistence device based on a hash algorithm, characterized in that, include: The splitting module is used to divide the hash-based memory structure HashMap into multiple sub-HashMaps, each sub-HashMap corresponding to a file for storing key-value pairs; The memory mapping module is used to map the file corresponding to each child HashMap to the page cache using memory mapping. The locator module is used to locate the corresponding child HashMap using a hash algorithm based on the number of child HashMaps when a new key-value pair is added. The supplement module is used to supplement key-value pairs to the end of the memory-mapped file corresponding to the sub-HashMap; A recording module is used to record the starting position of the key-value pair in the memory-mapped file; The storage module is used to put the starting position as the new value and the original key into the file corresponding to the sub-HashMap; The query module is used to retrieve the location of the memory-mapped file from the child HashMap when querying a key; The parsing module is used to parse the value from the location of the memory-mapped file.

5. An electronic device, characterized in that, include: Memory, used to store instructions; A processor is configured to invoke instructions stored in the memory to implement the memory structure persistence method based on a hash algorithm as described in any one of claims 1-3.

6. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement the memory structure persistence method based on the hash algorithm as described in any one of claims 1-3.

Images