Data storage method, system, device and medium based on multi-path merging and multi-level cache

Abstract

The application provides a data storage method, system, device and medium based on multi-path merging and multi-level caching, comprising: writing the sorted first-level cache data which does not exceed a first preset capacity into a second-level cache file; using a multi-path merging algorithm, writing a plurality of second-level cache files which do not exceed a second preset capacity into a third-level cache file, and writing the sorted and merged plurality of third-level cache files into a fourth-level cache file. The application uses a multi-level caching method, for example, storing data of 64M or less than 64M in size into a second-level cache file under the ins directory, merging a plurality of second-level cache files into an mrg file, and finally writing a plurality of mrg files. Therefore, in practice, the application can realize writing a file of up to 20G (i.e. 10 mrg files of 2G size) at a time, which greatly improves the storage efficiency compared with writing a file of 64M size at a time in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, and particularly relates to a data storage method, system, device and medium based on multi-path merging and multi-level caching. BACKGROUND

[0002] In the era of big data, massive data also brings certain challenges and opportunities, for example, the storage cost pressure brought by huge data volume, the need for different storage support due to the diversification of data (such as logs / pictures / videos / documents...), the need for fast and effective analysis capability due to the increasing speed of data generation, and the like.

[0003] Taking massive data storage as an example, the prior art usually directly inserts data into the memory storage space. This way is not a problem for a small amount of data, but as the data volume increases exponentially, thousands of data are stored by using the traditional technology, which undoubtedly greatly increases the storage time and cost. Therefore, there is an urgent need in the art for a more low-cost and effective data storage scheme. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a data storage method, system, device and medium based on multi-path merging and multi-level caching, which is used to solve the technical problem that the prior art cannot be applied to massive data storage.

[0005] To achieve the above-mentioned purpose and other related purposes, a first aspect of the present application provides a data storage method based on multi-path merging and multi-level caching, comprising: writing the sorted first-level cache data which does not exceed a first preset capacity into a second-level cache file; using a multi-path merging algorithm, writing a plurality of second-level cache files which do not exceed a second preset capacity into a third-level cache file, and writing the sorted and merged plurality of third-level cache files into a fourth-level cache file.

[0006] In some embodiments of the first aspect of the present application, the second-level cache file is a sorted dat file located in the ins directory; the dat file is limited to storing data with a maximum size of 64M.

[0007] In some embodiments of the first aspect of the present application, the first preset capacity is set to 64M, so as to write data with a size of 64M or less than 64M into the dat file.

[0008] In some embodiments of the first aspect of the present application, the writing of the plurality of second-level cache files with a capacity not exceeding a second preset capacity into a third-level cache file comprises: if sizes of the plurality of second-level cache files to be written currently do not exceed the second preset capacity, then writing all the second-level cache files into the third-level cache file; if the sizes of the plurality of second-level cache files to be written currently exceed the second preset capacity, then writing a plurality of second-level cache files with a size of the second preset capacity into the third-level cache file.

[0009] In some embodiments of the first aspect of the present application, the multi-way merge algorithm comprises a loop traversal method, a minimum heap k-way merge sorting method or a winner number k-way merge sorting method.

[0010] In some embodiments of the first aspect of the present application, the third-level cache file is an idfs file located in an mrg directory, and the fourth-level cache file is a permanently stored idfs file.

[0011] To achieve the above object and other related objects, the second aspect of the present application provides a data storage system based on multi-way merge and multi-level cache, comprising: a second-level cache module configured to write data with a capacity not exceeding a first preset capacity into second-level cache files arranged in sequence; and a multi-level cache module configured to write a plurality of second-level cache files with a capacity not exceeding a second preset capacity into a third-level cache file by using a multi-way merge algorithm, and write a plurality of third-level cache files after sorting and merging into a fourth-level cache file.

[0012] To achieve the above object and other related objects, the third aspect of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the data storage method based on multi-way merge and multi-level cache.

[0013] To achieve the above object and other related objects, the fourth aspect of the present application provides a computer device, comprising: a processor and a memory; the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, so that the device executes the data storage method based on multi-way merge and multi-level cache.

[0014] As described above, the data storage method, system, device and medium based on multi-path merging and multi-level caching of the present application have the following beneficial effects: the prior art needs to write these dat files one by one when facing tens of thousands of 64M size dat files, and the total storage content will be updated after each data insertion, which will result in huge resource waste and greatly increase the storage time, which is extremely inefficient and inconvenient. Therefore, the present application uses a multi-level caching method, for example, data of 64M or less than 64M size is first stored in the secondary cache file under the ins directory, then several secondary cache files are merged into the mrg file, and finally several mrg files are written together. Therefore, in fact, the present application can realize one-time writing of a file with a size of up to 20G (i.e. 10 2G size mrg files), which greatly improves the storage efficiency compared with the prior art of writing 64M size files one by one. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The flowchart shown is a data storage method based on multi-path merging and multi-level caching in an embodiment of the present application.

[0016] Figure 2 The schematic diagram shows the insertion of image data in an embodiment of the present application.

[0017] Figure 3 The structural schematic diagram of the data storage system based on multi-path merging and multi-level caching in an embodiment of the present application is shown.

[0018] Figure 4 The structural schematic diagram of the computer device in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] The embodiments of the present application will be described below through specific concrete examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the present specification. The present application can also be implemented or applied through other different specific embodiments, and each detail in the present specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0020] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting, unless otherwise specifically indicated. Spatially relative terms, such as "upper," "lower," "left," "right," "below," "below," "bottom," "top," and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be further understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures. For example, if a device described is turned over in use, a fixed surface can be up

[0021] In this application, unless otherwise specified and limited, the terms "mounting", "connecting", "connection", "fixing", "holding" and the like are to be broadly understood, for example, can be fixed connection, can also be detachable connection, or integrally connected; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in this application can be understood according to the specific circumstances.

[0022] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "first", "second", "third", "fourth" and the like in the description and in the claims, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms "first", "second", "third", "fourth", and the like, if any, in the description and in the claims can change according to the context. Furthermore, the terms "comprise", "comprising", "include", "including", and the like, mean and encompass the presence of stated features, operations, elements, components, items, and / or groups thereof, but do not preclude the presence or addition of one or more other features, operations, elements, components, items, and / or groups thereof. It is further understood that the use of the terms "or", "and / or", as used herein, is to be interpreted as inclusive, i.e., the usage of "or" allows for the potential existence of one or more items, and the usage of "and / or" allows for the potential existence of one or more items as well as the potential absence of one or more items. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Only when the combination of elements, functions, or operations are inherently mutually exclusive is an exception to this definition.

[0023] To address the limitations of existing technologies in storing massive amounts of data and achieving a low-cost, efficient data storage solution, this invention provides a multi-way merging and multi-level caching solution. Data of 64MB or less is first stored in a secondary cache file under the `ins` directory. Several secondary cache files are then merged into an `mrg` file, and finally, all `mrg` files are written together. Therefore, this invention can effectively write files up to 20GB (i.e., ten 2GB `mrg` files) at once, significantly improving storage efficiency compared to writing 64MB files one at a time in existing technologies.

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the invention.

[0025] like Figure 1 The diagram illustrates a flowchart of a data storage method based on multi-way merging and multi-level caching in one embodiment of the present invention.

[0026] It should be noted that the data storage method based on multi-way merging and multi-level caching described above can be applied to controllers, such as ARM (Advanced RISC Machines) controllers, FPGA (Field Programmable Gate Array) controllers, SoC (System on Chip) controllers, DSP (Digital Signal Processing) controllers, or MCU (Microcontroller Unit) controllers; it can also be applied to computer devices including components such as memory, memory controllers, one or more processing units (CPUs), peripheral interfaces, RF circuits, audio circuits, speakers, microphones, input / output (I / O) subsystems, displays, other output or control devices, and external ports, such as desktop computers, laptops, tablets, smartphones, smart bracelets, smartwatches, smart helmets, and smart TVs; it can also be applied to servers, which can be deployed on one or more physical servers according to various factors such as function and load, or can be composed of distributed or centralized server clusters.

[0027] In this embodiment, the data storage method based on multi-way merging and multi-level caching mainly includes steps S11 and S12, which will be explained in detail below.

[0028] Step S11: Sort the first-level cache data, which does not exceed the first preset capacity, and write it to the second-level cache file.

[0029] It should be noted that the so-called multi-level cache can include a level cache, a second level cache, a third level cache and a fourth level cache. The first level cache is the memory; the second level cache is the ins cache; the third level cache is the mrg cache; and the fourth level cache is the permanent storage. The second level cache file refers to the dat file in the ins directory, the third level cache file refers to the idfs file in the mrg directory, and the fourth level cache file refers to the idfs file in the permanent storage dat folder.

[0030] Taking the dat file as an example of the second level cache file, each dat file is set to a file with a maximum storage capacity of 64M in this embodiment, and each ins file stores up to 64 dat files, so the first preset capacity is 64M. Data with a size of 64M or less than 64M can be written into the dat file. It should be noted that the dat file itself has no capacity size limit, and the present application limits it to store up to 64M of data, in order to adapt to the memory with a maximum cache of 64M, and to avoid affecting the program running or occupying too much memory. It should be understood that the bottom layer stores {Id XMath.XInt, Data[]byte} in the present application, Id is a large integer, and the length of Data is 2 bytes encoding, 1 byte occupies 1 byte, so the value range of byte is represented in binary as: 0000 0000-1111 1111, which is 256 numbers, so the size of Data will not exceed 256*256=64k.

[0031] S12: using a multi-way merge algorithm, writing a plurality of second level cache files with a capacity not exceeding a second preset capacity into a third level cache file, and sorting and merging a plurality of third level cache files to write into a fourth level cache file.

[0032] Specifically, the writing of a plurality of second level cache files with a capacity not exceeding a second preset capacity into a third level cache file includes: if the size of the plurality of second level cache files to be written currently does not exceed the second preset capacity, then writing all second level cache files into the third level cache file; if the size of the plurality of second level cache files to be written currently exceeds the second preset capacity, then writing a plurality of second level cache files with a size of the second preset capacity into the third level cache file.

[0033] For example, the preset storage capacity can be set to 2G, i.e. the storage capacity of each mrg file is 2G. If the capacity of the second level cache dat file under the ins directory is less than 2G, for example, the user input data is not much, and there is only one dat file in 2 hours, then the dat file needs to be merged into the mrg file. If the capacity of the second level cache file under the ins directory is greater than 2G, then the corresponding number of dat files with a size of 2G are merged into the mrg file at most.

[0034] It should be noted that the first preset capacity and the second preset capacity in the present example are preset values, and the setting of the first preset capacity as 64M and the setting of the second preset capacity as 2G in the above embodiment are only one implementation of the present application, and are not a limitation on the preset values themselves.

[0035] Further, after storing a plurality of mrg files, the plurality of mrg heap files can be sorted and merged into a large heap file, which is then written into a permanent storage file. For example, there are 10 mrg files, each with a capacity of 2G, and the 10 mrg files are sorted into a large heap file, which is written into a permanent storage file using a multi-way merge algorithm.

[0036] It should be noted that the multi-way merge algorithm is the basis of external sorting, and currently commonly used multi-way merge algorithms include a loop traversal method, a minimum heap k-way merge sorting method, and a winner tree k-way merge sorting method. The loop traversal method refers to comparing the first elements of all k arrays, finding the smallest one and taking it out, taking the next element in the array where the smallest element is located, and repeating the previous process to take out the smallest element, and repeating this process until all elements are found. The minimum heap k-way merge sorting method refers to first taking one element from each of the k sequences, all of which are already sorted in ascending order and do not need to be considered further, and the element taken from each sequence is the smallest in that sequence. The winner tree k-way merge sorting method refers to first constructing a winner tree, with almost every leaf node of the heap corresponding to an input sequence to form a complete binary tree, and after one round of winner selection, the element at the top is the smallest and there is a path from the leaf node to the root of the heap. By taking subsequent elements from the original sequence and adjusting them upwards like the winner tree, the elements that meet the conditions are placed on top, and then the root is compared, and the next smallest element is found. This process is repeated.

[0037] For ease of understanding, the following will be described in conjunction with Figure 2For example, in this embodiment, the data to be inserted is an image Img, and a schema file is provided under a large directory (data directory). The schema is the organization and structure of the database, and the schema includes schema objects, such as tables, columns, data types, views, stored procedures, relationships, primary keys, foreign keys, and the like. In this example, the schema file stores all the tables that can be stored in this server. From the schema, fields such as ID Bytes, Isindex, and Name can be learned to find the corresponding folder. For example, the image Img is defined as {ID Bytes: 20, Isindex: False, Name: Img}, i.e., the ID number occupies 20 bytes, is not indexed, and the name is "Img". Thus, a folder with the name Img is generated, which has an ID Bytes of 20 and is not indexed. The folder has three files, namely, an ins file, an mrg file, and a dat file.

[0038] Therefore, the process of data insertion is as follows:

[0039] First, data less than or equal to 64M is written from the memory to the dat file, and the dat files are stored in the ins file one by one. Each ins file stores up to 64 dat files.

[0040] Second, as indicated by the direction of arrow ①, a plurality of dat files under the ins directory with a maximum capacity of 2G are merged into an mrg file at a time using a multi-way merge algorithm. Each mrg file has a capacity of 2G.

[0041] Finally, as indicated by the direction of arrow ②, up to 10 mrg files are merged into a large heap file and then written to the idfs file under the permanent storage dat folder using a multi-way merge algorithm.

[0042] It is worth noting that if the multi-level cache is not used with the multi-way merge algorithm, when facing tens of thousands of 64M size dat files, these dat files need to be written one by one, and the total storage content will be updated after each data insertion, which will cause huge waste of resources and greatly increase the storage time, which is extremely inefficient and inconvenient. Therefore, the present application uses a multi-level cache method, for example, data of 64M or less than 64M size is first stored in the secondary cache file under the ins directory, and then a plurality of secondary cache files are merged into the mrg file, and finally a plurality of mrg files are written together. Therefore, in fact, the present application can realize one-time writing of a file with a size of up to 20G (i.e. 10 mrg files of 2G size), which greatly improves the storage efficiency compared with the prior art of writing 64M size files one by one.

[0043] As shown in Figure 3 The multi-way merge and multi-level cache-based data storage system 300 includes a secondary cache module 301 and a multi-level cache module 302.

[0044] The secondary cache module 301 is used to sort and write the first-level cache data not exceeding the first preset capacity into the secondary cache file; the multi-level cache module 302 is used to use the multi-way merge algorithm to write a plurality of secondary cache files not exceeding the second preset capacity into the third-level cache file, and sort and merge a plurality of the third-level cache files to write into the fourth-level cache file.

[0045] In some examples, the first-level cache is the memory; the second-level cache is the ins cache; the third-level cache is the mrg cache; and the fourth-level cache is the permanent storage. The secondary cache file is a sorted dat file under the ins directory; the dat file is limited to storing data with a maximum size of 64M; and the first preset capacity is set to 64M. Therefore, the secondary cache module 301 writes data with a size of 64M or less than 64M into the dat file.

[0046] In some examples, when the multi-level cache module 302 writes a plurality of secondary cache files not exceeding the second preset capacity into the third-level cache file, it needs to distinguish: if the size of the plurality of secondary cache files to be written currently does not exceed the second preset capacity, all secondary cache files are written into the third-level cache file; if the size of the plurality of secondary cache files to be written currently exceeds the second preset capacity, a plurality of secondary cache files with a size of the second preset capacity are written into the third-level cache file.

[0047] It should be noted that the data storage system based on multi-way merging and multi-level caching provided in the embodiment has similar implementation as the data storage method based on multi-way merging and multi-level caching, and thus will not be described again.

[0048] It should be understood that the division of each module of the above system is only a logical division of functions, and in actual implementation, all or part of the modules can be integrated into one physical entity or physically separated. The modules can be all implemented in the form of software called by a processing element, all implemented in the form of hardware, or part of the modules implemented in the form of software called by a processing element and part of the modules implemented in the form of hardware. For example, the secondary cache module can be a separately set processing element, or can be integrated in a chip of the above system, or can be stored in the form of program code in a memory of the above system and called and executed by a processing element of the above system to implement the function of the secondary cache module. The implementation of other modules is similar. In addition, all or part of the modules can be integrated together or independently implemented. The processing element described herein can be an integrated circuit having a signal processing capability. In the implementation process, each step of the above method or each module can be completed by an integrated logic circuit of hardware in the processing element or an instruction in the form of software.

[0049] For example, the above modules can be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of program code called by a processing element, the processing element can be a general-purpose processor such as a central processing unit (CPU) or other processor capable of calling program code. For another example, the modules can be integrated together to implement in the form of a system on a chip (SOC).

[0050] For example, Figure 4As shown, a structural schematic diagram of a computer device in an embodiment of the present application is shown. The computer device provided in the example includes a processor 41, a memory 42, and a communicator 43. The memory 42 is connected with the processor 41 and the communicator 43 through a system bus and completes communication therebetween. The memory 42 is configured to store a computer program. The communicator 43 is configured to communicate with other devices. The processor 41 is configured to run the computer program, so that the computer device performs each step of the data storage method based on multi-way merging and multi-level caching as described above.

[0051] The system bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus. The communication interface is configured to realize communication between the database access device and other devices (for example, a client, a read-write library, and a read-only library). The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, for example, at least one disk memory.

[0052] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. The processor can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0053] The present application also provides a computer-readable storage medium having a computer program stored thereon. The computer program is executed by a processor to implement the data storage method based on multi-way merging and multi-level caching.

[0054] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by computer program related hardware. The aforementioned computer program can be stored in a computer readable storage medium. The program executes the steps of the above-mentioned method embodiments when executed; and the aforementioned storage medium includes ROM, RAM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, flash memory, U disk, mobile hard disk, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. In addition, any connection can be appropriately referred to as a computer readable medium. For example, if instructions are sent from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave is included in the definition of the medium. However, it should be understood that the computer readable storage medium and the data storage medium do not include connections, carriers, signals or other transitory media, but are intended to be directed to non-transitory, tangible storage media. As used in the application, magnetic disks and optical disks include compact disks (CD), laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, wherein magnetic disks typically magnetically copy data, and optical disks optically copy data with a laser.

[0055] In the embodiments provided in the present application, the computer readable storage medium can include read-only memory, random access memory, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, flash memory, U disk, mobile hard disk, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. In addition, any connection can be appropriately referred to as a computer readable medium. For example, if instructions are sent from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave is included in the definition of the medium. However, it should be understood that the computer readable storage medium and the data storage medium do not include connections, carriers, signals or other transitory media, but are intended to be directed to non-transitory, tangible storage media. As used in the application, magnetic disks and optical disks include compact disks (CD), laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, wherein magnetic disks typically magnetically copy data, and optical disks optically copy data with a laser.

[0056] In summary, the present application provides a data storage method, system, device and medium based on multi-path merging and multi-level caching. Since the prior art needs to write these dat files one by one when facing tens of thousands of 64M size dat files, and the total storage content will be updated after each data insertion, which will result in huge resource waste and greatly increase the storage time, it is extremely inefficient and inconvenient. Therefore, the present application uses a multi-level caching method, for example, data of 64M or less than 64M size is first stored in the second cache file under the ins directory, then a plurality of second cache files are merged into the mrg file, and finally a plurality of mrg files are written together. Therefore, in fact, the present application can realize one-time writing of a file with a size of up to 20G (i.e. 10 mrg files with a size of 2G), which greatly improves the storage efficiency compared with the prior art of one-time writing of a file with a size of 64M. Therefore, the present application effectively overcomes the various shortcomings of the prior art and has high industrial utilization value.

[0057] The above embodiments are only illustrative of the principles of the present application and its effects, and are not intended to limit the present application. Any modification or change made by any person skilled in the art without departing from the spirit and scope of the present application shall be covered by the claims of the present application.

Claims

1. A data storage method based on multiway merging and multistage buffering, characterized by, The method comprises the following steps: sequencing and writing first cache data not exceeding a first preset capacity into a second cache file; using a multi-way merge algorithm, writing a plurality of second cache files not exceeding a second preset capacity into a third cache file, and sequencing and merging a plurality of third cache files to write into a fourth cache file; the first cache is a memory; the second cache is an ins cache; the third cache is an mrg cache; and the fourth cache is a permanent storage; the second cache file is a sequenced dat file under an ins directory; and the dat file is limited to storing data with a maximum size of 64M; the first preset capacity is set to 64M, so as to write data with a size of 64M or less than 64M into the dat file; and the second preset capacity is 2G.

2. The data storage method based on multi-path merging and multi-level caching according to claim 1, characterized in that, The step of writing a plurality of second cache files not exceeding the second preset capacity into a third cache file comprises the following steps: if sizes of a plurality of second cache files to be written currently do not exceed the second preset capacity, then all the second cache files are written into the third cache file; if sizes of a plurality of second cache files to be written currently exceed the second preset capacity, then a plurality of second cache files with a size of the second preset capacity are written into the third cache file.

3. The data storage method based on multi-path merging and multi-level caching according to claim 2, characterized in that, The multi-way merge algorithm comprises a loop traversal method, a minimum heap k-way merge sorting method, or a winner number k-way merge sorting method.

4. A data storage system based on multiway merging and multistage buffering, characterized by, The method comprises the following steps: a second cache module is configured to sequence and write first cache data not exceeding a first preset capacity into a second cache file; a multi-level cache module is configured to use a multi-way merge algorithm to write a plurality of second cache files not exceeding a second preset capacity into a third cache file, and sequence and merge a plurality of third cache files to write into a fourth cache file; the first cache is a memory; the second cache is an ins cache; the third cache is an mrg cache; and the fourth cache is a permanent storage; the second cache file is a sequenced dat file under an ins directory; and the dat file is limited to storing data with a maximum size of 64M; the first preset capacity is set to 64M, so as to write data with a size of 64M or less than 64M into the dat file; and the second preset capacity is 2G.

5. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the multi-way merge and multi-level cache-based data storage method in any one of claims 1 to 3.

6. A computer device, comprising: The device comprises the following components: a processor and a memory; the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory, so that the device executes the multi-way merge and multi-level cache-based data storage method in any one of claims 1 to 3.

Images