Data storage method, device, equipment, readable storage medium and program product

Abstract

This application relates to a data storage method, apparatus, device, readable storage medium, and program product. The method includes: first, when generating a sorted string table (SSTable) file, obtaining the current data input mode, including normal mode and bypass insertion mode; then, generating SSTable file sequence numbers according to the data input mode; finally, when merging multiple SSTables, determining the order of the multiple SSTable files based on their sequence numbers, and storing the data according to the determination result. This method enables the determination of the order of data within multiple SSTable files during merging by using their respective sequence numbers.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a data storage method, apparatus, device, readable storage medium, and program product. Background Technology

[0002] In the field of data storage, the LSM tree is a commonly used data storage structure, mainly composed of a memory table (Memtable), an immutable memory table (Immtable), and a sorted string table (SSTable). When data is input in normal mode, the data is first written to the Memtable. When the data volume of the Memtable reaches a threshold, it is converted into an Immtable. Both the Memtable and the Immtable exist in memory. The Immtable is then compressed into an SSTable and persisted to disk. During the data writing process, in addition to normal mode data input, there is also bypass data insertion. Bypass data insertion does not follow the normal data input process but directly generates an SSTable and places it in the L0 level SSTable of the LSM tree.

[0003] During data writing, duplicate keys will cause new data to overwrite old data during data merging. Therefore, determining the order in which data is written is crucial. Because bypass insertion and normal data input occur concurrently, the current SSTable generation method may be unable to determine the order of data writing. Summary of the Invention

[0004] Therefore, it is necessary to provide a data storage method, apparatus, device, readable storage medium, and program product that can determine the old and new order of SSTable files in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a data storage method, including:

[0006] When generating the sorted string table SSTable file, obtain the current data input mode, which includes normal mode and bypass insertion mode;

[0007] Generate the sequence number of the SSTable file based on the data input pattern;

[0008] When merging multiple SSTables, the order of the SSTable files is determined by their serial numbers, and the data is stored accordingly.

[0009] In the above embodiments, firstly, when generating the sorted string table (SSTable) file, the current data input mode, including normal mode and bypass insertion mode, is obtained. Then, based on the data input mode, an SSTable file sequence number is generated. Finally, when merging multiple SSTables, the order of the multiple SSTable files is determined based on their sequence numbers, and data is stored according to the determination result. In this way, a unique sequence number is generated for each SSTable file during generation. When merging multiple SSTable files, their order can be determined based on their respective sequence numbers, thereby determining the order of the data within the multiple SSTable files.

[0010] In one embodiment, the sequence number of the SSTable file is generated according to the data input pattern, including:

[0011] Retrieve the first and second historical serial numbers;

[0012] Based on the data input pattern, the first historical number and the second historical number are processed to obtain the first number and the second number;

[0013] Use the first number and the second number as the sequence number of the SSTable file, update the historical first number according to the first number, and update the historical second number according to the second number.

[0014] In the above embodiments, based on the data input mode, the sequence number of the SSTable file can be determined, including the first number and the second number. Based on the sequence number of the SSTable file, the order of the SSTable files can be determined. Therefore, an SSTable file only needs to store one SSTable file sequence number, and it is not necessary to store the time sequence number for each key in the SSTable file, thus saving hard disk space.

[0015] In one embodiment, the historical first number and historical second number are processed according to the data input pattern to obtain a first number and a second number, including:

[0016] If the data input mode is normal mode, the first historical number is incremented to obtain the first number, and the second historical number is cleared to obtain the second number.

[0017] In the above embodiments, the generation method of the SSTable file sequence number when the data input mode is normal mode is determined, so that the order of the old and new data can be determined based on the SSTable file sequence number.

[0018] In one embodiment, the historical first number and historical second number are processed according to the data input pattern to obtain a first number and a second number, including:

[0019] If the data input mode is bypass insertion mode, the second historical number is incremented to obtain the second number, and the first historical number is used as the first number.

[0020] In the above embodiments, the method for generating the sequence number of the SSTable file when the data input mode is bypass insertion mode is determined, so that the order of the data can be determined based on the sequence number of the SSTable file.

[0021] In one embodiment, if multiple data inputs start simultaneously, and all data inputs are in bypass insertion mode, the historical first number and historical second number are processed according to the data input mode to obtain the first number and the second number, including:

[0022] Acquire multiple data inputs from their respective data sources;

[0023] If the data sources of the multiple data inputs each meet the preset conditions, then the first historical number will be used as the first number of the multiple data inputs.

[0024] If the data sources of the multiple data inputs do not meet the preset conditions, the historical first number is processed according to the data sources of the multiple data inputs to obtain the first number corresponding to each of the multiple data inputs.

[0025] In the above embodiments, multiple batches of bypass insertion data from different data sources can be supported simultaneously. If multiple bypass insertion mode data inputs start simultaneously, the first and second serial numbers of the SSTable files generated by the multiple data inputs are determined based on the data source to which the multiple data inputs belong, and a certain order can be forcibly maintained according to the requirements of multiple data inputs.

[0026] In one embodiment, determining the order of multiple SSTable files based on their sequence numbers includes:

[0027] Retrieve the first and second numbers corresponding to each of the multiple SSTable files;

[0028] The order of the multiple SSTable files (old and new) is determined based on their respective first numbers.

[0029] If multiple SSTable files have the same first number, then the order of the multiple SSTable files is determined by their respective second numbers.

[0030] In the above embodiments, the order of multiple SSTable files can be determined based on the SSTable file number. When SSTable files are merged, the age of the data can be judged and the data can be stored, thereby enabling data input in normal mode and data input in bypass insertion mode to be performed concurrently.

[0031] Secondly, this application also provides a data storage device, the device comprising:

[0032] The acquisition module is used to obtain the current data input mode when generating the sorted string table SSTable file. The data input mode includes normal mode and bypass insertion mode.

[0033] The generation module is used to generate the sequence number of the SSTable file based on the data input pattern;

[0034] The judgment module is used to determine the order of multiple SSTable files when merging them, based on the file numbers, and then store the data according to the judgment result.

[0035] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the data storage methods described in the first aspect above.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the data storage methods described in the first aspect above.

[0037] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the data storage methods described in the first aspect above.

[0038] The aforementioned data storage method, apparatus, device, readable storage medium, and program product first obtain the current data input mode, including normal mode and bypass insertion mode, when generating a sorted string table (SSTable) file. Then, based on the data input mode, an SSTable file sequence number is generated. Finally, when merging multiple SSTables, the order of the SSTables is determined based on their sequence numbers, and data is stored according to the determination result. In this way, a unique sequence number is generated for each SSTable file during generation. During the merging of multiple SSTables, the order of the data within each SSTable file can be determined based on its sequence number, thus enabling the determination of the data's order within the multiple SSTable files. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating a data storage method in one embodiment;

[0041] Figure 2 This is a flowchart illustrating the data storage method in another embodiment;

[0042] Figure 3 This is a flowchart illustrating the data storage method in another embodiment;

[0043] Figure 4 This is a flowchart illustrating the data storage method in another embodiment;

[0044] Figure 5 This is a flowchart illustrating the data storage method in another embodiment;

[0045] Figure 6 This is a structural block diagram of a data storage device in one embodiment;

[0046] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] In one exemplary embodiment, such as Figure 1 As shown, a data storage method is provided. Taking the application of this method to a terminal as an example, the terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. It is understood that this method can also be applied to a server, and can also be applied to a system including both a terminal and a server, and is implemented through the interaction between the terminal and the server, including the following steps:

[0049] Step 101: When generating the sorted string table SSTable file, obtain the current data input mode.

[0050] The data input modes include normal mode and bypass insertion mode. Normal mode accepts input data from normal input / output (I / O) interfaces. Data is first written to a Memtable. When the Memtable's capacity reaches a threshold, it is converted to an Immtable. Both the Memtable and Immtable reside in memory. The Immtable is subsequently compressed into an SSTable and persisted to disk. Bypass insertion mode accepts input data from other modules. Instead of following the normal data input process, an SSTable is directly generated and placed in the L0 level SSTable of the LSM tree, then merged and pushed down with the L0 level data. Bypass insertion mode is faster than normal mode and saves system resources.

[0051] When generating an SSTable file, the current data input mode (normal mode or bypass insertion mode) can be determined based on the data writing method.

[0052] Step 102: Generate the sequence number of the SSTable file according to the data input mode.

[0053] Based on the current data input pattern, a unique sequence number is generated for each SSTable file when it is generated.

[0054] Step 103: When merging multiple SSTables, determine the order of the SSTable files based on their serial numbers, and store the data according to the determination result.

[0055] Data in SSTable files is stored in key-value pairs. When merging multiple SSTables, since SSTable files in L0 of the LSM tree can contain overlapping key-value ranges, multiple files may cover the same key. During the data merging process, new data will overwrite old data for duplicate keys. Therefore, it is particularly important to determine the order of the data with duplicate keys.

[0056] When multiple SSTable files contain duplicate keys, the order of the SSTable files is determined by their serial numbers. Based on this order, the order of the data containing the duplicate keys can be determined. According to the determination result, during the sinking and merging process, the new data of the duplicate keys can overwrite the old data for data storage.

[0057] In the above embodiments, firstly, when generating the sorted string table (SSTable) file, the current data input mode, including normal mode and bypass insertion mode, is obtained. Then, based on the data input mode, an SSTable file sequence number is generated. Finally, when merging multiple SSTables, the order of the multiple SSTable files is determined based on their sequence numbers, and data is stored according to the determination result. In this way, a unique sequence number is generated for each SSTable file during generation. When merging multiple SSTable files, their order can be determined based on their respective sequence numbers, thereby determining the order of the data within the multiple SSTable files.

[0058] In the embodiments of this application, step 102 above generates the sequence number of the SSTable file according to the data input pattern, such as... Figure 2 As shown, it may include:

[0059] Step 201: Obtain the first historical ID and the second historical ID.

[0060] The first and second historical IDs can be stored in global system variables and are globally unique. The initial first and second historical IDs can be initialized when the LSM tree version is created. Before acquiring the first and second historical IDs, a lock must be acquired; only under the protection of the lock can the first and second historical IDs be acquired and modified.

[0061] Step 202: Based on the data input pattern, process the first historical number and the second historical number to obtain the first number and the second number.

[0062] Specifically, the first historical number is processed to obtain the first number, and the second historical number is processed to obtain the second number. Optionally, the processing of the first and second historical numbers can be incremental or zeroing. The corresponding processing method is selected according to the data input mode to obtain the first and second numbers corresponding to the current data input mode.

[0063] Step 203: Use the first number and the second number as the sequence number of the SSTable file, update the historical first number according to the first number, and update the historical second number according to the second number.

[0064] The first and second numbers obtained after processing based on the first and second historical numbers will be used as the serial numbers of the newly generated SSTable files. The serial numbers of the SSTable files are unique and ordered, and the order of the old and new SSTable files can be determined based on the serial numbers.

[0065] Then, based on the current first number, determine the historical first number, and based on the current second number, determine the historical second number, to be used as the sequence number for generating the SSTable file the next time. After the historical first number and historical second number update operations are completed, release the lock.

[0066] Optionally, since the sequence number in the SSTable file can be used to determine the old and new order of duplicate keys, the original time sequence number saved for each key in the SSTable file can be deleted without having to persist the time sequence number to the hard drive, thus saving hard drive space.

[0067] In the above embodiments, based on the data input mode, the sequence number of the SSTable file can be determined, including the first number and the second number. Based on the sequence number of the SSTable file, the order of the SSTable files can be determined. Therefore, an SSTable file only needs to store one SSTable file sequence number, and it is not necessary to store the time sequence number for each key in the SSTable file, thus saving hard disk space.

[0068] In one embodiment, if the data input mode is normal mode, the step of processing the historical first number and the historical second number according to the data input mode to obtain the first number and the second number includes: if the data input mode is normal mode, incrementing the historical first number to obtain the first number, and clearing the historical second number to obtain the second number.

[0069] Optionally, the first and second historical numbers can be numerical. If the current input data is in normal mode, after obtaining the first historical number, it is incremented by 1 to obtain the first number. This first number is then used as the first historical number. The second historical number is cleared to obtain the second number. Based on the first and second numbers, the sequence number of the SSTable file generated from the current input data can be obtained. If the current input data requires multiple SSTable files, the first historical number is incremented each time a file is generated, and so on. For example, if the current input data generates three SSTable files, and the first and second historical numbers are both 0, then the sequence numbers of the three SSTable files are (1,0), (2,0), and (3,0), respectively.

[0070] In the above embodiments, the generation method of the SSTable file sequence number when the data input mode is normal mode is determined, so that the order of the old and new data can be determined based on the SSTable file sequence number.

[0071] If the data input mode is bypass insertion mode, the steps to process the first historical number and the second historical number according to the data input mode to obtain the first number and the second number include: if the data input mode is bypass insertion mode, increment the second historical number to obtain the second number, and use the first historical number as the first number.

[0072] Optionally, if the current input data is in bypass insertion mode and is a batch of data, after obtaining the first and second historical numbers, the first historical number remains unchanged as the first number of the currently generated SSTable file, and the second historical number is incremented by 1 to obtain the second number of the currently generated SSTable file, thus obtaining the SSTable file's sequence number. For example, if the first historical number is 2 and the second historical number is 0, and the current input data generates one SSTable file, then the SSTable file's sequence number is (2,1). It can be understood that if the current input data generates multiple SSTable files, only the first and second historical numbers need to be obtained the first time. The sequence numbers of other SSTable files are simply incremented by the second number based on (2,1), such as (2,2) and (2,3). Similarly, if bypass insertion continues after normal input data for the same data source, the second number is incremented based on the sequence number of the previously generated SSTable file, such as (2,3), to obtain the new SSTable file's sequence number.

[0073] Optionally, if multiple data inputs start simultaneously, and these inputs include both bypass insertion mode and normal insertion mode, as described above, for normal insertion mode, after obtaining the first historical number and the second historical number, the first historical number is incremented to obtain the first number, and the second historical number is cleared to obtain the second number. This process is repeated when multiple SSTables need to be generated. For bypass insertion mode, after obtaining the first historical number and the second historical number, the first historical number is used as the first number of the currently generated SSTable file. When multiple SSTable files need to be generated, the first number is always the first historical number at the start of bypass insertion, and the second historical number is incremented to become the second number of the currently generated SSTable file.

[0074] In the above embodiments, the method for generating the sequence number of the SSTable file when the data input mode is bypass insertion mode is determined, so that the order of the data can be determined based on the sequence number of the SSTable file.

[0075] In an optional embodiment of this application, if multiple data inputs start simultaneously, and all data input modes are bypass insertion modes, the steps for processing the historical first number and historical second number according to the data input mode to obtain the first number and the second number are as follows: Figure 3 As shown, it includes:

[0076] Step 301: Obtain the data sources for each of the multiple data inputs.

[0077] Among them, multiple data inputs can be data that needs to be processed from multiple different modules. Obtaining the data source of each multiple data input, that is, obtaining the module to which each multiple data input belongs, can be determined based on the identifier carried by the data input.

[0078] Step 302: If the data sources of the multiple data inputs meet the preset conditions, then the first historical number is used as the first number of the multiple data inputs.

[0079] The preset conditions can be set according to actual storage needs, stored in the system, and modified according to actual conditions; this application does not impose any restrictions on this. In this embodiment, the preset conditions can be that the multiple modules corresponding to the multiple data inputs are business-unrelated, meaning the data do not have the same key value. After obtaining the modules to which each of the multiple data inputs belongs, a preset configuration table is queried according to each module to determine whether the modules are business-unrelated, i.e., do not have the same key value. If so, it is determined that the data source to which each of the multiple data inputs belongs meets the preset conditions. At this time, the multiple data inputs can share the same first number, that is, the historical first number is used as the first number of the multiple data inputs.

[0080] Among them, the multiple data inputs from their respective data sources that meet the preset conditions are secured by a locking and unlocking mechanism to ensure the security of acquiring and modifying the first and second historical numbers. For example, when a certain data input is currently being performed, before determining the first and second numbers, the lock is acquired first, and then the first and second historical numbers are acquired. Taking the first and second historical numbers as an example where both the first and second historical numbers are currently 1, the first historical number is used as the first number, and the second historical number 1 is incremented to become the second number 2 of the currently generated SSTable file. That is, the sequence number of the currently generated SSTable file is (1,2). After updating the second historical number 1 according to the second number 2, the lock is released after the operation is completed. At this time, another data input is performed. First, the lock is acquired, and then the first historical number and the second historical number are acquired. At this time, the first historical number is 1 and the second historical number is 2. The first historical number is used as the first number, and the second historical number 2 is incremented to become the second number 3 of the currently generated SSTable file. That is, the sequence number of the currently generated SSTable file is (1,3). After updating the second historical number 2 according to the second number 3, the lock is released after the operation is completed. At this time, the first historical number is 1 and the second historical number is 3. This process is repeated until the multi-channel data input is completed, thus ensuring the uniqueness of the second number.

[0081] Step 303: If the data sources of the multiple data inputs do not meet the preset conditions, the historical first number is processed according to the data sources of the multiple data inputs to obtain the first number corresponding to each of the multiple data inputs.

[0082] If multiple data inputs belong to modules with related business functions (i.e., they may share the same key), then the historical first number is processed according to the requirements of each module to obtain the first number corresponding to each data input. The first numbers of the generated SSTable files from these multiple data inputs will be different. If the requirement of a module is that data cannot be overwritten, then the historical first number is incremented to obtain the first number corresponding to that module, and the historical second number is cleared to obtain the second number corresponding to that module.

[0083] Optionally, taking two data inputs as an example, one data input comes from module E, and the other data input comes from module F. When data from modules E and F are simultaneously input through the bypass insertion mode, the preset configuration table is consulted according to the data source to determine that the data from module E cannot overwrite the data from module F. Therefore, it is necessary to ensure that the data from module F is always newer than the data from module E. At this time, the current historical first number is obtained as 5, and this historical first number 5 is used as the first number of the SSTable file generated by module E. If module E needs to generate multiple SSTable files, the sequence number of the generated SSTable files is updated based on (5,0) with the second number, such as (5,1) and (5,2). Module F obtains the current historical first number as 5, increments the historical first number, and clears the historical second number, resulting in the first number of the SSTable file generated by module F being 6. If module F needs to generate multiple SSTable files, the sequence number of the generated SSTable files is updated based on (6,0) with the second number, such as (6,1) and (6,2).

[0084] In the above embodiments, multiple batches of bypass insertion data from different data sources can be supported simultaneously. If multiple bypass insertion modes are included, the first and second serial numbers of the SSTable files generated by the multiple data inputs are determined based on the data source to which the multiple data inputs belong, and a certain order can be forcibly maintained according to the requirements of multiple data inputs.

[0085] In one embodiment, after generating an SSTable file sequence number for each SSTable file in the manner described above, when merging multiple SSTable files, the step of determining the order of the multiple SSTable files based on their respective sequence numbers is as follows: Figure 4 As shown, it includes:

[0086] Step 401: Obtain the first and second numbers corresponding to each of the multiple SSTable files.

[0087] Retrieve the first and second SSTable file numbers of multiple SSTable files.

[0088] Step 402: Determine the order of the SSTable files based on their respective first numbers.

[0089] The larger the first number, the newer the SSTable file; the smaller the first number, the older the SSTable file.

[0090] Step 403: If the first number of each of the multiple SSTable files is the same, then the new and old order of the multiple SSTable files is determined according to the second number of each of the multiple SSTable files.

[0091] If multiple SSTable files have the same first number, then the second number is further compared. Similarly, the larger the second number, the newer the SSTable file is, and the smaller the second number, the older the SSTable file is.

[0092] The above judgment process is illustrated with an example below. There is data input A in normal mode, and data inputs B, C, and D in bypass insertion mode. The first and second historical numbers are both 0, i.e., (0, 0). Data input A generates three SSTable files, with sequence numbers (1, 0), (2, 0), and (3, 0) respectively. At this point, the first historical number is updated to 1, 2, and 3, and the second historical number is updated to 0. Then, data input B in bypass insertion mode generates three SSTable files, with sequence numbers (3, 1), (3, 2), and (3, 3) respectively. Next, data input A in normal mode generates one more SSTable file, incrementing the first historical number and resetting the second historical number to zero, resulting in a single SSTable file with the sequence number (4, 0). At this point, the first historical number is updated to 4, and the second historical number is updated to 0.

[0093] At this point, data inputs C and D in bypass insertion mode are input simultaneously, and the data sources of data inputs C and D meet the preset conditions. If C first generates one SSTable file with the sequence number (4,1), and then D generates one SSTable file with the sequence number (4,2), then C continues to generate another SSTable file with the sequence number (4,3). If data input A in normal mode then generates another SSTable file, the first historical sequence number increments, the second historical sequence number is reset to zero, resulting in one SSTable file with the sequence number (5,0).

[0094] Then, in bypass insertion mode, data inputs M and N are input simultaneously, and the data sources for data inputs M and N are from different modules, which does not meet the preset conditions. It is necessary to ensure that the data in data input N is always newer than the data in data input M. At this point, the current historical first number is obtained as 5, and the historical second number is obtained as 0. Therefore, the first number of the SSTable file generated by data input M is 5. If data input M needs to generate multiple SSTable files, the sequence number of the generated SSTable files is updated based on (5,0), such as (5,1) and (5,2). Data input N obtains the current historical first number as 5, increments the historical first number, and obtains the first number of the SSTable file generated by data input N as 6. The historical second number is cleared. Therefore, if data input N needs to generate multiple SSTable files, the sequence number of the generated SSTable files is updated based on (6,0), such as (6,1) and (6,2).

[0095] If data input B in bypass insertion mode continues to input data, the second number will continue to increment based on the sequence numbers (3,1), (3,2), and (3,3) of the SSTable files generated by data input B, such as (3,4) and (3,5). If data input C in bypass insertion mode continues to input data, the second number will continue to increment based on the sequence numbers (4,1), (4,2), and (4,3) of the SSTable files generated by data input C and data input D, such as (4,4) and (4,5).

[0096] Sort all the SSTable files above in the order from oldest to newest as follows: (1,0)(2,0) (3,0)(3,1)(3,2)(3,3)(3,4)(3,5)(4,0)(4,1)(4,2)(4,3)(4,4)(4,5)(5,0)(5,1)(5,2)(6,0)(6,1)(6,2).

[0097] In the above embodiments, the order of multiple SSTable files can be determined based on the SSTable file number. When SSTable files are merged, the age of the data can be judged and the data can be stored, thereby enabling data input in normal mode and data input in bypass insertion mode to be performed concurrently.

[0098] In the embodiments of this application, such as Figure 5 As shown, a data storage method is provided, including:

[0099] Step 501: When generating the sorted string table SSTable file, obtain the current data input mode.

[0100] Step 502: Obtain the first historical ID and the second historical ID.

[0101] Step 503: Based on the data input pattern, process the first historical number and the second historical number to obtain the first number and the second number.

[0102] Step 504: Use the first number and the second number as the sequence number of the SSTable file, update the historical first number according to the first number, and update the historical second number according to the second number.

[0103] Step 505: Obtain the first and second numbers corresponding to each of the multiple SSTable files.

[0104] Step 506: Determine the order of the multiple SSTable files based on their respective first numbers.

[0105] Step 507: If the first number of each of the multiple SSTable files is the same, then determine the old and new order of the multiple SSTable files according to the second number of each of the multiple SSTable files.

[0106] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0107] Based on the same inventive concept, this application also provides a data storage device for implementing the data storage method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more data storage device embodiments provided below can be found in the limitations of the data storage method described above, and will not be repeated here.

[0108] In one exemplary embodiment, such as Figure 6 As shown, a data storage device 600 is provided, including: an acquisition module 601, a generation module 602, and a judgment module 603, wherein:

[0109] The acquisition module 601 is used to acquire the current data input mode when generating the sorted string table SSTable file. The data input mode includes normal mode and bypass insertion mode.

[0110] The generation module 602 is used to generate the sequence number of the SSTable file according to the data input pattern;

[0111] The judgment module 603 is used to determine the order of the multiple SSTable files based on their serial numbers when merging multiple SSTables, and to store the data according to the judgment result.

[0112] In one embodiment, the generation module 602 is specifically used to obtain a historical first number and a historical second number; process the historical first number and the historical second number according to the data input pattern to obtain a first number and a second number; use the first number and the second number as the sequence number of the SSTable file, and update the historical first number according to the first number and update the historical second number according to the second number.

[0113] In one embodiment, the generation module 602 is specifically used to increment the historical first number to obtain a first number and clear the historical second number to obtain a second number if the data input mode is normal mode.

[0114] In one embodiment, the generation module 602 is specifically used to increment the historical second number to obtain a second number if the data input mode is a bypass insertion mode, and use the historical first number as the first number.

[0115] In one embodiment, if multiple data inputs start simultaneously, and the data input modes of the multiple data inputs are all bypass insertion modes, the generation module 602 is specifically used to obtain the data source to which each of the multiple data inputs belongs; if the data source to which each of the multiple data inputs belongs meets the preset conditions, then the historical first number is used as the first number of the multiple data inputs; if the data source to which each of the multiple data inputs belongs does not meet the preset conditions, then the historical first number is processed according to the data source to which each of the multiple data inputs belongs to obtain the first number corresponding to each of the multiple data inputs.

[0116] In one embodiment, the determination module 603 is specifically used to obtain the first number and the second number corresponding to each of the multiple SSTable files; determine the new and old order of the multiple SSTable files according to the first number corresponding to each of the multiple SSTable files; if the first numbers corresponding to the multiple SSTable files are the same, then determine the new and old order of the multiple SSTable files according to the second number corresponding to each of the multiple SSTable files.

[0117] Each module in the aforementioned data storage device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0118] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data storage method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0119] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0120] In one exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: when generating a sorted string table (SSTable) file, obtaining the current data input mode, which includes a normal mode and a bypass insertion mode; generating an SSTable file sequence number according to the data input mode; and when performing sinking and merging on multiple SSTables, determining the new and old order of the multiple SSTable files according to their sequence numbers, and storing the data according to the determination result.

[0121] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining a historical first number and a historical second number; processing the historical first number and the historical second number according to the data input pattern to obtain a first number and a second number; using the first number and the second number as the sequence number of the SSTable file, and updating the historical first number according to the first number and updating the historical second number according to the second number.

[0122] In one embodiment, when the processor executes the computer program, it further performs the following steps: if the data input mode is normal mode, incrementing the historical first number to obtain a first number, and clearing the historical second number to obtain a second number.

[0123] In one embodiment, when the processor executes the computer program, it further performs the following steps: if the data input mode is a bypass insertion mode, incrementing the historical second number to obtain a second number, and using the historical first number as the first number.

[0124] In one embodiment, if multiple data inputs start simultaneously, and all data inputs are in bypass insertion mode, the processor, when executing the computer program, further implements the following steps: obtaining the data source to which each of the multiple data inputs belongs; if the data source to which each of the multiple data inputs belongs meets a preset condition, then using the historical first number as the first number of the multiple data inputs; if the data source to which each of the multiple data inputs belongs does not meet the preset condition, then processing the historical first number according to the data source to which each of the multiple data inputs belongs to obtain the first number corresponding to each of the multiple data inputs.

[0125] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining a first number and a second number corresponding to each of the multiple SSTable files; determining the new and old order of the multiple SSTable files based on the first number corresponding to each of the multiple SSTable files; if the first numbers corresponding to the multiple SSTable files are the same, then determining the new and old order of the multiple SSTable files based on the second number corresponding to each of the multiple SSTable files.

[0126] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon. When the computer program is executed by a processor, it performs the following steps: when generating a sorted string table (SSTable) file, obtaining the current data input mode, which includes a normal mode and a bypass insertion mode; generating an SSTable file sequence number based on the data input mode; and when performing sinking and merging of multiple SSTables, determining the new and old order of the multiple SSTable files based on their sequence numbers, and storing the data based on the determination result.

[0127] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining a first historical number and a second historical number; processing the first historical number and the second historical number according to the data input pattern to obtain a first number and a second number; using the first number and the second number as the sequence number of the SSTable file, and updating the first historical number according to the first number and updating the second historical number according to the second number.

[0128] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if the data input mode is normal mode, increment the historical first number to obtain the first number, and clear the historical second number to obtain the second number.

[0129] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if the data input mode is a bypass insertion mode, increment the historical second number to obtain a second number, and use the historical first number as the first number.

[0130] In one embodiment, if multiple data inputs start simultaneously, and all data inputs are in bypass insertion mode, the computer program, when executed by the processor, further implements the following steps: obtaining the data source to which each of the multiple data inputs belongs; if the data source to which each of the multiple data inputs belongs meets a preset condition, then using the historical first number as the first number of the multiple data inputs; if the data source to which each of the multiple data inputs belongs does not meet the preset condition, then processing the historical first number according to the data source to which each of the multiple data inputs belongs to obtain the first number corresponding to each of the multiple data inputs.

[0131] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining a first number and a second number corresponding to each of the multiple SSTable files; determining the new and old order of the multiple SSTable files based on the first number corresponding to each of the multiple SSTable files; if the first numbers corresponding to the multiple SSTable files are the same, then determining the new and old order of the multiple SSTable files based on the second number corresponding to each of the multiple SSTable files.

[0132] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: when generating a sorted string table (SSTable) file, obtaining the current data input mode, including a normal mode and a bypass insertion mode; generating an SSTable file sequence number based on the data input mode; and when performing sinking and merging of multiple SSTables, determining the new and old order of the multiple SSTable files based on their sequence numbers, and storing the data based on the determination result.

[0133] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining a first historical number and a second historical number; processing the first historical number and the second historical number according to the data input pattern to obtain a first number and a second number; using the first number and the second number as the sequence number of the SSTable file, and updating the first historical number according to the first number and updating the second historical number according to the second number.

[0134] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if the data input mode is normal mode, increment the historical first number to obtain the first number, and clear the historical second number to obtain the second number.

[0135] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if the data input mode is a bypass insertion mode, increment the historical second number to obtain a second number, and use the historical first number as the first number.

[0136] In one embodiment, if multiple data inputs start simultaneously, and all data inputs are in bypass insertion mode, the computer program, when executed by the processor, further implements the following steps: obtaining the data source to which each of the multiple data inputs belongs; if the data source to which each of the multiple data inputs belongs meets a preset condition, then using the historical first number as the first number of the multiple data inputs; if the data source to which each of the multiple data inputs belongs does not meet the preset condition, then processing the historical first number according to the data source to which each of the multiple data inputs belongs to obtain the first number corresponding to each of the multiple data inputs.

[0137] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining a first number and a second number corresponding to each of the multiple SSTable files; determining the new and old order of the multiple SSTable files based on the first number corresponding to each of the multiple SSTable files; if the first numbers corresponding to the multiple SSTable files are the same, then determining the new and old order of the multiple SSTable files based on the second number corresponding to each of the multiple SSTable files.

[0138] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0140] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0141] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data storage method, characterized in that, The method includes: When generating the sorted string table SSTable file, the current data input mode is obtained, which includes normal mode and bypass insertion mode; Obtain the first historical ID and the second historical ID; process the first historical ID and the second historical ID according to the data input pattern to obtain the first ID and the second ID; Use the first number and the second number as the sequence number of the SSTable file, update the historical first number according to the first number, and update the historical second number according to the second number; When merging multiple SSTables, obtain the first number and second number corresponding to each of the multiple SSTable files; The order of the multiple SSTable files is determined based on their respective first numbers; If the first number corresponding to multiple SSTable files is the same, then the new and old order of the multiple SSTable files is determined according to the second number corresponding to each of the multiple SSTable files, and the data is stored according to the determination result.

2. The method according to claim 1, characterized in that, Before obtaining the first historical ID and the second historical ID, the method further includes: Acquire the lock, and under the protection of the lock, acquire the first historical number and the second historical number.

3. The method according to claim 1, characterized in that, The step of processing the first historical number and the second historical number according to the data input pattern to obtain the first number and the second number includes: If the data input mode is normal mode, the first historical number is incremented to obtain the first number, and the second historical number is cleared to obtain the second number.

4. The method according to claim 1, characterized in that, The step of processing the first historical number and the second historical number according to the data input pattern to obtain the first number and the second number includes: If the data input mode is bypass insertion mode, the second historical number is incremented to obtain the second number, and the first historical number is used as the first number.

5. The method according to claim 4, characterized in that, If multiple data inputs begin simultaneously, and all data inputs are in bypass insertion mode, the step of processing the first historical number and the second historical number according to the data input mode to obtain the first number and the second number includes: Obtain the data source to which each of the multiple data inputs belongs; If the data sources of the multiple data inputs each meet the preset conditions, then the historical first number is used as the first number of the multiple data inputs; If the data sources of the multiple data inputs do not meet the preset conditions, the historical first number is processed according to the data sources of the multiple data inputs to obtain the first number corresponding to each of the multiple data inputs.

6. The method according to claim 5, characterized in that, The preset condition is that the services of the data sources to which the multiple data inputs belong are unrelated.

7. A data storage device, characterized in that, The device includes: The acquisition module is used to acquire the current data input mode when generating the sorted string table SSTable file. The data input mode includes normal mode and bypass insertion mode. A generation module is used to obtain a historical first number and a historical second number; process the historical first number and the historical second number according to the data input pattern to obtain a first number and a second number; use the first number and the second number as the sequence number of the SSTable file, and update the historical first number according to the first number and update the historical second number according to the second number; The judgment module is used to obtain the first number and second number corresponding to each of the multiple SSTable files when performing sinking and merging; determine the old and new order of the multiple SSTable files according to the first number corresponding to each of the multiple SSTable files; if the first number corresponding to each of the multiple SSTable files is the same, then determine the old and new order of the multiple SSTable files according to the second number corresponding to each of the multiple SSTable files, and perform data storage according to the judgment result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images