Data storage method and device

Abstract

The invention discloses a data storage method and device, relates to the technical field of data processing, and mainly aims to solve the problem of poor data read-write efficiency of an existing database. Comprising the steps that a slot identifier of a target row and to-be-stored column data are obtained, and the slot identifier is generated by a distribution key of the target row; an index file is called, recording information between different slot identifiers and data pages is stored in the index file, and the data pages are used for storing data; and determining a target data page corresponding to the slot identifier based on the record information, and storing the to-be-stored column data in the target data page.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data storage method and device. Background Art

[0002] With the advent of the big data era, whether the storage function of the data storage engine is powerful enough has become the focus of users. In particular, how to quickly store massive data to achieve accurate reading is the development focus of the data storage engine.

[0003] At present, when existing data is migrated or stored or load-balanced, it is usually migrated or stored according to fixed columns or rows of the entire data table or the entire block of data. However, migrating or storing according to fixed columns or rows of the entire data table or the entire block of data requires the redeployment of multiple storage nodes, and is prone to data loss, affecting the speed and accuracy of data migration, thereby greatly reducing the efficiency of data recovery during database transformation and affecting the storage effect of data in the database. Summary of the invention

[0004] In view of this, the present invention provides a data storage method and device, the main purpose of which is to solve the problem of poor efficiency in reading and writing data in existing databases.

[0005] According to one aspect of the present invention, there is provided a data storage method, comprising:

[0006] Obtaining a slot identifier of a target row and column data to be stored, wherein the slot identifier is generated by a distribution key of the target row;

[0007] Retrieving an index file, wherein the index file stores record information between different slot identifiers and data pages, wherein the data pages are used to store data;

[0008] A target data page corresponding to the slot identifier is determined based on the record information, and the column data to be stored is stored in the target data page.

[0009] Further, the determining the target data page corresponding to the slot identifier based on the index file includes:

[0010] If the slot identifier is found in the index file, the target data page of the record information matching the slot identifier is retrieved;

[0011] If the slot identifier is not found in the index file, a target data page of the slot identifier is generated, and record information of the slot identifier and the target data page is generated and stored in the index file.

[0012] Furthermore, before obtaining the slot identifier of the target row and the column data to be stored, the method further includes:

[0013] Create multiple data pages and index pages including public page headers and non-public page headers in a preset storage area;

[0014] The public page header stores basic page information, the non-public page header of the index page stores index level and index type, and the non-public page header of the data page stores slot identification and page attribute information.

[0015] Furthermore, after storing the to-be-stored column data in the target data page, the method further includes:

[0016] Obtaining a target index page corresponding to the target data page;

[0017] Generate a data index according to storage information of the column data to be stored in the target data page;

[0018] The data index is stored in the target index page, and the index level and index type of the index page are updated based on the data index.

[0019] Furthermore, the target data page includes a first write page and a current write page, and storing the to-be-stored column data in the target data page includes:

[0020] If the slot identifier is found in the index file, the column data to be stored is stored in the current write page.

[0021] If the slot identifier is not found in the index file, the column data to be stored is stored in the first write page.

[0022] Furthermore, before obtaining the slot identifier of the target row and the column data to be stored, the method further includes:

[0023] After responding to a storage request of the data table, the row data in the data table is retrieved and stored in the transaction buffer in row units;

[0024] In response to a row data storage request submitted by the transaction buffer, the row data is converted into column data to be stored.

[0025] Furthermore, after storing the to-be-stored column data in the target data page, the method further includes:

[0026] In response to a read instruction of target data, determining a read slot identifier of the target data, and retrieving the index file;

[0027] If the read slot identifier is found in the index file, a reference data page matching the read slot identifier and a reference index page corresponding to the reference data page are searched based on all non-public page headers;

[0028] The storage information of the target data is queried from the reference index page, and the data content is read from the reference data page according to the storage information.

[0029] Furthermore, the method further comprises:

[0030] Creating a system page, wherein the system page stores home page information of all index pages;

[0031] When the system is restarted, the system page is read, and all slot identifiers and data pages corresponding to the slot identifiers are read from the home page information to reconstruct the index file.

[0032] According to another aspect of the present invention, there is provided a data storage device, comprising:

[0033] An acquisition module, used to acquire a slot identifier of a target row and column data to be stored, wherein the slot identifier is generated by a distribution key of the target row;

[0034] A calling module, used to call an index file, wherein the index file stores record information between different slot identifiers and data pages, and the data pages are used to store data;

[0035] A storage module is used to determine a target data page corresponding to the slot identifier based on the record information, and store the column data to be stored in the target data page.

[0036] Furthermore, the storage module is specifically used to retrieve the target data page of the record information matching the slot identifier if the slot identifier is found from the index file; if the slot identifier is not found from the index file, generate the target data page of the slot identifier, and generate the record information of the slot identifier and the target data page, and store them in the index file.

[0037] Furthermore, the device also includes:

[0038] A creation module is used to create multiple data pages and index pages including public page headers and non-public page headers in a preset storage area; wherein the public page header stores page basic information, the non-public page header of the index page stores index level and index type, and the non-public page header of the data page stores slot identification and page attribute information.

[0039] Furthermore, the device further comprises: a generating module,

[0040] The acquisition module is further used to acquire the target index page corresponding to the target data page;

[0041] The generating module is used to generate a data index according to the storage information of the column data to be stored in the target data page;

[0042] The storage module is further used to store the data index in the target index page, and update the index level and index type of the index page based on the data index.

[0043] Furthermore, the target data page includes a first-written page and a currently written page.

[0044] The storage module is further configured to store the column data to be stored in the current write page if the slot identifier is found in the index file, and store the column data to be stored in the first write page if the slot identifier is not found in the index file.

[0045] Furthermore, the device also includes: a conversion module, which is used to respond to a storage request for the data table, retrieve the row data in the data table in units of rows and store them in the transaction buffer; when responding to a row data storage request submitted by the transaction buffer, convert the row data into column data to be stored.

[0046] Furthermore, the creation module is also used to respond to a read instruction of the target data, determine the read slot identifier of the target data, and call the index file; if the read slot identifier is queried in the index file, then based on all non-public page headers, query the reference data page matching the read slot identifier and the reference index page corresponding to the reference data page; query the storage information of the target data from the reference index page, and read the data content from the reference data page according to the storage information.

[0047] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute an operation corresponding to the above-mentioned data storage method.

[0048] According to another aspect of the present invention, there is provided a terminal, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0049] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned data storage method.

[0050] By means of the above technical solution, the technical solution provided by the embodiment of the present invention has at least the following advantages:

[0051] The present invention provides a data storage method and device. Compared with the prior art, the embodiment of the present invention obtains the slot identifier of a target row and the column data to be stored, wherein the slot identifier is generated by the distribution key of the target row; retrieves an index file, wherein the index file stores record information between different slot identifiers and data pages, and the data page is used to store data; determines a target data page corresponding to the slot identifier based on the record information, and stores the column data to be stored in the target data page, thereby realizing the storage of column data after a mapping relationship is formed between the slot identifier and the page, thereby greatly improving the management efficiency of data storage, thereby improving the speed and accuracy of data reading and writing.

[0052] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0054] Figure 1 A flow chart of a data storage method provided by an embodiment of the present invention is shown;

[0055] Figure 2 A schematic diagram of a method for querying index records and data pages provided by an embodiment of the present invention is shown;

[0056] Figure 3 A block diagram of a data storage device provided by an embodiment of the present invention is shown;

[0057] Figure 4 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0058] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0059] The embodiment of the present invention provides a data storage method, such as Figure 1As shown, the method includes:

[0060] 101. Obtain the slot identifier of the target row and the column data to be stored.

[0061] In the embodiment of the present invention, as the execution end of the data storage method, the current execution subject can be a server for processing various business data, such as a cloud server, or a terminal device with data function and storage function, which is not specifically limited in the embodiment of the present invention. Among them, the target behavior is a row in the data form to be stored, so as to retrieve the data in this row as the column data to be stored. In addition, in the embodiment of the present invention, the slot identifier is generated by the distribution key of the target row, that is, each row in the data form corresponds to a distribution key, and the value obtained by calculating the hash value of the distribution key is used as the slot number, so as to serve as the retrieval basis for each row of data.

[0062] 102. Retrieve the index file.

[0063] In the embodiment of the present invention, an index file is pre-stored in the current execution end. At this time, the index file stores record information between different slot identifiers and data pages. The record information is used to characterize the record content when each slot identifier is recorded in the data page to store column data, so that the data page corresponding to this slot identifier can be quickly found. This embodiment of the present invention does not make specific limitations. In addition, in the embodiment of the present invention, the data page is used to store data, that is, multiple data pages are pre-created in the current execution end to store column data in the form of pages, and one data page stores the column data corresponding to a row. This embodiment of the present invention does not make specific limitations.

[0064] 103. Determine a target data page corresponding to the slot identifier based on the record information, and store the column data to be stored in the target data page.

[0065] In the embodiment of the present invention, after the current execution end retrieves the index file, it first searches for the record information. At this time, if there is a target data page whose slot identifier has been recorded in the record information, the target data page corresponding to the slot identifier is found, and the column data to be stored is stored in the target data page. If the target data page corresponding to the slot identifier is not recorded in the record information, it means that data has not been stored for this target row before, so a new data page corresponding to the slot identifier is generated as the target data page, and the column data to be stored is stored in the target data page.

[0066] In another embodiment of the present invention, for further definition and explanation, the step of determining the target data page corresponding to the slot identifier based on the index file includes:

[0067] If the slot identifier is found in the index file, the target data page of the record information matching the slot identifier is retrieved;

[0068] If the slot identifier is not found in the index file, a target data page of the slot identifier is generated, and record information of the slot identifier and the target data page is generated and stored in the index file.

[0069] In the embodiment of the present invention, in order to speed up the data storage and avoid the tediousness of query, when the current execution end determines the target data page, specifically, first searches the slot identifier from the index file to determine whether there is record information. If the slot identifier can be found in the index file, it means that there is record information that has stored the column data, so the target data page whose slot identifier matches the record information can be retrieved. If the slot identifier is not found in the index file, it means that the column data has not been stored before, so a target data page corresponding to the slot identifier is generated, and a new record information is generated and updated in the index file, so that it can be used as a basis for subsequent queries.

[0070] It should be noted that the data page can be the home page for the first generation of the slot identifier, and can be the current page for the matching after the slot identifier has been generated. In a specific implementation scenario, when the slot identifier cannot be found in the record information, it means that the data storage content corresponding to the slot identifier has not been recorded. Therefore, the data page of the generated slot identifier is the first write page. Correspondingly, when the slot identifier is found in the record information, it means that the data storage content corresponding to the slot identifier has been recorded. Therefore, the data page corresponding to the slot identifier is found to be the current write page, and the embodiment of the present invention does not make specific limitations.

[0071] In another embodiment of the present invention, for further definition and explanation, before the step of obtaining the slot identifier of the target row and the column data to be stored, the method further includes:

[0072] A plurality of data pages and index pages including a common page header and a non-common page header are created in a preset storage area.

[0073] In order to improve the accuracy and convenience of data storage and reading, and to achieve the purpose of flexible management of data, the current execution end pre-creates multiple pages and divides them into data pages and index pages. Specifically, when creating multiple pages, you can choose to divide the area in the preset storage area in the cache or hard disk area of ​​the current execution subject as the basic unit for storing data. In addition, the size of the data stored in each page is the same, and both contain a public page header and a non-public page header part. At this time, the index page is used to store the index content of the data storage, the public page header stores the page basic information, the non-public page header of the index page stores the index level and index type, and the non-public page header of the data page stores the slot identification and page attribute information. Among them, the page basic information includes the page type, the stored data type and the next page number, the page management protocol, the version number and the flag information, etc. The index level is used to characterize the level of the index formed by the data stored in the data page, the index type is used to characterize the type of index formed by the data stored in the data page, and the page attribute information includes the column index, the column type and the compression level information, etc., which are not specifically limited in the embodiment of the present invention.

[0074] In another embodiment of the present invention, for further definition and explanation, after the step of storing the to-be-stored column data in the target data page, the method further includes:

[0075] Obtaining a target index page corresponding to the target data page;

[0076] Generate a data index according to storage information of the column data to be stored in the target data page;

[0077] The data index is stored in the target index page, and the index level and index type of the index page are updated based on the data index.

[0078] In order to speed up the writing efficiency of data, after storing the column data in the target data page, the current execution end obtains the target index page of the target data page. Among them, if the target data page is newly generated, the corresponding target index page is also newly generated. If the target data page already exists, then the existing target index page can be obtained, and the embodiment of the present invention does not make specific limitations. Furthermore, a data index is generated according to the storage information of the column data to be stored in the target data page, wherein the storage information may include but is not limited to the storage location, storage type, etc., so as to generate a corresponding data index. In addition, the current execution end stores the generated data index in the target index page, and updates the corresponding index level and index type. At this time, since the column data is stored in the order of instructions or time, the corresponding data index can also be stored in the target index page under the previous data index in the order of instructions or time, and the corresponding index level and index type are updated, such as the first-level index, the second-level index, etc., and the full-text index, the ordinary index, etc., and the embodiment of the present invention does not make specific limitations.

[0079] In another embodiment of the present invention, for further definition and explanation, the step of storing the to-be-stored column data in the target data page includes:

[0080] If the slot identifier is found in the index file, the column data to be stored is stored in the current write page.

[0081] If the slot identifier is not found in the index file, the column data to be stored is stored in the first write page.

[0082] Since the target data page includes the first write page and the current write page, in order to meet the management effectiveness of different data storage, when storing the column data to be stored in the target data page, in a specific implementation scenario, first query whether there is a slot identifier from the index file. If so, the column data to be stored is stored in the current write page. If the slot identifier is not found, the column data to be stored is stored in the first write page.

[0083] It should be noted that since the slot identifier is obtained by calculating the hash value of the distribution key of each row, a maximum of 65535 slot identifiers can be obtained in the storage space in the embodiment of the present invention, so that the current execution end can retrieve according to the slot identifier when reading and writing data. The embodiment of the present invention does not make specific limitations.

[0084] In another embodiment of the present invention, for further definition and explanation, before the step of obtaining the slot identifier of the target row and the column data to be stored, the method further includes:

[0085] After responding to a storage request of the data table, the row data in the data table is retrieved and stored in the transaction buffer in row units;

[0086] In response to a row data storage request submitted by the transaction buffer, the row data is converted into column data to be stored.

[0087] In order to speed up the data storage and improve the effectiveness of data storage, the current execution end may also receive a storage request generated for the data table before obtaining the slot identifier. At this time, the storage request includes but is not limited to the update of the storage disk or the backup of the storage disk, etc., which is not specifically limited in the embodiment of the present invention. After the current execution end responds to the storage request, it retrieves each row data in the data table in row units and stores it in order to the transaction buffer. Then, after the transaction buffer submits the row data storage request, the current execution end converts the row data into the column data to be stored, so that when writing, the column data corresponding to the target row can be directly retrieved, which is not specifically limited in the embodiment of the present invention. In addition, the transaction buffer is an area used to temporarily store uncommitted data changes in the transaction during the database transaction processing. The current execution end can create a corresponding transaction buffer before executing steps 101-103 to meet different data storage requirements.

[0088] In another embodiment of the present invention, for further definition and explanation, after the step of storing the to-be-stored column data in the target data page, the method further includes:

[0089] In response to a read instruction of target data, determining a read slot identifier of the target data, and retrieving the index file;

[0090] If the read slot identifier is found in the index file, a reference data page matching the read slot identifier and a reference index page corresponding to the reference data page are searched based on all non-public page headers;

[0091] The storage information of the target data is queried from the reference index page, and the data content is read from the reference data page according to the storage information.

[0092] In order to achieve the purpose of effective reading, after receiving the read instruction of the target data, the current execution end determines the read slot identifier of the target data, such as querying the distribution key of the row for calculation, and calling the index file. Among them, since the index file stores the record information between different slot identifiers and data pages, the slot identifier is queried from the index file. At this time, if the non-public page header of the created page contains information such as the slot identifier and page attribute information, the corresponding reference data page and the reference index page corresponding to the reference data page can be queried according to the slot identifier. Furthermore, the storage information of the target data is queried from the reference index page, so that the data content is read from the reference data page according to the storage information.

[0093] In another embodiment of the present invention, for further definition and description, the steps further include:

[0094] Create system page;

[0095] When the system is restarted, the system page is read, and all slot identifiers and data pages corresponding to the slot identifiers are read from the home page information to reconstruct the index file.

[0096] In order to avoid the system being unable to perform normal data recovery when it crashes and to achieve rapid recovery of read and write functions, the current execution end creates a system page while creating a data page and a storage page. The system page stores the home page information of all index pages. After the system of the current execution end is restarted, this system page is read, and all slot identifiers are read from the home page information of all index pages. At this time, since each slot identifier corresponds to all data pages of the column data of a target row, when the system crashes, the slot identifier in the home page information of the index page and the first write position of the corresponding column data and the current write position of the column data corresponding to the slot identifier can be quickly read, thereby achieving rapid recovery of the index and normal read and write functions.

[0097] In a scenario of an embodiment of the present invention, Figure 2 In the query method for index records and data pages shown, the index file includes a current page index record information and a home page index record information. At this time, each slot identifier corresponds to a current page index record information and a home page index record information, so that after determining the slot identifier, the corresponding data page as the current write page, or the data page as the first write page, can be found from the index record information. The embodiment of the present invention does not make specific limitations.

[0098] An embodiment of the present invention provides a data storage method. Compared with the prior art, the embodiment of the present invention obtains a slot identifier of a target row and column data to be stored, wherein the slot identifier is generated by a distribution key of the target row; retrieves an index file, wherein the index file stores record information between different slot identifiers and data pages, and the data page is used to store data; determines a target data page corresponding to the slot identifier based on the record information, and stores the column data to be stored in the target data page, thereby realizing storage of column data after a mapping relationship is formed between the slot identifier and the page, thereby greatly improving the management efficiency of data storage, thereby improving the speed and accuracy of data reading and writing.

[0099] Furthermore, as a response to the above Figure 1 To implement the method shown, an embodiment of the present invention provides a data storage device, such as Figure 3 As shown, the device comprises:

[0100] An acquisition module 21 is used to acquire a slot identifier of a target row and column data to be stored, wherein the slot identifier is generated by a distribution key of the target row;

[0101] A calling module 22, used to call an index file, wherein the index file stores record information between different slot identifiers and data pages, and the data pages are used to store data;

[0102] The storage module 23 is used to determine a target data page corresponding to the slot identifier based on the record information, and store the column data to be stored in the target data page.

[0103] Furthermore, the storage module is specifically used to retrieve the target data page of the record information matching the slot identifier if the slot identifier is found from the index file; if the slot identifier is not found from the index file, generate the target data page of the slot identifier, and generate the record information of the slot identifier and the target data page, and store them in the index file.

[0104] Furthermore, the device also includes:

[0105] A creation module is used to create multiple data pages and index pages including public page headers and non-public page headers in a preset storage area; wherein the public page header stores page basic information, the non-public page header of the index page stores index level and index type, and the non-public page header of the data page stores slot identification and page attribute information.

[0106] Furthermore, the device further comprises: a generating module,

[0107] The acquisition module is further used to acquire the target index page corresponding to the target data page;

[0108] The generating module is used to generate a data index according to the storage information of the column data to be stored in the target data page;

[0109] The storage module is further used to store the data index in the target index page, and update the index level and index type of the index page based on the data index.

[0110] Furthermore, the target data page includes a first-written page and a currently written page.

[0111] The storage module is further configured to store the column data to be stored in the current write page if the slot identifier is found in the index file, and store the column data to be stored in the first write page if the slot identifier is not found in the index file.

[0112] Furthermore, the device also includes: a conversion module, which is used to respond to a storage request for the data table, retrieve the row data in the data table in units of rows and store them in the transaction buffer; when responding to a row data storage request submitted by the transaction buffer, convert the row data into column data to be stored.

[0113] Furthermore, the creation module is also used to respond to a read instruction of the target data, determine the read slot identifier of the target data, and call the index file; if the read slot identifier is queried in the index file, then based on all non-public page headers, query the reference data page matching the read slot identifier and the reference index page corresponding to the reference data page; query the storage information of the target data from the reference index page, and read the data content from the reference data page according to the storage information.

[0114] An embodiment of the present invention provides a data storage device. Compared with the prior art, the embodiment of the present invention obtains a slot identifier of a target row and column data to be stored, wherein the slot identifier is generated by a distribution key of the target row; retrieves an index file, wherein the index file stores record information between different slot identifiers and data pages, and the data page is used to store data; determines a target data page corresponding to the slot identifier based on the record information, and stores the column data to be stored in the target data page, thereby storing column data after a mapping relationship is formed between the slot identifier and the page, thereby greatly improving the management efficiency of data storage, thereby improving the speed and accuracy of data reading and writing.

[0115] According to an embodiment of the present invention, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the data storage method in any of the above method embodiments.

[0116] Figure 4 A schematic diagram of the structure of a terminal provided according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the terminal.

[0117] like Figure 4 As shown, the terminal may include: a processor (processor) 302, a communication interface (Communications Interface) 304, a memory (memory) 306, and a communication bus 308.

[0118] The processor 302 , the communication interface 304 , and the memory 306 communicate with each other via the communication bus 308 .

[0119] The communication interface 304 is used to communicate with other devices such as clients or other servers.

[0120] The processor 302 is used to execute the program 310, and specifically can execute the relevant steps in the above data storage method embodiment.

[0121] Specifically, the program 310 may include program codes, which include computer operation instructions.

[0122] The processor 302 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention. The one or more processors included in the terminal may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0123] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0124] The program 310 may be specifically configured to enable the processor 302 to perform the following operations:

[0125] Obtaining a slot identifier of a target row and column data to be stored, wherein the slot identifier is generated by a distribution key of the target row;

[0126] Retrieving an index file, wherein the index file stores record information between different slot identifiers and data pages, wherein the data pages are used to store data;

[0127] A target data page corresponding to the slot identifier is determined based on the record information, and the column data to be stored is stored in the target data page.

[0128] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data storage method, characterized in that: include: Obtaining a slot identifier of a target row and column data to be stored, wherein the slot identifier is generated by a distribution key of the target row; Retrieving an index file, wherein the index file stores record information between different slot identifiers and data pages, wherein the data pages are used to store data; A target data page corresponding to the slot identifier is determined based on the record information, and the column data to be stored is stored in the target data page.

2. The method according to claim 1, characterized in that Determining the target data page corresponding to the slot identifier based on the index file includes: If the slot identifier is found in the index file, the target data page of the record information matching the slot identifier is retrieved; If the slot identifier is not found in the index file, a target data page of the slot identifier is generated, and record information of the slot identifier and the target data page is generated and stored in the index file.

3. The method according to claim 2, characterized in that Before obtaining the slot identifier of the target row and the column data to be stored, the method further includes: Create multiple data pages and index pages including public page headers and non-public page headers in a preset storage area; The public page header stores basic page information, the non-public page header of the index page stores index level and index type, and the non-public page header of the data page stores slot identification and page attribute information.

4. The method according to claim 3, characterized in that After storing the to-be-stored column data in the target data page, the method further includes: Obtaining a target index page corresponding to the target data page; Generate a data index according to storage information of the column data to be stored in the target data page; The data index is stored in the target index page, and the index level and index type of the index page are updated based on the data index.

5. The method according to claim 3, characterized in that: The target data page includes a first write page and a current write page, and storing the to-be-stored column data into the target data page includes: If the slot identifier is found in the index file, the column data to be stored is stored in the current write page; If the slot identifier is not found in the index file, the column data to be stored is stored in the first write page.

6. The method according to claim 1, characterized in that Before obtaining the slot identifier of the target row and the column data to be stored, the method further includes: After responding to a storage request of the data table, the row data in the data table is retrieved and stored in the transaction buffer in row units; In response to a row data storage request submitted by the transaction buffer, the row data is converted into column data to be stored.

7. The method according to claim 1, characterized in that: After storing the to-be-stored column data in the target data page, the method further includes: In response to a read instruction of target data, determining a read slot identifier of the target data, and retrieving the index file; If the read slot identifier is found in the index file, a reference data page matching the read slot identifier and a reference index page corresponding to the reference data page are searched based on all non-public page headers; The storage information of the target data is queried from the reference index page, and the data content is read from the reference data page according to the storage information.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Creating a system page, wherein the system page stores home page information of all index pages; When the system is restarted, the system page is read, and all slot identifiers and data pages corresponding to the slot identifiers are read from the home page information to reconstruct the index file.

9. A data storage device, characterized in that: include: An acquisition module, used to acquire a slot identifier of a target row and column data to be stored, wherein the slot identifier is generated by a distribution key of the target row; A calling module, used to call an index file, wherein the index file stores record information between different slot identifiers and data pages, and the data pages are used to store data; A storage module is used to determine a target data page corresponding to the slot identifier based on the record information, and store the column data to be stored in the target data page.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 are implemented.

11. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claim 1.

Images