A data storage method, device, medium and product based on metadata

By including information such as primary key, column name, byte offset and bit offset in the metadata, the problem of wasted space in the prior art storage time sequence data is solved, and more efficient storage space utilization is achieved.

CN119377227BActive Publication Date: 2025-04-22BEIJING LE MA SHI INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411942293.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-27
Publication Date
2025-04-22
Estimated Expiration
2044-12-27

AI Technical Summary

Technical Problem

In the prior art, when storing timing data, metadata only includes primary keys and column names, resulting in the need to be allocated according to the maximum footprint during storage, resulting in wasted storage space and excessive footprint.

Method used

By including information such as primary key, column name, byte offset and bit offset in the metadata, different byte offsets and bit offsets are set according to the data type, thereby efficiently storing data.

Benefits of technology

It effectively reduces the use of storage space and saves the use of storage space, which has significantly improved compared with the prior art.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119377227B_ABST
    Figure CN119377227B_ABST
Patent Text Reader

Abstract

The present application discloses a data storage method, device, medium and product based on metadata. It relates to the field of data processing technology. The method includes: receiving a table creation request, the table creation request carries a first table name and a first metadata, the table creation request is used to request to establish a first metadata table for the data to be stored, and the first metadata table is used to store the first metadata; querying the first table name from the second metadata table, the second metadata table is an existing metadata table; if the first table name is not found, then establishing the first metadata table and parsing the first metadata to determine the content included in the first metadata; when the first metadata includes a primary key, a column name, a byte offset and a bit offset, the first table name, the primary key, the column name, the byte offset and the bit offset are stored in the first metadata table, so as to store the data to be stored through the first table name, the primary key, the column name, the byte offset and the bit offset. The present application reduces the space occupied by data storage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a metadata-based data storage method, device, medium and product. Background Art

[0002] With the development of Internet of Things technology, various intelligent devices and sensors have begun to be deployed in large numbers. These devices continuously collect information such as environment and equipment status and send it to the central system for processing. The concept of Industry 4.0 has promoted the development of manufacturing industry in a more intelligent direction, which has led to the generation of a large amount of real-time data, which often has time series characteristics.

[0003] With the advancement of technology, data collection has become easier, which also means that the amount of data has increased dramatically. Especially for continuous monitoring application scenarios, such as status monitoring of factory production lines, a large amount of time series data will be generated. Many applications require real-time analysis and processing of data so that timely decisions can be made. For example, the monitoring of rail transit systems requires real-time analysis of vehicle status to prevent failures.

[0004] Time series data usually involves the insertion of new data to store the newly added data. The usual method is as follows: First, set up a metadata table for the time series data to be stored. The metadata table stores the metadata of the data to be stored. The metadata is generally stored in the form of key-value pairs, where the key is the primary key and the value is the column name; then design the table name; after the metadata table is designed, the primary key and column name of the time series data to be stored can be stored in the metadata table; in order to ensure that each data to be stored can be stored, it will be allocated according to the data type with the largest space occupation, resulting in excessive storage space. Moreover, when the collected device data has tens of thousands of columns, a large amount of storage space will be wasted. Summary of the invention

[0005] The purpose of this application is to provide a metadata-based data storage method, device, medium and product to solve the problems described in the background technology.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a metadata-based data storage method, comprising:

[0008] receiving a table creation request, the table creation request carrying a first table name and first metadata, the table creation request being used to request creation of a first metadata table for the data to be stored, the first metadata table being used to store the first metadata;

[0009] Query the first table name from a second metadata table, where the second metadata table is an existing metadata table;

[0010] If not found, creating a first metadata table and parsing the first metadata to determine the content included in the first metadata;

[0011] When the first metadata includes a primary key, a column name, a byte offset and a bit offset, the first table name, the primary key, the column name, the byte offset and the bit offset are stored in the first metadata table to store the data to be stored through the first table name, the primary key, the column name, the byte offset and the bit offset.

[0012] Optionally, after querying the first table name from the second metadata table, and the second metadata table is an existing metadata table, the method further includes:

[0013] If found, a reminder is generated that the first table name already exists.

[0014] Optionally, after querying the first table name from a second metadata table, where the second metadata table is an existing metadata table, the method further includes:

[0015] When the first metadata includes a primary key, a column name, a data type, and a data length, arranging the first metadata according to the data type;

[0016] The primary key, column name, data type and data length are stored in the first metadata table according to the arranged order, so as to store the data to be stored by means of the primary key, column name, data type and data length.

[0017] Optionally, the data type includes a switch quantity, an enumeration quantity and a first data type, and the first data type is other data types included in the data to be stored except the switch quantity and the enumeration quantity; then when the first metadata includes a primary key, a column name, a data type and a data length, arranging the first metadata according to the data type includes: arranging the column names in the order of the switch quantity, the enumeration quantity and the first data type.

[0018] Optionally, the table creation request is further used to request the creation of a first actual data table for the data to be stored, and the first actual data table is used to store the data to be stored; then when the first metadata includes a primary key, a column name, a byte offset, and a bit offset, the first table name, the primary key, the column name, the byte offset, and the bit offset are stored in the first metadata table, so that the data to be stored is stored by using the first table name, the primary key, the column name, the byte offset, and the bit offset, the method further includes:

[0019] Creating the first actual data table;

[0020] storing the data to be stored in the first actual data table according to the first table name, primary key, column name, byte offset and bit offset;

[0021] Alternatively, the data to be stored is stored in the first actual data table according to the primary key, column name, data type and data length.

[0022] Optionally, when the first metadata includes a primary key, a column name, a byte offset, and a bit offset, after storing the first table name, the primary key, the column name, the byte offset, and the bit offset in the first metadata table to store the data to be stored by using the first table name, the primary key, the column name, the byte offset, and the bit offset, the method further includes:

[0023] receiving a data query request, wherein the query request carries a second table name;

[0024] Query the second table name in the second metadata table;

[0025] If found, obtaining the second metadata corresponding to the second table name;

[0026] When the second metadata includes a primary key, a column name, a byte offset, and a bit offset, querying data corresponding to the primary key, the column name, the byte offset, and the bit offset in a second actual data table to obtain target data, the second actual data table being a stored data table;

[0027] Alternatively, when the second metadata includes a primary key, a column name, a data type, and a data length, the data corresponding to the primary key, the column name, the data type, and the data length are queried in the second actual data table to obtain the target data.

[0028] Optionally, after obtaining the second metadata corresponding to the second table name if found, the method further includes:

[0029] When the terminal does not store the second metadata table, the second metadata table is sent to the terminal, so that the terminal stores the second metadata table.

[0030] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the methods described in the first aspect.

[0031] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0032] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the methods in the first aspect.

[0033] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0034] The metadata-based data storage method provided in the embodiment of the present application sets the first metadata for the data to be stored, and the first metadata includes the primary key, column name, byte offset and bit offset set in advance according to the data type, so that the data can be stored according to the primary key, column name, byte offset and bit offset, and different byte offsets and bit offsets are set for different data types to save the use of storage space. With respect to the prior art, the metadata includes the primary key and column name, so when performing data storage, in order to ensure that each data to be stored can be stored, it will be allocated according to the maximum occupied space, resulting in excessive storage space being occupied. Therefore, compared with the prior art, the present application reduces the occupancy of storage space and saves storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 This is an application environment diagram of a metadata-based data storage method in one embodiment of the present application;

[0037] Figure 2 This is another application environment diagram of a metadata-based data storage method in an embodiment of the present application;

[0038] Figure 3 A flowchart of a metadata-based data storage method provided in one embodiment of the present application;

[0039] Figure 4 A schematic diagram of a process flow of a metadata-based data storage method provided in yet another embodiment of the present application;

[0040] Figure 5 A schematic diagram of a flow chart of a metadata-based data storage method provided in another embodiment of the present application;

[0041] Figure 6 A flowchart of a metadata-based data storage method provided in one embodiment of the present application;

[0042] Figure 7A flowchart of a data query method provided in one embodiment of the present application;

[0043] Figure 8 A flowchart of a data query method provided in one embodiment of the present application;

[0044] Fig. 9 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0047] The metadata-based data storage method provided in the embodiment of the present application can be applied to Figure 1 and Figure 2 In the application environment shown, the application environment includes a terminal and a server. The terminal communicates with the server through a network. The data storage system can store data that the server needs to process. The data storage system can be set up separately, integrated on the server, or placed on the cloud or other servers. The terminal is installed with a client of the application, which can communicate with the server through the user interface of the client, and can send the actual data and metadata to be stored to the server. After the server receives the actual data and metadata, it first stores the metadata, and then stores the actual data according to the requirements of the metadata. The server can feedback the storage results to the terminal.

[0048] The terminals may be, but are not limited to, various desktop computers, laptops, smart phones, tablet computers, IoT devices and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers, or a cloud server.

[0049] Further, see Figure 2 The server in the present application may be an HBase server or an HBase server cluster, and the server stores the second metadata table and the second metadata.

[0050] Furthermore, the server is also configured with coprocessors, for example, MasterObserver to monitor table creation requests, and RegionObserver coprocessors to monitor data writing and query operations.

[0051] In an exemplary embodiment, Figure 3 As shown, combined with Figure 4 , provides a metadata-based data storage method, which is executed by a computer device, and can be specifically executed by a computer device such as a server. In the embodiment of the present application, the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 201 to 203:

[0052] Step 201: receiving a table creation request, wherein the table creation request carries a first table name and first metadata, and the table creation request is used to request to create a first metadata table for the data to be stored, and the first metadata table is used to store the first metadata.

[0053] When there is newly collected first actual data that needs to be stored in the server, the terminal first sends a table creation request to the server through the client to request the server to create a metadata table for the newly collected actual data. In this application, the new metadata table is denoted as the first metadata table. Further, the data to be stored is stored in the first metadata table.

[0054] The first table name is the table name of the first actual data table, and the first actual data is the data to be stored.

[0055] For example, the format of the table creation request is:

[0056] create'table_name', {NAME=>'cf', VERSIONS=>1};

[0057] METADATA=>{'door1_is_open'=>'boolean:0:0:1',

[0058] 'door2_is_open'=>'boolean:0:1:1'

[0059] 'door3_is_open'=>'boolean:0:1:1'

[0060] 'door4_is_open'=>'boolean:0:1:1'

[0061] 'door5_is_open'=>'boolean:0:1:1'

[0062] 'door6_is_open'=>'boolean:0:1:1'

[0063] 'door7_is_open'=>'boolean:0:1:1'

[0064] 'door8_is_open'=>'boolean:0:1:1'

[0065] 'speed'=>'unit:1:0:4'

[0066] 'temprature'=>'float:5:0:4'}

[0067] Among them, table_name is the table name;

[0068] METADATA is the metadata information of all column data in key-value format, where key is the column name, such as door1_is_open, value, and door2_is_open above; value is specific data, including data type, byte offset, bit offset, and data length of the field. For example, boolean:0:0:1 means that the data type is boolean, the byte offset is 0, the bit offset is 0, and the data length is 1.

[0069] The specific format is as follows:

[0070] 'col_name' => 'Data type: [byte offset]: [bit offset]: [data length]'

[0071] The number in the brackets is optional, but the data type is required. The supported data types are:

[0072] boolean, enum, uint8, int8, uint16, int16, uint32, int32, int64, float32, double64.

[0073] Step 202: query the first table name from a second metadata table, where the second metadata table is an existing metadata table.

[0074] For data stored on the server, a metadata table will be created for it when it is stored to facilitate data storage and query.

[0075] Step 203: If no metadata is found, a first metadata table is created and the first metadata is parsed to determine the content included in the first metadata.

[0076] The metadata is used to guide how to store the data to be stored.

[0077] If no query is found, it means that the server has not stored the first metadata table yet, and has not stored the to-be-stored data corresponding to the first metadata table.

[0078] Step 204: When the first metadata includes a primary key, a column name, a byte offset, and a bit offset, the first table name, the primary key, the column name, the byte offset, and the bit offset are stored in the first metadata table to store the data to be stored through the first table name, the primary key, the column name, the byte offset, and the bit offset.

[0079] Among them, the first table name, primary key, column name, byte offset and bit offset are set by the user in advance according to the data attributes of the data to be stored. For example, the byte offset and bit offset can be set in advance according to the data type of the data to be stored. For example, when the data to be stored is puppet type data, the bit offset set for each data to be stored increases sequentially. When the amount of data to be stored is greater than one byte, the byte offset can also be set in the first metadata to continue storing the data to be stored, thereby achieving the purpose of continuing to store data by increasing the byte offset.

[0080] Exemplarily, still taking the above example, after the table is created, the metadata table obtained is as shown in Table 1 below.

[0081] Table 1 Metadata table

[0082]

[0083] When the table creation request is successful, the client sends the data to be stored to the server, so that the server stores the data to be stored according to the first metadata table.

[0084] The metadata-based data storage method provided in the embodiment of the present application sets the first metadata for the data to be stored, and the first metadata includes the primary key, column name, byte offset and bit offset set in advance according to the data type, so that the data can be stored according to the primary key, column name, byte offset and bit offset, and different byte offsets and bit offsets are set for different data types to save the use of storage space. With respect to the prior art, the metadata includes the primary key and column name, so when performing data storage, in order to ensure that each data to be stored can be stored, it will be allocated according to the maximum occupied space, resulting in excessive storage space being occupied. Therefore, compared with the prior art, the present application reduces the occupancy of storage space and saves storage space.

[0085] Alternatively, see Figure 5 In another exemplary embodiment of the present application, after step 202, the method further includes the following step 301:

[0086] Step 301: If found, a reminder that the first table name already exists is generated.

[0087] If the first table name already exists, the same table name cannot be used for the convenience of storing and querying data. Therefore, it is necessary to generate a reminder that the first table name already exists to remind the user to change the first table name.

[0088] Alternatively, see Figure 5 In another exemplary embodiment of the present application, after step 203, the method further includes the following steps 401 and 402:

[0089] Step 401: When the first metadata includes a primary key, a column name, a data type, and a data length, the first metadata is arranged according to the data type.

[0090] Arranging the first metadata according to the data type can be understood as: arranging metadata of the same data type together, and after the arrangement of metadata of one data type is completed, arranging metadata of another data type.

[0091] Step 402: store the primary key, column name, data type and data length in the first metadata table according to the arranged order, so as to store the data to be stored by means of the primary key, column name, data type and data length.

[0092] In this embodiment, the first metadata does not include byte offset and bit offset, so the data to be stored can be directly stored according to the data type and data length.

[0093] Optionally, in another exemplary embodiment of the present application, the data type includes a switch quantity, an enumeration quantity and a first data type, and the first data type is other data types included in the data to be stored except the switch quantity and the enumeration quantity; then step 401 includes: arranging the column names in the order of switch quantity, enumeration quantity and the first data type.

[0094] Here, it can be understood that the data whose quantity type is switch quantity is arranged first, then the quantity type whose data type is enumeration quantity is arranged, and finally the data of the first data type is arranged.

[0095] Since the data of the switch quantity has only two possible values, it can be stored using one bit, so one byte can store 8 switch quantities. Compared with the prior art in which one switch quantity is stored using one byte, storage space is greatly saved.

[0096] Alternatively, see Figure 5 , and combined with Figure 6In another exemplary embodiment of the present application, the table creation request is also used to request the creation of a first actual data table for the data to be stored, and the first actual data table is used to store the data to be stored. Then after step 204, the method further includes the following steps 501, 502 and 503:

[0097] Step 501: Create the first actual data table.

[0098] Step 502: store the data to be stored in the first actual data table according to the first table name, primary key, column name, byte offset and bit offset.

[0099] Alternatively, in step 503, the data to be stored is stored in the first actual data table according to the primary key, column name, data type and data length.

[0100] For step 502, since the first metadata includes a byte offset and a bit offset, when storing data, the byte position is first determined according to the byte offset, and then the bit in the byte is determined according to the bit offset, thereby realizing data storage.

[0101] For step 503, the first metadata has a data type, so it is stored according to the data type. For example, data with a switch data type is stored first, then data with an enumeration data type is stored, and finally other types of data are stored.

[0102] For example, in combination Figure 6 ,The metadata-based data storage method also includes the following process:

[0103] Step 101: The terminal receives data to be stored sent by a collection device. The data to be stored carries a data name. When storing the data, the data name is used as a table name.

[0104] In step 102, the terminal queries whether the terminal itself has cached the metadata table of the data to be stored according to the data name; if it is stored, the data to be stored is stored at the location specified by the metadata in the metadata table, thereby converting the data into bytes, and executing step 104; if it is not stored, a data search request is sent to the server to determine whether the metadata of the data to be stored is stored in the server, and executing step 103.

[0105] Step 103, if the metadata of the data to be stored is stored in the server, the metadata is pulled from the server and cached locally in the terminal. The terminal then stores the data to be stored at the location specified by the metadata in the metadata table, thereby converting the data into bytes, and then executing step 104; if the metadata of the data to be stored is not stored in the server, a non-storage reminder is output to remind the user to re-create a table for the data to be stored.

[0106] Step 104 , the terminal determines whether the data to be stored needs to be sent in batches. If so, step 105 is executed; if not, the data to be stored is directly sent to the server, and step 106 is executed.

[0107] In step 105, the terminal stores the data to be stored locally in the terminal, and sends it to the server when the cache upper limit is reached or the local storage time in the terminal reaches a predetermined time, and executes step 106; if the cache upper limit is not reached and the local storage time in the terminal does not reach the predetermined time, repeat step 101.

[0108] Step 106, determine whether it is necessary to continue storing data in the server; if so, re-execute step 101; if not, end.

[0109] Alternatively, see Figure 7 , and combined with Figure 8 In another exemplary embodiment of the present application, after step 204, the method further includes steps 601 to 605 as follows:

[0110] Step 601: Receive a data query request, where the query request carries a second table name.

[0111] Step 602: Search the second metadata table for the second table name.

[0112] Step 603: If found, obtain the second metadata corresponding to the second table name.

[0113] If the second table name is found, the terminal may also determine whether the second metadata is stored in the terminal. If not, the second metadata is pulled from the server and stored locally in the terminal.

[0114] If no data is found, an error reminder is generated to remind the user that no data corresponding to the second table name is stored in the server.

[0115] Step 604: when the second metadata includes a primary key, a column name, a byte offset, and a bit offset, the second actual data table is queried for data corresponding to the primary key, the column name, the byte offset, and the bit offset to obtain target data.

[0116] The second actual data is all the actual data stored in the server.

[0117] Alternatively, in step 605, when the second metadata includes a primary key, a column name, a data type, and a data length, data corresponding to the primary key, the column name, the data type, and the data length are queried in the second actual data table to obtain target data.

[0118] Exemplarily, the server stores the three basic information of birth date, address, and height of three students named A, B, and C. The query request received is "query the height of student A". Then, when performing data query, the second table name is first queried from the second metadata table; then the byte offset and bit offset corresponding to the height of student A are queried from the metadata corresponding to the second table name; the data to be queried can be directly located according to the byte offset and bit offset, without traversing other data. With the prior art, the second table name is first queried from the second metadata table; then the keyword "A" of student is queried from the second table name, and no more specific information can be queried; then the data corresponding to A needs to be queried in the second actual data according to the keyword "A", and the data corresponding to A also includes birth date, address, and height. Further, according to the keyword "height", "birth date, address, height" are traversed in sequence until the specific data corresponding to "height" is found.

[0119] Alternatively, see Figure 7 , and combined with Figure 8 In another exemplary embodiment of the present application, after step 603, the method further includes the following step 701:

[0120] Step 701: When the terminal does not store the second metadata table, the second metadata table is sent to the terminal, so that the terminal stores the second metadata table.

[0121] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data storage related data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a metadata-based data storage method can be implemented.

[0122] Those skilled in the art will understand that Fig. 9The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0123] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0124] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0125] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0127] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0128] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A data storage method based on metadata, characterized in that: include: receiving a table creation request, the table creation request carrying a first table name and first metadata, the table creation request being used to request creation of a first metadata table for the data to be stored, the first metadata table being used to store the first metadata; Query the first table name from a second metadata table, where the second metadata table is an existing metadata table; If not found, creating a first metadata table and parsing the first metadata to determine the content included in the first metadata; When the first metadata includes a primary key, a column name, a byte offset, and a bit offset, storing the first table name, the primary key, the column name, the byte offset, and the bit offset in the first metadata table, so as to store the data to be stored by using the first table name, the primary key, the column name, the byte offset, and the bit offset; The storing the data to be stored comprises: According to the data type of the data to be stored, a byte offset and a bit offset are set in advance for the data to be stored; The storage is increased in sequence according to the bit offsets set for each data to be stored. When the amount of data to be stored is greater than one byte, the byte offset is increased and the data to be stored is continued to be stored to achieve the purpose of storing data.

2. The metadata-based data storage method according to claim 1, characterized in that: After querying the first table name from the second metadata table, and the second metadata table is an existing metadata table, the method further includes: When the first metadata includes a primary key, a column name, a data type, and a data length, arranging the first metadata according to the data type; The primary key, column name, data type and data length are stored in the first metadata table according to the arranged order, so as to store the data to be stored by means of the primary key, column name, data type and data length.

3. The metadata-based data storage method according to claim 2, characterized in that: The data types include switch quantities, enumeration quantities and first data types, and the first data type is other data types included in the data to be stored except the switch quantities and enumeration quantities; then when the first metadata includes a primary key, a column name, a data type and a data length, arranging the first metadata according to the data type includes: arranging the column names in the order of switch quantities, enumeration quantities and the first data type.

4. The metadata-based data storage method according to claim 1, characterized in that: The table creation request is also used to request to create a first actual data table for the data to be stored, and the first actual data table is used to store the data to be stored; then when the first metadata includes a primary key, a column name, a byte offset, and a bit offset, the first table name, the primary key, the column name, the byte offset, and the bit offset are stored in the first metadata table, so that the data to be stored is stored by the first table name, the primary key, the column name, the byte offset, and the bit offset, the method further includes: Creating the first actual data table; storing the data to be stored in the first actual data table according to the first table name, primary key, column name, byte offset and bit offset; Alternatively, the data to be stored is stored in the first actual data table according to the primary key, column name, data type and data length.

5. The metadata-based data storage method according to claim 1, characterized in that: When the first metadata includes a primary key, a column name, a byte offset, and a bit offset, after storing the first table name, the primary key, the column name, the byte offset, and the bit offset in the first metadata table to store the data to be stored by using the first table name, the primary key, the column name, the byte offset, and the bit offset, the method further includes: receiving a data query request, wherein the query request carries a second table name; Query the second table name in the second metadata table; If found, obtaining the second metadata corresponding to the second table name; When the second metadata includes a primary key, a column name, a byte offset, and a bit offset, querying data corresponding to the primary key, the column name, the byte offset, and the bit offset in a second actual data table to obtain target data, the second actual data table being a stored data table; Alternatively, when the second metadata includes a primary key, a column name, a data type, and a data length, the data corresponding to the primary key, the column name, the data type, and the data length are queried in the second actual data table to obtain the target data.

6. The metadata-based data storage method according to claim 5, characterized in that: After obtaining the second metadata corresponding to the second table name if found, the method further includes: When the terminal does not store the second metadata table, the second metadata table is sent to the terminal, so that the terminal stores the second metadata table.

7. The metadata-based data storage method according to any one of claims 1 to 6, characterized in that: After querying the first table name from the second metadata table and the second metadata table is an existing metadata table, the method further includes: If the first table name is found in the second metadata table, a reminder that the first table name already exists is generated.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the metadata-based data storage method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the metadata-based data storage method described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the metadata-based data storage method described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Method and device for high-performance query based on Hbase row keys

    CN106528674A

  • SQL-based data storage method and device, equipment and storage medium

    CN108170824A