Method, device and equipment for storing column-stored data in database and medium

Abstract

The invention discloses a method, a device and equipment for storing column-stored data in a database and a storage medium. The method comprises the following steps of: writing a to-be-stored data table; storing one column according to the column head of each column of data in the to-be-stored data table to form an original storage column; generating fusion data based on data corresponding to each row in the data table to be stored, and performing column storage on the fusion data to form a fusion storage column; and taking the fused storage column as a new column, placing the new column in the original storage column, generating a fused column storage table, and storing the fused column storage table in a database. By means of the method, on the basis of original column storage data, a new fusion column is introduced, multiple columns of a data table are fused together in the new fusion column, during multi-column query, data can be preferably obtained from the fusion column, and the access performance of multi-column query in a column storage mode is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of database technology, and in particular to a method, device, equipment and storage medium for storing column-based data in a database. Background Art

[0002] In data analysis scenarios, more and more data analysis scenarios use column storage. Traditional column-based databases have very good results in analysis of a small number of columns. However, when performing ad hoc queries on multiple columns, column-based databases have very high random I / O and poor results due to the scattered storage of different columns. Summary of the invention

[0003] The embodiments of the present invention provide a method, apparatus, device and storage medium for storing column-based data in a database, which solves the problem that when a column-based database is subjected to ad hoc queries on multiple columns, the random I / O is very high and the effect is poor due to the dispersed storage of different columns.

[0004] In a first aspect, an embodiment of the present invention provides a method for storing column-based data in a database, comprising:

[0005] Write the data table to be stored;

[0006] Store one column according to the column header of each column of data in the data table to be stored, to form an original storage column;

[0007] Generate fused data based on the data corresponding to each row in the data table to be stored, and store the fused data in columns to form fused storage columns;

[0008] The fused storage column is placed as a new column after the original storage column to generate a fused column storage table, and the fused column storage table is stored in a database.

[0009] In a second aspect, an embodiment of the present invention further provides a device for storing column-based data in a database, the device comprising:

[0010] A data writing module, used for writing data into the data table to be stored;

[0011] An original column data generating module, used for storing one column according to the column header of each column of data in the data table to be stored, to form an original storage column;

[0012] A fused column data generating module, used to generate fused data based on the data corresponding to each row in the data table to be stored, and store the fused data in columns to form fused storage columns;

[0013] The data storage module is used to place the fused storage column as a new column after the original storage column to generate a fused column storage table, and store the fused column storage table in a database.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0015] one or more processors;

[0016] a storage device for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for storing data in columns in a database provided by an embodiment of the present disclosure.

[0018] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to implement the method for storing column-based data in a database provided by the embodiments of the present disclosure.

[0019] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for storing data in columns in a database provided by an embodiment of the present disclosure.

[0020] The present invention discloses a method, device, equipment and storage medium for storing column-based data in a database, the method comprising: writing a data table to be stored; storing a column according to the column header of each column of data in the data table to be stored to form an original storage column; generating fused data based on the data corresponding to each row in the data table to be stored, and storing the fused data in columns to form a fused storage column; placing the fused storage column as a new column after the original storage column to generate a fused column storage table, and storing the fused column storage table in the database. Utilizing this method: on the basis of the original column-based data, a new fused column is introduced, in which multiple columns of the data table are fused together, and when performing a query on multiple columns, data can be preferably obtained from this fused column, thereby improving the access performance of querying multiple columns in the column storage mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart of a method for storing data in columns in a database provided by an embodiment of the present disclosure;

[0023] Figure 2 An example diagram of a data storage method provided by an embodiment of the present disclosure;

[0024] Figure 3 An example diagram of another data storage method provided by an embodiment of the present disclosure;

[0025] Figure 4 An example diagram of another data storage method provided by an embodiment of the present disclosure;

[0026] Figure 5 An example diagram of another data storage method provided by an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of the structure of a device for storing data in columns in a database provided by an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0035] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0036] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0037] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0038] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0039] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0040] Embodiment 1

[0041] Figure 1 A flowchart of a method for storing column-based data in a database provided in an embodiment of the present disclosure is applicable to providing a solution to the problem that when a column-based database is subjected to ad hoc queries on multiple columns, the random I / O is very high and the effect is poor due to the scattered storage of different columns. The method can be executed by a method device for storing column-based data in a database, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC, a server, etc.

[0042] like Figure 1 As shown, a method for storing column-based data in a database provided by an embodiment of the present disclosure may specifically include the following steps:

[0043] S110, writing the data table to be stored.

[0044] In this embodiment, the data table to be stored may be a data table waiting to be stored in a database.

[0045] This step is used to write the data table to be stored

[0046] S120 , storing one column according to the column header of each column of data in the data table to be stored, to form an original storage column.

[0047] In this embodiment, the column header may be an attribute of each column or the name of the column. Figure 2 This is an example diagram of a method for data storage provided by an embodiment of the present disclosure. Figure 2 As shown, each column of data in the data table to be stored is stored in a column according to the column header to form an original storage column. Figure 2 The first four columns in are the original storage columns.

[0048] On the basis of the above embodiment, storing one column according to the column header of each column of data in the data table to be stored, and forming an original storage column includes the following steps:

[0049] a1) obtaining a preset first column header set; the first column header set includes at least one column header in a data table to be stored.

[0050] b1) The column data corresponding to each column header in the first column header set in the data table to be stored is used as the original column data.

[0051] c1) storing the original column data in each column according to the column header to form an original storage column.

[0052] In this embodiment, the first column header set may be a set of column headers of columns for which data analysis is to be performed.

[0053] Specifically, a preset first column header set is obtained; the first column header set includes at least one column header in the data table to be stored, and for each row of data written, the column data corresponding to the column headers in the first column header set in the data table to be stored is used as the original column data, and each column of the original column data is stored in one column according to the column header to form an original storage column.

[0054] For example, Figure 2 This is an example diagram of another method for data storage provided by an embodiment of the present disclosure, such as Figure 2As shown, the first, second, third, and fourth columns are all original columns, the fifth column is a fusion column of the first and second columns, and the sixth column is a fusion column of the third and fourth columns.

[0055] For example, Figure 3 This is an example diagram of another method for data storage provided by an embodiment of the present disclosure, such as Figure 3 As shown, the first column header set includes the first and third column headers, and the column data corresponding to the column headers in the first column header set in the data table to be stored is used as the original column data, and one column is stored according to the column header of each column of the original column data to form an original storage column.

[0056] Using this method, only the columns that need to be analyzed can be stored by column, and the data that does not need to be analyzed by column can be stored in the fused column. The sum of the data of all columns cannot be missing any column of the original data, saving storage space overhead.

[0057] S130 , generating fused data based on the data corresponding to each row in the data table to be stored, and storing the fused data in columns to form fused storage columns.

[0058] In this embodiment, the data corresponding to each row in the data table to be stored are connected in sequence to generate fused data, and the fused data is stored in the corresponding columns in each fused storage column.

[0059] For example, Figure 3 As shown, the fifth column is a fusion storage column.

[0060] On the basis of the above embodiment, generating fused data based on the data corresponding to each row in the data table to be stored, and storing the fused data in columns, forming a fused storage column includes the following steps:

[0061] a2) Obtaining a set of column headers in the data table to be stored that is preset in each fusion storage column.

[0062] b2) For each row of data written, group each row of data in the data table to be stored according to the set of column headers to generate group data corresponding to each fused storage column.

[0063] c2) Organize each group of data in sequence and generate corresponding fused data.

[0064] d2) storing the corresponding columns of each fused data into each fused storage column.

[0065] Specifically, obtain the set of column headers in the data table to be stored that is pre-set in each fusion storage column. Group each row of data in the data table to be stored according to the set of column headers, and generate group data corresponding to each fusion storage column. Connect each group of data in sequence to generate corresponding fusion data. Store each fusion data in a corresponding column in each fusion storage column. There are many ways to organize data in the database, such as: json format, delimiter separation, xml format, TLV method, etc.

[0066] For example, Figure 5 This is an example diagram of another data storage method provided by an embodiment of the present disclosure. The data of columns 1-4 are preset, the data of columns 1 and 2 corresponding to the same row are fused to generate a fifth column as a fused storage column, and the data of columns 3 and 4 corresponding to the same row are fused to generate a fifth column as another fused storage column, such as Figure 5 As shown, there can be multiple fused columns, and the fifth and sixth columns are fused data columns.

[0067] By using this method, data is divided into multiple fused columns according to the frequency of use, and the sum of data of all columns cannot be missing from any column of the original data, thereby reducing unnecessary disk I / O overhead during access.

[0068] S140: Place the fused storage column as a new column after the original storage column to generate a fused column storage table, and store the fused column storage table in a database.

[0069] Specifically, the fused storage column is placed as a new column after the original storage column to generate a fused column storage table, and the fused column storage table is stored in the database. Figure 2 , Figure 3 , Figure 4 and Figure 5 Both are fused column storage tables.

[0070] On the basis of the above embodiment, fused data is generated based on the data corresponding to each row in the data table to be stored, and the fused data is stored in columns. Before forming the fused storage column, the following steps are also included:

[0071] a3) obtaining a preset second column header set; the second column header set includes at least one column header in the data table to be stored.

[0072] b3) taking the column data corresponding to each column header in the second column header set in the data table to be stored as a new data table to be stored.

[0073] In this embodiment, the first column header set may be a set of column headers of data columns that do not need to be analyzed by column.

[0074] Specifically, a preset second column header set is obtained; the second column header set includes at least one column header in the data table to be stored, and column data corresponding to each column header in the second column header set in the data table to be stored is used as a new data table to be stored.

[0075] Based on the above embodiment, ad hoc query of data in the database includes the following steps:

[0076] a4) Obtain ad hoc query information generated based on user needs.

[0077] b4) Determine the fused column storage table corresponding to the ad hoc query information.

[0078] c4) Determine the I / O cost of executing ad hoc query information when the access objects are the fused storage column and the original storage column respectively.

[0079] d4) The object with the smallest I / O cost is selected as the access object corresponding to the ad hoc query information.

[0080] In this embodiment, the ad hoc query information is a non-predefined query performed by a user in a data warehouse or database according to an immediate demand.

[0081] Specifically, obtain ad hoc query information generated according to user needs. Determine the fused column storage table corresponding to the ad hoc query information. Determine the I / O cost of executing the ad hoc query information when the access objects are the fused storage column and the original storage column respectively. The access object with the smallest I / O cost is selected as the access object corresponding to the execution of the ad hoc query information.

[0082] Based on the above embodiment, determining the fused column storage table corresponding to the ad hoc query information includes the following steps:

[0083] b41) extracting SQL statements from ad hoc query information;

[0084] b42) Parsing the SQL statement according to the preset SQL parsing engine to obtain the corresponding fusion column storage table in the SQL statement.

[0085] Specifically, an SQL statement in the ad hoc query information is extracted, an SQL parsing engine is generated according to the SQL grammar, and the SQL statement is parsed by the SQL parsing engine to obtain a fused column storage table in the SQL statement.

[0086] The present invention discloses a method for storing column-based data in a database, the method comprising: writing a data table to be stored; storing a column according to the column header of each column of data in the data table to be stored to form an original storage column; generating fused data based on the data corresponding to each row in the data table to be stored, and storing the fused data in columns to form a fused storage column; placing the fused storage column as a new column after the original storage column to generate a fused column storage table, and storing the fused column storage table in the database. Using this method: on the basis of the original column-based data, a new fused column is introduced, and multiple columns of the data table are fused together in the new fused column. When performing a query on multiple columns, data can be preferably obtained from the fused column, thereby improving the access performance of querying multiple columns in the column storage mode.

[0087] Figure 6 The present invention also provides a schematic diagram of a method and device structure for storing data in columns in a database. Figure 6 As shown, a data writing module 210, an original column data generating module 220, a fused column data generating module 230 and a data storage module 240.

[0088] A data writing module 210 is used to write data into a data table to be stored;

[0089] The original column data generating module 220 is used to store one column according to the column header of each column of data in the data table to be stored, so as to form an original storage column;

[0090] A fused column data generating module 230 is used to generate fused data based on the data corresponding to each row in the data table to be stored, and store the fused data in columns to form fused storage columns;

[0091] The data storage module 240 is configured to place the fused storage column as a new column after the original storage column to generate a fused column storage table, and store the fused column storage table in a database.

[0092] The technical solution provided by the embodiment of the present disclosure utilizes this method: based on the original column storage data, a new fused column is introduced, in which multiple columns of the data table are merged together. When performing multi-column queries, data can be preferably obtained from this fused column, thereby improving the access performance of querying multiple columns in the column storage mode.

[0093] Furthermore, the original column data generating module 220 may be used to:

[0094] Obtaining a preset first column header set; the first column header set includes at least one column header in the data table to be stored;

[0095] Using the column data corresponding to each column header in the first column header set in the data table to be stored as the original column data;

[0096] One column is stored according to the column header of each column of original column data to form the original storage column.

[0097] Furthermore, the fused column data generating module 230 may be used to:

[0098] Acquire a set of column headers in the data table to be stored that are preset in each fused storage column;

[0099] Grouping each row of data in the data table to be stored according to the set of column headers to generate group data corresponding to each fused storage column;

[0100] Connecting the groups of data in sequence respectively to generate corresponding fused data;

[0101] The corresponding columns of each fusion data are stored in each fusion storage column.

[0102] Furthermore, the device can also be used for:

[0103] Before generating fused data based on the data corresponding to each row in the data table to be stored and storing the fused data in columns to form fused storage columns, the method further includes:

[0104] Obtaining a preset second column header set; the second column header set includes at least one column header in the data table to be stored;

[0105] The column data corresponding to each column header in the second column header set in the data table to be stored is used as the new data table to be stored.

[0106] Furthermore, the device can also be used for:

[0107] Also included is ad hoc querying of data in the database, including:

[0108] Obtain ad hoc query information generated based on user needs;

[0109] Determine the fused column storage table corresponding to the ad hoc query information;

[0110] Determine an I / O cost of executing the ad hoc query information when access objects are respectively the fused storage column and the original storage column;

[0111] The access object with the smallest I / O overhead is used as the access object corresponding to the ad hoc query information.

[0112] Furthermore, the device can also be used for:

[0113] Extracting SQL statements from the ad hoc query information;

[0114] The SQL statement is parsed according to a preset SQL parsing engine to obtain a corresponding fused column storage table in the SQL statement.

[0115] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not described in detail in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0116] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is given. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0117] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0118] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a method for storing data in columns in a database.

[0120] In some embodiments, the method of storing data in columns in a database may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method of storing data in columns in a database described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method of storing data in columns in a database by any other appropriate means (e.g., by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0123] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0126] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for storing column data in a database, characterized in that: include: Write the data table to be stored; Store one column according to the column header of each column of data in the data table to be stored, to form an original storage column; Generate fused data based on the data corresponding to each row in the data table to be stored, and store the fused data in columns to form fused storage columns; The fused storage column is placed as a new column after the original storage column to generate a fused column storage table, and the fused column storage table is stored in a database.

2. The method according to claim 1, characterized in that The step of storing one column according to the column header of each column of data in the data table to be stored to form an original storage column includes: Obtaining a preset first column header set; the first column header set includes at least one column header in the data table to be stored; Using the column data corresponding to each column header in the first column header set in the data table to be stored as the original column data; For each row of data written, one column is stored according to the column header of each column of original column data to form the original storage column.

3. The method according to claim 1, characterized in that The generating fused data based on the data corresponding to each row in the data table to be stored, and storing the fused data in columns to form fused storage columns, includes: Acquire a set of column headers in the data table to be stored that are preset in each fused storage column; For each row of data written, grouping each row of data in the data table to be stored according to the set of column headers, and generating group data corresponding to each fused storage column; Organizing the groups of data in sequence respectively, and generating corresponding fused data; The corresponding columns of each fusion data are stored in each fusion storage column.

4. The method according to claim 1, characterized in that: Before generating fused data based on the data corresponding to each row in the data table to be stored and storing the fused data in columns to form fused storage columns, the method further includes: Obtaining a preset second column header set; the second column header set includes at least one column header in the data table to be stored; The column data corresponding to each column header in the second column header set in the data table to be stored is used as the new data table to be stored.

5. The method according to claim 1, characterized in that Also included is ad hoc querying of data in the database, including: Obtain ad hoc query information generated based on user needs; Determine the fused column storage table corresponding to the ad hoc query information; Determine an I / O cost of executing the ad hoc query information when access objects are respectively the fused storage column and the original storage column; The access object with the smallest I / O overhead is used as the access object corresponding to the ad hoc query information.

6. The method according to claim 5, characterized in that The determining the fused column storage table corresponding to the ad hoc query information includes: Extracting SQL statements from the ad hoc query information; The SQL statement is parsed according to a preset SQL parsing engine to obtain a corresponding fused column storage table in the SQL statement.

7. A device for storing data in columns in a database, characterized in that: include: A data writing module, used for writing data into the data table to be stored; An original column data generating module, used for storing one column according to the column header of each column of data in the data table to be stored, to form an original storage column; A fused column data generating module, used to generate fused data based on the data corresponding to each row in the data table to be stored, and store the fused data in columns to form fused storage columns; The data storage module is used to place the fused storage column as a new column after the original storage column to generate a fused column storage table, and store the fused column storage table in a database.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for storing data in columns in a database according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for storing column-based data in a database according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the method for storing data in columns in a database according to any one of claims 1 to 6.

Images