Data table index scheme generation method and device, computer equipment and storage medium

Abstract

The invention relates to a data table index scheme generation method and device, computer equipment, a storage medium and a computer program product, which can be applied to data table query in a data storage system. The method comprises the following steps: receiving a plurality of scanning events aiming at a data table; the scanning event is used for recording a filtering condition used when the data table is queried; respectively extracting filtering column sets respectively corresponding to the plurality of scanning events from the filtering conditions respectively corresponding to the plurality of scanning events; aiming at each scanning event in the plurality of scanning events, enumerating a combination formed by at least one element in the filtering column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event; counting the number of scanning events hit by each candidate column index scheme according to the candidate column index schemes corresponding to the plurality of scanning events; and selecting an index scheme of the data table from each candidate column index scheme. By adopting the method, the data search efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer equipment, storage medium and computer program product for generating a data table index scheme. Background Art

[0002] With the development of computer technology, data tables in data storage systems are used more and more widely. In data search, data tables can be used to quickly obtain the data users need.

[0003] In the traditional technology, in the process of realizing the data search function of the data table, the computing engine in the data storage system storing the data table will use the constructed index and the received index field to search the data. The index is a separate, physical storage structure that sorts the values ​​of one or more columns in the data table of the data storage system. The role of the index is equivalent to the catalog of a book.

[0004] However, traditional methods usually build indexes according to common index fields. If the built index is inaccurate, it will lead to low data search efficiency. Summary of the invention

[0005] Based on this, it is necessary to provide a data table index scheme generation method, device, computer equipment, computer-readable storage medium and computer program product that can improve data search efficiency in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for generating a data table index scheme. The method comprises:

[0007] Receiving a plurality of scanning events for a data table; the scanning events are used to record filtering conditions used when querying the data table;

[0008] Extracting, from the filter conditions corresponding to the plurality of scanning events, the filter column sets corresponding to the plurality of scanning events respectively;

[0009] For each scanning event in the multiple scanning events, enumerate a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event;

[0010] According to the candidate column index schemes corresponding to the plurality of scan events, counting the number of scan events hit by each candidate column index scheme;

[0011] Based on the number of scan events hit by each of the candidate column indexing schemes, an indexing scheme for the data table is selected from each of the candidate column indexing schemes.

[0012] In a second aspect, the present application also provides a data table index scheme generation device. The device comprises:

[0013] A receiving module, used for receiving a plurality of scanning events for a data table; the scanning events are used for recording filtering conditions used when querying the data table;

[0014] An extraction module, configured to extract, from the filter conditions corresponding to the plurality of scanning events, filter column sets corresponding to the plurality of scanning events respectively;

[0015] An enumeration module is used to enumerate, for each scanning event in the multiple scanning events, a combination of at least one element in a filter column set corresponding to the scanning event, and obtain a candidate column index scheme corresponding to the scanning event;

[0016] A statistics module, configured to count the number of scan events hit by each candidate column index scheme according to the candidate column index schemes corresponding to the multiple scan events;

[0017] A selection module is used to select an index scheme for the data table from each of the candidate column index schemes based on the number of scan events hit by each of the candidate column index schemes.

[0018] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0019] Receiving a plurality of scanning events for a data table; the scanning events are used to record filtering conditions used when querying the data table;

[0020] Extracting, from the filter conditions corresponding to the plurality of scanning events, the filter column sets corresponding to the plurality of scanning events respectively;

[0021] For each scanning event in the multiple scanning events, enumerate a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event;

[0022] According to the candidate column index schemes corresponding to the plurality of scan events, counting the number of scan events hit by each candidate column index scheme;

[0023] Based on the number of scan events hit by each of the candidate column indexing schemes, an indexing scheme for the data table is selected from each of the candidate column indexing schemes.

[0024] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0025] Receiving a plurality of scanning events for a data table; the scanning events are used to record filtering conditions used when querying the data table;

[0026] Extracting, from the filter conditions corresponding to the plurality of scanning events, the filter column sets corresponding to the plurality of scanning events respectively;

[0027] For each scanning event in the multiple scanning events, enumerate a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event;

[0028] According to the candidate column index schemes corresponding to the multiple scan events, count the number of scan events hit by each candidate column index scheme;

[0029] Based on the number of scan events hit by each of the candidate column indexing schemes, an indexing scheme for the data table is selected from each of the candidate column indexing schemes.

[0030] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0031] Receiving a plurality of scanning events for a data table; the scanning events are used to record filtering conditions used when querying the data table;

[0032] Extracting, from the filter conditions corresponding to the plurality of scanning events, the filter column sets corresponding to the plurality of scanning events respectively;

[0033] For each scanning event in the multiple scanning events, enumerate a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event;

[0034] According to the candidate column index schemes corresponding to the multiple scan events, count the number of scan events hit by each candidate column index scheme;

[0035] Based on the number of scan events hit by each of the candidate column indexing schemes, an indexing scheme for the data table is selected from each of the candidate column indexing schemes.

[0036] The above-mentioned data table index scheme generation method, device, computer equipment, storage medium and computer program product can obtain the filter column set used by the data table when being queried by receiving multiple scan events for the data table, extracting the filter column set corresponding to each of the multiple scan events from the filter conditions corresponding to each of the multiple scan events, and obtaining the candidate column index scheme corresponding to the scan event by enumerating the combination of at least one element in the filter column set corresponding to the scan event for each of the multiple scan events, and can obtain the candidate column index scheme corresponding to each of the multiple scan events by analyzing the filter column set used by the data table when being queried, and then can count the number of scan events hit by each candidate column index scheme based on the candidate column index scheme corresponding to each of the multiple scan events, and select the index scheme of the data table from each candidate column index scheme based on the number of scan events hit by each candidate column index scheme. In the whole process, the index scheme of the data table can be generated by analyzing the usage and structural characteristics of the data table, and the generated index scheme of the data table will be more in line with the data search requirements of the data table, and can improve the data search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 An application environment diagram of a method for generating a data table index scheme in an embodiment;

[0038] Figure 2 A schematic diagram of a flow chart of a method for generating a data table index scheme in one embodiment;

[0039] Figure 3 An architectural diagram of a scanning event report in one embodiment;

[0040] Figure 4 This is a diagram of a data lake engine kernel event reporting architecture in one embodiment;

[0041] Figure 5 A schematic diagram of an index optimization prompt in one embodiment;

[0042] Figure 6 A schematic diagram of index optimization tips in another embodiment;

[0043] Figure 7 A schematic diagram showing an option of adding an index field in one embodiment;

[0044] Figure 8 A schematic diagram showing an option of adding an index field in another embodiment;

[0045] Fig. 9 A schematic diagram of a flow chart of a method for generating a data table index scheme in another embodiment;

[0046] Fig.10A schematic diagram of a set of filter columns corresponding to a scan event in an embodiment;

[0047] Fig.11 A schematic diagram of a flow chart of a method for generating a data table index scheme in yet another embodiment;

[0048] Fig.12 A schematic diagram of interface changes of a display interface in an embodiment;

[0049] Fig.13 A schematic diagram of a flow chart of a method for generating a data table index scheme in yet another embodiment;

[0050] Fig.14 It is a structural block diagram of a data table index scheme generating device in one embodiment;

[0051] Fig.15 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] The data table index scheme generation method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the server 102 communicates with the data storage system 104 through the network. The data storage system 104 stores the data that the server 102 needs to process, that is, multiple scan events for the data table. The data storage system 104 can be integrated on the server 102, or it can be placed on the cloud or other servers. The server 102 receives multiple scan events for the data table from the data storage system 104. The scan event is used to record the filter conditions used when querying the data table. From the filter conditions corresponding to each of the multiple scan events, the filter column sets corresponding to the multiple scan events are respectively extracted. For each of the multiple scan events, a combination of at least one element in the filter column set corresponding to the scan event is enumerated to obtain the candidate column index scheme corresponding to the scan event. According to the candidate column index schemes corresponding to the multiple scan events, the number of scan events hit by each candidate column index scheme is counted, and based on the number of scan events hit by each candidate column index scheme, the index scheme of the data table is selected from each candidate column index scheme. Among them, server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0054] In one embodiment, Figure 2 As shown, a method for generating a data table index scheme is provided, which can be executed by a terminal or a server alone, or by a terminal and a server in collaboration. In the embodiment of the present application, the method is applied to a server as an example for illustration, and includes the following steps:

[0055] Step 202: Receive multiple scan events for the data table; the scan event is used to record the filter conditions used when querying the data table.

[0056] Among them, the data table refers to a data structure stored in a data storage system for storing data. For example, the data table in this embodiment may specifically refer to a database table, that is, a data structure stored in a database for storing data. The database table usually includes a table header (column name) and row data (data record), and has clearly defined data structure, data type, data length and primary key and other attributes. The data in the table can be accessed, modified, queried and other operations can be performed through various programming languages ​​such as SQL. For another example, the data table in this embodiment may specifically refer to a data lake table, that is, a data structure stored in a data lake for storing data. A data lake table is usually a set of data records containing partitions, columns and rows, and may contain various types of data, such as structured, semi-structured and unstructured data. The data lake table may be composed of multiple data sources and may be queried and accessed through a unified interface. For example, the data table may specifically be an event table related to an event, in which a plurality of event-related filter columns for filtering things may be included, such as event number, event occurrence time, event occurrence date, event-related data such as description, etc.

[0057] Among them, a data lake is an unstructured or semi-structured data storage system that allows the storage of thousands of structured, semi-structured and unstructured data without the need for explicit predefined schemas. Data lakes usually use scalable and distributed storage methods, such as Hadoop Distributed File System (HDFS), to provide fast analysis and query of the data stored in them. Therefore, compared with traditional databases, data lakes are more flexible and adaptable, and can effectively store and process various types of data.

[0058] Among them, the scanning event refers to an event generated when the user queries the data table, which is used to record the filtering conditions used by the user. The filtering condition refers to the condition used when filtering the data in the data table. The user can set the filtering condition by himself when querying the data table. For example, the filtering condition can specifically include the filtering column used when filtering the data in the data table. For another example, the filtering condition can specifically include the filtering row used when filtering the data in the data table.

[0059] Specifically, the server will receive multiple scan events for the data table, and the scan event is used to record the filter conditions used when querying the data table. In a specific application, the scan event for the data table is reported to the server through the data storage system that stores the data table. In a specific application, an event capture process is implanted in the computing engine kernel in the data storage system. When a scan for the data table occurs, the computing engine kernel in the data storage system will record the scan event for the data table and report the scan event to the server through the background process. In this way, all scan events for the data table will be reported to the server.

[0060] In a specific application, the data storage system may be a data lake, and the architecture diagram of the scan event report may be as follows: Figure 3 As shown, in the data storage system, the computing engine kernel includes an event reporter, an event processor, and a format processor. The user triggers a scan of a data table through an interface call. The event reporter reports the scan event for the data table to the event processor based on the scan of the data table. The event processor uploads the scan event for the data table to the format processor for unified format processing, and uploads the scan event for the data table after unified format processing to the message queue. The message queue uploads the scan event for the data table to the data lake table for storage. At the same time, the event reporter in the computing engine kernel of the data storage system will interact with the server to directly report the scan event for the data table. In a specific application, the scan of the data table can be submitted to the event reporter through the COMMIT command. The COMMIT command is used to save the modifications made by the transaction to the data lake. It saves all transactions after the last COMMIT or ROLLBACK (callback) command to the data lake. The event reporter will report the scan event for the data table based on the COMMIT information. In a specific application, such as Figure 4 As shown, in addition to triggering a scan of a data table, the interface call can also trigger operations such as creation, deletion, snapshot, and modification of the data table. Insertion, deletion, update, table measurement, and other processing can be implemented in the event handler.

[0061] Step 204: extracting filter column sets corresponding to the plurality of scanning events from the filter conditions corresponding to the plurality of scanning events respectively.

[0062] Among them, the filter column refers to the index field used to filter the data in the data table. For example, when the data table is an event table, and the data in the data table can be filtered by fields such as event number, event occurrence time, event occurrence date, event related data (such as description), the filter column can be a single column index such as event number, event occurrence time, event occurrence date, event related data (such as description), or a double column index such as event number + event occurrence time, event occurrence time + event occurrence date, event occurrence date + event related data (such as description), or a three column index such as event number + event occurrence time + event occurrence date, event occurrence time + event occurrence date + event related data (such as description), or a four column index such as event number + event occurrence time + event occurrence date + event related data (such as description).

[0063] Specifically, the filter condition includes the filter column used when filtering the data in the data table. For example, when the data table is an event table, and the user uses the single column index of event number to filter events during query, the filter condition will include the filter column of event number. For another example, if the user uses the three column index of event number + event date + event time to filter events during query, the filter condition will include the filter column of event number + event date + event time.

[0064] Therefore, for each of the multiple scanning events, the server can directly extract the corresponding filter column set of the scanning event from the corresponding filter condition of the scanning event. In a specific application, the server will use multiple candidate filter columns as keywords to traverse the corresponding filter conditions of the scanning event to extract the corresponding filter column set from the corresponding filter conditions of the scanning event. The candidate filter column refers to all filter columns that may appear in the filter condition, and can be configured according to the actual application scenario.

[0065] In a specific application, after receiving multiple scanning events for a data table, the server will summarize and organize the multiple scanning events for the data table into a data table scanning event detail table, and then extract the corresponding filter column sets of the multiple scanning events from the corresponding filter conditions of the multiple scanning events in the data table scanning event detail table. By summarizing and organizing into a data table scanning event detail table in this way, it is possible to classify and organize the multiple scanning events for the data table. For example, the multiple scanning events for the data table can be summarized and organized in units of one day, and the server can generate a data table index scheme every day according to the multiple scanning events for the data table received on that day.

[0066] Step 206 , for each scanning event in the multiple scanning events, enumerate the combination consisting of at least one element in the filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event.

[0067] Among them, the elements in the filter column set are filter columns, and the filter column set includes at least one filter column, and the filter columns included may be single-column indexes, double-column indexes, three-column indexes, four-column indexes, etc. For example, when the data table is an event table, and the filter column included in the filter column set is a single-column index, the included filter column may specifically be at least one of event number, event occurrence time, event occurrence date, and event-related data (such as description). For another example, when the data table is an event table, and the filter column included in the filter column set is a double-column index, the included filter column may specifically be at least one of event number + event occurrence time, event number + event occurrence date, event number + event-related data (such as description), etc. For another example, when the data table is an event table, and the filter column included in the filter column set is a four-column index, the included filter column may specifically be event number + event occurrence time + event occurrence date + event-related data (such as description). The candidate column index scheme refers to a candidate scheme that may be used as an index scheme for a data table. Each combination consisting of at least one element in the filter column set is a candidate column index scheme.

[0068] Specifically, for each scanning event in the multiple scanning events, the server will enumerate the combination of at least one element in the filter column set corresponding to the scanning event, and use each combination of at least one element in the filter column set as a candidate column index scheme to obtain the candidate column index scheme corresponding to the scanning event. The at least one element is at least one of a single-column index element and a multi-column index element.

[0069] In a specific application, when at least one element is a single-column index element, the server will treat each single-column index element as a combination. When at least one element is a multi-column index element, the server will perform column splitting on each multi-column index element to obtain a combination of each multi-column index element. When at least one element includes a single-column index element and a multi-column index element, the server will treat each single-column index element as a combination and perform column splitting on each multi-column index element to obtain a combination of each multi-column index element.

[0070] Step 208 , according to the candidate column indexing schemes corresponding to the plurality of scan events, count the number of scan events hit by each candidate column indexing scheme.

[0071] Specifically, based on the candidate column index schemes corresponding to multiple scanning events, the server can determine the total number of times each candidate column index scheme appears in the candidate column index schemes corresponding to the multiple scanning events, and then use the total number as the number of scanning events hit by each candidate column index scheme.

[0072] It should be noted that, when the filter column set corresponding to the scan event includes only one element, there is no duplicate candidate column index scheme in the candidate column index scheme corresponding to the scan event obtained. However, when the filter column set corresponding to the scan event includes at least two elements and there are multiple column index elements in at least two elements, there may be duplicate candidate column index schemes in the candidate column index scheme corresponding to the scan event obtained. Therefore, the server will determine the total number of times each candidate column index scheme appears in the candidate column index schemes corresponding to the multiple scan events based on the number of times each candidate column index scheme appears in the candidate column index schemes corresponding to the multiple scan events, and then the total number can be used as the number of scan events hit by each candidate column index scheme.

[0073] For example, assume that there are 4 scan events (scan event 1, scan event 2, scan event 3 and scan event 4), and the candidate column index scheme corresponding to scan event 1 is scheme 1, the candidate column index schemes corresponding to scan event 2 are scheme 1, scheme 1, scheme 2, scheme 3, the candidate column index scheme corresponding to scan event 3 is scheme 2, and the candidate column index schemes corresponding to scan event 4 are scheme 1, scheme 2, scheme 2, scheme 3. Based on this, it can be statistically obtained that the number of scan events hit by scheme 1 is 4, the number of scan events hit by scheme 2 is 4, and the number of scan events hit by scheme 3 is 2.

[0074] Step 210 : selecting an index scheme for the data table from the candidate column index schemes based on the number of scan events hit by the candidate column index schemes.

[0075] The index scheme of the data table refers to a scheme used when searching for data in the data table, and specifically may be an index field used when searching for data in the data table.

[0076] Specifically, the server can select an index scheme for the data table from each candidate column index scheme based on the number of scan events hit by each candidate column index scheme. In a specific application, the more scan events hit, it usually means that the candidate column index scheme is used more by users. Therefore, the server can directly determine the candidate column index scheme with the largest number of hit scan events as the index scheme for the data table. In a specific application, the server can also implement the selection of the index scheme for the data table by interacting with the user. Specifically, the server can first filter out at least one recommended index field to be added from each candidate column index scheme based on the number of scan events hit by each candidate column index scheme, and push the recommended index field to be added to the user for selection, so that the user selects the index scheme for the data table from at least one recommended index field to be added.

[0077] The above-mentioned data table index scheme generation method can obtain the filter column set used by the data table when being queried by receiving multiple scan events for the data table, extracting the filter column set corresponding to each of the multiple scan events from the filter conditions corresponding to each of the multiple scan events, and obtaining the candidate column index scheme corresponding to the scan event by enumerating the combination of at least one element in the filter column set corresponding to the scan event for each of the multiple scan events. The candidate column index scheme corresponding to each of the multiple scan events can be obtained by analyzing the filter column set used by the data table when being queried, and then the number of scan events hit by each candidate column index scheme can be counted based on the candidate column index scheme corresponding to each of the multiple scan events, and the index scheme of the data table can be selected from each candidate column index scheme based on the number of scan events hit by each candidate column index scheme. In the whole process, the index scheme of the data table can be generated by analyzing the usage and structural characteristics of the data table, and the generated index scheme of the data table will be more in line with the data search requirements of the data table, and the data search efficiency can be improved.

[0078] In one embodiment, for each scanning event in the plurality of scanning events, enumerating a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event includes:

[0079] For each scanning event in the multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a single column index element, each single column index element is taken as a combination to obtain a candidate column index scheme corresponding to the scanning event.

[0080] The single column index element refers to a single column index in the filter column set. For example, when the data table is an event table, the single column index element may specifically be a single column index such as event number, event occurrence time, event occurrence date, event related data (such as description), etc.

[0081] Specifically, for each scanning event among multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a single-column index element, the server will directly treat each single-column index element as a combination to obtain a candidate column index scheme corresponding to the scanning event. For example, when the filter column set corresponding to the scanning event includes two single-column index elements, namely, event number and event occurrence time, the server will directly treat the two single-column index elements, namely, event number and event occurrence time, as a combination to obtain a candidate column index scheme corresponding to the scanning event, and the number of the candidate column index schemes is two, specifically, the event number and event occurrence time are grouped separately.

[0082] In this embodiment, when at least one element in the filter column set corresponding to the targeted scanning event is a single-column index element, each single-column index element is regarded as a combination, which can realize the enumeration of the combinations formed by at least one element in the filter column set to obtain the candidate column index scheme corresponding to the targeted scanning event.

[0083] In one embodiment, for each scanning event in the plurality of scanning events, enumerating a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event includes:

[0084] For each scanning event in the multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a multi-column index element, at least two index element subsequences corresponding to each multi-column index element are obtained by respectively performing column splitting on each multi-column index element;

[0085] For each multi-column index element, the multi-column index element and at least two index element subsequences corresponding to the multi-column index element are respectively taken as a combination to obtain a candidate column index scheme corresponding to the scan event.

[0086] Among them, the multi-column index element refers to the multi-column index in the filter column set. For example, the multi-column index element can be a double-column index, a three-column index, a four-column index, etc. in the filter column set. For example, when the data table is an event table, the multi-column index element can be a double-column index such as event number + event time, event time + event date, event date + event related data (such as description), a three-column index such as event number + event time + event date, event time + event date + event related data (such as description), and a four-column index such as event number + event time + event date + event related data (such as description).

[0087] Column splitting refers to splitting the column indexes in a multi-column index element to obtain multiple possible combinations of the column indexes in the multi-column index element. For example, by splitting a double-column index, two single-column indexes can be obtained. For another example, by splitting a three-column index, three double-column indexes and three single-column indexes can be obtained. For another example, by splitting a four-column index, four three-column indexes, six double-column indexes and four single-column indexes can be obtained.

[0088] Among them, the index element subsequence refers to a sequence composed of at least one column index in the multi-column index element, and the number of columns is less than the number of columns of the corresponding multi-column index element. For example, when the multi-column index element is a double-column index, the number of columns of its index element subsequence is a single column. For another example, when the multi-column index element is a three-column index, the number of columns of its index element subsequence is a single column or a double column. For another example, when the multi-column index element is a four-column index, the number of columns of its index element subsequence is a single column, a double column, or a triple column.

[0089] Specifically, for each scanning event in multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a multi-column index element, the server will split the column index in each multi-column index element respectively to obtain multiple combinations that may be formed by the column index in the multi-column index element, that is, at least two index element subsequences corresponding to the multi-column index element. After obtaining at least two index element subsequences corresponding to each multi-column index element, for each multi-column index element, the server will take the multi-column index element and the at least two index element subsequences corresponding to the multi-column index element as a combination to obtain a candidate column index scheme corresponding to the scanning event. Each combination obtained is a candidate column index scheme corresponding to the scanning event.

[0090] In the present embodiment, when at least one element in the filter column set corresponding to the targeted scanning event is a multi-column index element, by performing column splitting on each multi-column index element respectively, at least two index element subsequences corresponding to each multi-column index element can be obtained, and then the targeted multi-column index element and the at least two index element subsequences corresponding to the targeted multi-column index element can be respectively taken as a combination to realize enumeration of combinations consisting of at least one element in the filter column set, thereby obtaining a candidate column index scheme corresponding to the targeted scanning event.

[0091] In one embodiment, by performing column splitting on each multi-column index element, at least two index element subsequences corresponding to each multi-column index element are obtained, including:

[0092] For each multi-column index element, based on the number of columns of the targeted multi-column index element, determine the number of extracted columns of the index element subsequence that can be constituted by each column index element in the targeted multi-column index element; the number of extracted columns is less than the number of columns of the targeted multi-column index element;

[0093] According to the number of extracted columns of the index element subsequence, subsequence extraction processing is performed on the targeted multi-column index elements to obtain at least two index element subsequences corresponding to the targeted multi-column index elements.

[0094] Among them, the number of extracted columns refers to the number of columns of index elements extracted from the multi-column index elements each time when subsequence extraction is performed on the targeted multi-column index elements. For example, when the number of extracted columns is 1, it means that the number of columns of index elements extracted from the multi-column index elements each time is 1, and the extracted subsequence of index elements is a single-column index. For another example, when the number of extracted columns is 2, it means that the number of columns of index elements extracted from the multi-column index elements each time is 2, and the extracted subsequence of index elements is a double-column index. For another example, when the number of extracted columns is 3, it means that the number of columns of index elements extracted from the multi-column index elements each time is 3, and the extracted subsequence of index elements is a three-column index. It should be noted that the number of extracted columns is less than the number of columns of the targeted multi-column index elements. Specifically, the number of extracted columns can be obtained by enumerating column values ​​that are less than the number of columns of the targeted multi-column index elements.

[0095] Specifically, for each multi-column index element, the server will enumerate column values ​​that are less than the number of columns of the targeted multi-column index element, obtain the number of extracted columns of the index element subsequence that can be constituted by each column index element in the targeted multi-column index element, and then perform subsequence extraction on the targeted multi-column index element according to the number of extracted columns of the index element subsequence, enumerate the combinations of column index elements in the targeted multi-column index element that are the same as the number of extracted columns, and obtain at least two index element subsequences corresponding to the targeted multi-column index element.

[0096] For example, when the number of columns of the targeted multi-column index element is 4, the number of extracted columns of the subsequence of index elements that can be formed by each column index element in the multi-column index element is three, namely, the number of extracted columns is 3, the number of extracted columns is 2, and the number of extracted columns is 1. For another example, when the number of columns of the targeted multi-column index element is 3, the number of extracted columns of the subsequence of index elements that can be formed by each column index element in the multi-column index element is two, namely, the number of extracted columns is 2 and the number of extracted columns is 1.

[0097] To illustrate, assuming that the targeted multi-column index element is a double-column index (A+B), by enumerating column values ​​that are less than the number of columns of the targeted multi-column index element, it can be obtained that the number of extracted columns that can be formed by the column index elements in the targeted multi-column index element is 1, then the targeted multi-column index element can be subjected to subsequence extraction with the extracted column number of 1, and the combinations of column index elements in the targeted multi-column index element that are the same as the number of extracted columns of 1 (i.e., single-column index A and single-column index B) are enumerated to obtain two index element subsequences corresponding to the targeted multi-column index element, one index element subsequence includes the single-column index A, and the other index element subsequence includes the single-column index B.

[0098] In this embodiment, the number of extracted columns of the index element subsequence that can be constituted by each column index element in the multi-column index elements can be determined based on the number of columns of the targeted multi-column index elements, and then the subsequence extraction processing can be performed according to the number of extracted columns to obtain at least two index element subsequences corresponding to the targeted multi-column index elements. The obtained at least two index element subsequences exist as candidate column index schemes corresponding to the targeted scanning event. Therefore, in this way, the candidate column index schemes are actually expanded and enriched by analyzing the structural characteristics of the targeted multi-column index elements, and more diverse candidate column index schemes that are in line with the targeted scanning event can be obtained, which is conducive to selecting an index scheme for the data table that is more in line with the data search requirements of the data table from each candidate column index scheme, and can improve data search efficiency.

[0099] In one embodiment, subsequence extraction is performed on the targeted multiple columns of index elements according to the number of extracted columns of the index element subsequence, and at least two index element subsequences corresponding to the targeted multiple columns of index elements are obtained, including:

[0100] When the number of extracted columns of the index element subsequence is single, each column of index elements in the targeted multi-column index elements is taken as an index element subsequence to obtain at least two index element subsequences corresponding to the targeted multi-column index elements.

[0101] Specifically, when the number of extracted columns of the index element subsequence is single, since the number of extracted columns is less than the number of columns of the targeted multi-column index element, this indicates that the multi-column index element is a dual-column index, and the number of extracted columns is 1, the server will treat each column index element in the targeted multi-column index element as an index element subsequence, and obtain at least two index element subsequences corresponding to the targeted multi-column index element. For example, assuming that the multi-column index element is a dual-column index (A+B), the server treats each column index element in the targeted multi-column index element as an index element subsequence, and obtains two index element subsequences, one index element subsequence includes a single-column index A, and the other index element subsequence includes a single-column index B.

[0102] In this embodiment, when the number of extracted columns of the index element subsequence is determined to be single, by taking each column of index elements in the targeted multiple columns of index elements as an index element subsequence, it is possible to obtain at least two index element subsequences corresponding to the targeted multiple columns of index elements.

[0103] In one embodiment, subsequence extraction is performed on the targeted multiple columns of index elements according to the number of extracted columns of the index element subsequence, and at least two index element subsequences corresponding to the targeted multiple columns of index elements are obtained, including:

[0104] When there are multiple extracted column numbers in the index element subsequence, for each of the multiple extracted column numbers, according to the extracted column number, the combinations of column index elements with the same number in the multiple column index elements are enumerated to obtain the index element subsequence corresponding to the extracted column number;

[0105] Based on the index element subsequences corresponding to the multiple extracted column numbers, at least two index element subsequences corresponding to the multiple column index elements are obtained. Specifically, when the number of extracted column numbers of the index element subsequence is multiple, it means that the extraction needs to be performed in sequence according to the multiple extracted column numbers. For each of the multiple extracted column numbers, the server will enumerate the combinations of column index elements of the same number in the multiple column index elements according to the extracted column numbers, and obtain the index element subsequence corresponding to the extracted column numbers, and then based on the index element subsequences corresponding to the multiple extracted column numbers, at least two index element subsequences corresponding to the multiple column index elements are obtained.

[0106] For example, when the number of extracted columns is 2, according to the number of extracted columns, the server will enumerate the combination of two index elements in the multi-column index element (at least three-column index) to obtain the subsequence of index elements corresponding to the number of extracted columns. For further example, assuming that the multi-column index element is a three-column index (A+B+C), by enumerating the combination of two index elements in the three-column index, the subsequence of index elements corresponding to the number of extracted columns obtained is three double-column indexes, namely double-column index (A+B), double-column index (B+C) and double-column index (A+C).

[0107] In this embodiment, when it is determined that the number of extracted column numbers of the index element subsequence is multiple, by enumerating, for each of the multiple extracted column numbers, combinations of column index elements with the same number in the targeted multiple-column index elements according to the targeted extracted column number, the targeted extracted column number can be used to extract the corresponding index element subsequence, and based on this, at least two index element subsequences corresponding to the targeted multiple-column index elements can be obtained.

[0108] In one embodiment, based on the number of scan events hit by each candidate column index scheme, selecting an index scheme for the data table from each candidate column index scheme includes:

[0109] Based on the number of scan events hit by each candidate column index scheme, a candidate column index scheme with the largest number of hit scan events is determined, and the candidate column index scheme with the largest number of hit scan events is determined as the index scheme of the data table.

[0110] Specifically, after counting the number of scan events hit by each candidate column index scheme, the server will sort the candidate column index schemes based on the number of scan events hit by each candidate column index scheme, and determine the candidate column index scheme with the largest number of hit scan events based on the sorting results, and determine the candidate column index scheme with the largest number of hit scan events as the index scheme for the data table.

[0111] In specific applications, the number of columns in the column indexes of the candidate column index schemes is not exactly the same. When sorting the candidate column index schemes, the server can directly sort all the candidate column index schemes and determine a candidate column index scheme with the largest number of hit scan events as the index scheme for the data table. The server can also sort multiple candidate column index schemes with the same number of columns based on the number of columns in the column index to obtain the candidate column index scheme with the largest number of hit scan events for each number of columns, and use the candidate column index scheme with the largest number of hit scan events for each number of columns as the index scheme for the data table.

[0112] For example, when the number of columns in the column index in each candidate column index scheme includes 1, 2, and 3, the server can use the number of columns of the column index as the dimension to sort multiple candidate column index schemes with the number of columns of the column index being 1, the number of columns of the column index being 2, and the number of columns of the column index being 3, respectively, to obtain the candidate column index schemes with the largest number of hit scan events with the number of columns of the column index being 1, the number of columns of the column index being 2, and the number of columns of the column index being 3, and use the three obtained candidate column index schemes as the index schemes of the data table.

[0113] In this embodiment, the index scheme of the data table can be objectively selected from the candidate column index schemes by using the hit scan event number in this way. The selected index scheme of the data table is more in line with the data search requirements of the data table, and the data search efficiency can be improved.

[0114] In one embodiment, based on the number of scan events hit by each candidate column index scheme, selecting an index scheme for the data table from each candidate column index scheme includes:

[0115] Based on the number of scan events hit by each candidate column index scheme, at least one recommended index field is selected from each candidate column index scheme;

[0116] Adding an index field according to at least one suggestion generates an index optimization prompt; the index optimization prompt is used to indicate adding an index field from at least one suggested index field;

[0117] In response to an index field adding event triggered based on the index optimization hint, the added index field is determined as an index scheme of the data table.

[0118] The suggested index field to be added refers to a candidate column index scheme that is suggested to be added as an index field. The index optimization hint refers to a hint for optimizing the index, which is used to indicate adding an index field from at least one suggested index field to be added. The index field adding event refers to an event for adding an index field.

[0119] Specifically, based on the number of scan events hit by each candidate column index scheme, the server will filter out at least one recommended index field to be added from each candidate column index scheme according to the preconfigured filtering conditions, and generate an index optimization prompt based on the at least one recommended index field to be added, where the index optimization prompt is used to indicate adding an index field from the at least one recommended index field to be added. After generating the index optimization prompt, the server will monitor the index field adding event used to add the index field, and after monitoring the index field adding event, the server responds to the index field adding event triggered based on the index optimization prompt, determines the added index field, and determines the added index field as the index scheme of the data table.

[0120] Among them, the preconfigured filtering conditions can be configured according to the actual application scenario. For example, the preconfigured filtering conditions can be specifically the N candidate column index schemes with the largest number of hit scan events, where N is a positive integer and can be configured according to the actual application scenario. For another example, the preconfigured filtering conditions can be specifically the number of hit scan events being greater than the preconfigured event number threshold, and the event number threshold can be configured according to the actual application scenario.

[0121] In a specific application, after generating an index optimization prompt, the server will push the index optimization prompt to a display interface, so that the user can view the index optimization prompt through the display interface, and add an index field from at least one suggested index field based on the index optimization prompt. After the user determines that an index field needs to be added, an index field adding event will be triggered, and the server responds to the index field adding event triggered based on the index optimization prompt, determines the added index field, and determines the added index field as the index scheme of the data table.

[0122] In a specific application, the display interface may be the display interface of a terminal connected to the server, that is, the server will push the index optimization prompt to the connected terminal so that the display interface of the terminal displays the index optimization prompt. The display interface may also be the display interface of a display device external to the server. It should be noted that in the index optimization prompt, not only is at least one suggested index field added, but also the estimated daily reduction in query energy consumption corresponding to each suggested index field added is provided. Users can select index fields to add based on the estimated daily reduction in query energy consumption. In this embodiment, the method of calculating the estimated daily reduction in query energy consumption corresponding to the suggested index field addition is not limited, as long as the calculation can be achieved. In this way, an intuitive selection reference can be provided to the user, which is conducive to selecting an index scheme that is more in line with the data search needs of the data table.

[0123] In a specific application, index optimization hints can be as follows Figure 5 As shown, three recommended index fields are added (index field 1, index field 2, and index field 3), and each recommended index field can reduce the query energy consumption by an estimated daily amount, that is, the number of queries that can be reduced by X times, Y times, and Z times per day, respectively.

[0124] In a specific application, such as Figure 6 As shown, in the index optimization tips, in addition to the suggestion to add index fields and the estimated daily reduction in query energy consumption for each suggested index field, an index field adding entry is also provided. Users can trigger an index field adding event by selecting the index field adding entry.

[0125] In this embodiment, at least one suggested index field to be added can be provided for selection, so that the user can select a suitable index field to add from the at least one suggested index field based on experience. The index scheme for the data table is determined by combining the at least one suggested index field to be added obtained by data analysis and experience reference. The obtained index scheme for the data table will be more in line with the data search requirements of the data table and can improve data search efficiency.

[0126] In one embodiment, in response to an index field adding event triggered based on an index optimization hint, determining the added index field as an index scheme of the data table includes:

[0127] In response to an index field adding event triggered based on an index optimization prompt, generating an addable index field option according to at least one suggested index field addition;

[0128] In response to a selection event for at least one of the addable index field options, the index field indicated by the selection event is determined as an index scheme of the data table.

[0129] Specifically, the server responds to the index field adding event triggered based on the index optimization prompt, and will provide at least one suggested added index field as an addable index field option for the user to select. The user can select at least one option as the index field to be added based on experience, triggering a selection event for at least one option in the addable index field options. In response to the selection event for at least one option in the addable index field options, the server will determine the index field indicated by the selection event as the index scheme for the data table.

[0130] In a specific application, after generating the addable index field options, the server will push the addable index field options to the display interface, so that the user can view the addable index field options through the display interface, and select at least one option from the addable index field options to add the index field. After the user determines that an index field needs to be added, a selection event for at least one option in the addable index field options will be triggered. In response to the selection event for at least one option in the addable index field options, the server will determine the index field indicated by the selection event as the index scheme of the data table. The display interface can be the display interface of a terminal connected to the server, or it can be the display interface of a display device external to the server. In a specific application, such as Figure 7 As shown, the display interface shows the option to add index fields (such as Figure 7 The data table includes index field 1, index field 2, and index field 3) and the corresponding database name, data table name, data table belonging object and other information. The user can select at least one option from the addable index field options to add an index field. Figure 7 As shown, after selecting at least one option, you can execute the index field addition by checking the "Execute" control.

[0131] In this embodiment, in response to an index field addition event, addable index field options are generated based on at least one suggested index field to be added, and addable index field options can be quickly provided for users to directly select, thereby optimizing the index field adding method and reducing the complexity of operations, so that users can select appropriate options from the addable index field options for addition based on their experience. The addable index field options obtained by data analysis and experience references are combined to realize the determination of the index scheme for the data table. The obtained index scheme for the data table will be more in line with the data search requirements of the data table, and can improve the data search efficiency.

[0132] In one embodiment, the addable index field option includes a custom option; in response to a selection event for at least one option in the addable index field option, determining the index field indicated by the selection event as the index scheme of the data table includes:

[0133] In response to a selection event for at least one of the addable index field options, when the option indicated by the selection event is a custom option, obtaining a custom index field corresponding to the custom option;

[0134] Determine the custom index field as the index scheme of the data table.

[0135] Specifically, the addable index field option includes a custom option, through which the index field to be added can be customized. After a selection event for at least one option in the addable index field option is triggered, the server responds to the selection event for at least one option in the addable index field option, and if the option indicated by the selection event is a custom option, obtains a custom index field corresponding to the custom option, and determines the custom index field as an index scheme for the data table.

[0136] In a specific application, after generating the addable index field options, the server will push the addable index field options to the display interface, so that the user can view the addable index field options through the display interface, and select at least one option from the addable index field options to add the index field. The addable index field options include a custom option. When the user needs to customize the index field to be added, the custom option can be selected. When the user selects the custom option, the server can obtain the custom index field set by the user based on the display interface, and determine the custom index field as the index scheme of the data table.

[0137] In a specific application, such as Figure 8 As shown, the display interface displays the addable index field options including the custom option and the corresponding database name, data table name, data table belonging object and other information of the data table. When the user selects the custom option, the custom index field can be entered in the "Custom Index Field" under "Add Index Field". After entering the custom index field, the custom index field can be added by selecting the "Execute" control. After the user selects the "Execute" control, the server can obtain the custom index field corresponding to the custom option and determine the custom index field as the index scheme of the data table.

[0138] In this embodiment, custom options are provided to customize index fields, and the method of adding index fields is optimized so that users can configure custom index fields according to their experience and add them, obtain custom index fields corresponding to the custom options, and determine the custom index fields as the index scheme of the data table. The index scheme of the data table is determined by combining experience as a reference. The resulting index scheme of the data table will be more in line with the data search requirements of the data table and can improve data search efficiency.

[0139] In one embodiment, the data table index scheme method further includes:

[0140] The incremental index building interface of the computing engine in the data storage system storing the data table is called to instruct the computing engine in the data storage system to incrementally build the index according to the index scheme of the data table and generate an index file of the data table.

[0141] Among them, the incremental index building interface is used to incrementally build the index of the data table. For example, the incremental index building interface can be used for the incremental generation of the index file of the data table. The index file is a file organization system that classifies all data into different categories, allowing users to quickly find the required data. The index file is also called a directory file, which can use the resources indexed by the search engine so that users can use them quickly. The index file saves and manages files through a set of indexes and their corresponding relationships. When this method is used to retrieve data, it retrieves relevant data from the index.

[0142] Specifically, when the incremental index construction triggering condition is met, the server will trigger the incremental index construction and call the incremental index construction interface of the computing engine in the data storage system that stores the data table to instruct the computing engine in the data storage system to incrementally build the index according to the index scheme of the data table and generate the index file of the data table. In specific applications, the incremental index construction triggering condition can be configured according to the actual application scenario. For example, the incremental index construction triggering condition can be that the data volume of the data table is greater than the preset data volume threshold, that is, when the data volume in the data table is too much, the index file of the data table can be generated by incremental index construction. Among them, the preset data volume threshold can be configured according to the actual application scenario.

[0143] In a specific application, the index file of the generated data table will be saved on the data storage system storing the data table. The index file of the data table can be a positioning index file. The positioning index file is used to locate whether there is data in the data table. Then, when the data table is scanned next time, the computing engine in the data storage system will decide whether to use the index file to skip unnecessary file scanning according to the index status of the current index file. Specifically, if it is determined that the corresponding file does not exist in the data table according to the positioning index file, no file scanning is required. Only when it is determined that the corresponding file exists in the data table according to the positioning index file, further file scanning is performed.

[0144] In this embodiment, by calling the incremental index construction interface, the computing engine in the data storage system can be instructed to incrementally construct the index according to the index scheme of the data table, so as to realize the generation of the index file of the data table. Since the index file is generated by using the index scheme of the data table that is more suitable for the data search requirements of the data table, the generated index file will also be more suitable for the data search requirements of the data table, which can improve the data search efficiency. At the same time, when the data table changes, the incremental index construction method is adopted to avoid frequent and repeated reconstruction of the entire data table index, saving computing resources and storage resources.

[0145] In one embodiment, Fig. 9 As shown, a flow chart is used to illustrate the data table index scheme generation method of the present application, and the data table index scheme generation method specifically includes the following steps:

[0146] Step 902, receiving multiple scan events for the data table; the scan events are used to record the filter conditions used when querying the data table;

[0147] Step 904, extracting filter column sets corresponding to the plurality of scanning events from the filter conditions corresponding to the plurality of scanning events respectively;

[0148] Step 906, for each scanning event in the multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a single-column index element, jump to step 908; when at least one element in the filter column set corresponding to the scanning event is a multi-column index element, jump to step 910;

[0149] Step 908, taking each single column index element as a combination, obtaining a candidate column index scheme corresponding to the scan event, and jumping to step 912;

[0150] Step 910: Splitting each multi-column index element into columns to obtain at least two index element subsequences corresponding to each multi-column index element. For each multi-column index element, the multi-column index element and the at least two index element subsequences corresponding to the multi-column index element are respectively combined to obtain a candidate column index scheme corresponding to the scan event.

[0151] Step 912, according to the candidate column index schemes corresponding to the multiple scan events, count the number of scan events hit by each candidate column index scheme;

[0152] Step 914, based on the number of scan events hit by each candidate column index scheme, determine the candidate column index scheme with the largest number of hit scan events, and determine the candidate column index scheme with the largest number of hit scan events as the index scheme for the data table.

[0153] In one embodiment, the data table index scheme generation method of the present application is illustrated by generating a data lake table index scheme applied in a data lake.

[0154] The inventor believes that the data tables in the data lake are huge, and the traditional manual index creation is labor-intensive and error-prone. Manual index creation requires a deep understanding of the structure and data of the data lake table. For newly created data lake tables or data lake table structures that have changed significantly, manual index creation is difficult, and manual index creation takes up a lot of manpower and time, which will affect the efficiency of data lake table usage. Based on this, the present application provides a method for generating a data lake table index scheme based on algorithm analysis and big data analysis, which automatically analyzes the usage and structural characteristics of the data lake table and intelligently recommends index schemes.

[0155] In one embodiment, an event capture process is implanted in the kernel of the data lake engine. When a scan of a data lake table occurs, the kernel of the data lake engine records the scan event for the data lake table and reports the scan event to the server through a background process. In this way, all scan events for the data lake table are reported to the server. The server summarizes and organizes the multiple scan events received into a detailed list of scan events for the data lake table. After summarizing and organizing the detailed list of scan events for the data lake table, the server will take out the multiple scan events organized that day. The corresponding filter conditions of each scan event include filter columns, and each scan event will contribute to four categories of candidate column indexes (single column, double column, triple column, quad column).

[0156] Among them, the scan event records the filtering conditions used by the user when querying the data table. For example, suppose the data lake table is an event table and contains four columns (ID+TS+DAT+DATA), where ID represents the event number, TS represents the time when the event occurred, DAT represents the date when the event occurred, and DATA represents event-related data (such as description, etc.). If the user uses the ID column to locate the event, the data lake engine will generate a scan event containing only the ID column on the back end. If the user uses ID, DTA, and TS to filter events at the same time, the database engine will generate a scan event containing 3 columns (ID+DTA+TS) on the back end. These events will become candidate column schemes for index generation, that is, candidate column index schemes. Using such an index can speed up the data lake engine's scanning of the table.

[0157] In one embodiment, the server extracts filter column sets corresponding to each of the multiple scanning events from the filter conditions corresponding to each of the multiple scanning events, and for each scanning event in the multiple scanning events, enumerates a combination consisting of at least one element in the filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event, and based on the candidate column index schemes corresponding to the multiple scanning events, counts the number of scanning events hit by each candidate column index scheme.

[0158] In one embodiment, for each scanning event among multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a single-column index element, the server will treat each single-column index element as a combination to obtain a candidate column index scheme corresponding to the scanning event. When at least one element in the filter column set corresponding to the scanning event is a multi-column index element, the server will perform column splitting on each multi-column index element to obtain at least two index element subsequences corresponding to each multi-column index element, and for each multi-column index element, treat the multi-column index element and the at least two index element subsequences corresponding to the multi-column index element as a combination to obtain a candidate column index scheme corresponding to the scanning event.

[0159] In one embodiment, when performing column splitting, for each multi-column index element, the server determines the number of extracted columns of the index element subsequence that can be constituted by each column index element in the multi-column index element based on the number of columns of the multi-column index element; if the number of extracted columns is less than the number of columns of the multi-column index element, subsequence extraction processing is performed on the multi-column index element according to the number of extracted columns of the index element subsequence, to obtain at least two index element subsequences corresponding to the multi-column index element.

[0160] In one embodiment, when the number of extracted columns of the index element subsequence is single, the server will take each column index element in the targeted multi-column index elements as an index element subsequence, and obtain at least two index element subsequences corresponding to the targeted multi-column index elements. When the number of extracted columns of the index element subsequence is multiple, for each of the multiple extracted columns, the server will enumerate the combinations of column index elements of the same number in the targeted multi-column index elements according to the targeted extracted columns, and obtain the index element subsequence corresponding to the targeted extracted columns, and obtain at least two index element subsequences corresponding to the targeted multi-column index elements based on the index element subsequences corresponding to each of the multiple extracted columns.

[0161] In one embodiment, taking a scanning event including three-column indexes (TS+ID+DATA) as an example, the candidate column index schemes corresponding to the scanning event include: candidate single-column index schemes index1(TS), index1(ID) and index1(DATA), candidate double-column index schemes index2(ID+DATA), index2(TS+DATA) and index2(TS+ID), and candidate three-column index scheme index3(TS+ID+DATA).

[0162] In one embodiment, there are four scanning events, and each scanning event has a corresponding filter column set as follows: Fig.10 As an example, the process of focusing and analyzing the filter column sets corresponding to multiple scanning events in this embodiment and obtaining the number of scanning events hit by each candidate column index scheme is described by way of example. Fig.10 As shown, the filter column set of scan event 1 includes a single-column index TS, a double-column index ID+TS, and a triple-column index ID+DATA+TS. By enumerating the combinations of at least one element in the filter column set of scan event 1, the candidate column index schemes corresponding to scan event 1 can be obtained, including: a single-column index TS, a double-column index ID+TS, two index element subsequences of the double-column index (single-column index ID and single-column index TS), a triple-column index ID+DATA+TS, and six index element subsequences of the triple-column index (double-column index ID+DATA, double-column index ID+TS, double-column index DATA+TS, single-column index ID, single-column index DATA, single-column index TS).

[0163] The filter column set of scan event 2 includes a single-column index ID, a double-column index ID+TS, and a triple-column index ID+DATA+TS. By enumerating the combinations of at least one element in the filter column set of scan event 1, the candidate column index scheme corresponding to scan event 1 can be obtained, including: a single-column index ID, a double-column index ID+TS, two index element subsequences of the double-column index (single-column index ID and single-column index TS), a triple-column index ID+DATA+TS, and six index element subsequences of the triple-column index (double-column index ID+DATA, double-column index ID+TS, double-column index DATA+TS, single-column index ID, single-column index DATA, single-column index TS).

[0164] The filter column set of scanning event 3 includes a single-column index DAT, a double-column index DAT+TS, and a triple-column index ID+DATA+TS. By enumerating the combinations of at least one element in the filter column set of scanning event 1, the candidate column index scheme corresponding to scanning event 1 can be obtained, including: a single-column index DAT, a double-column index DAT+TS, two index element subsequences of the double-column index (single-column index DAT and single-column index TS), a triple-column index ID+DATA+TS, and six index element subsequences of the triple-column index (double-column index ID+DATA, double-column index ID+TS, double-column index DATA+TS, single-column index ID, single-column index DATA, single-column index TS).

[0165] The filter column set of scan event 4 includes a single-column index TS, a double-column index ID+DAT, and a triple-column index ID+DATA+TS. By enumerating the combinations of at least one element in the filter column set of scan event 1, the candidate column index scheme corresponding to scan event 1 can be obtained, including: a single-column index TS, a double-column index ID+DAT, two index element subsequences of the double-column index (single-column index ID and single-column index DAT), a triple-column index ID+DATA+TS, and six index element subsequences of the triple-column index (double-column index ID+DATA, double-column index ID+TS, double-column index DATA+TS, single-column index ID, single-column index DATA, single-column index TS).

[0166] Based on this, it can be concluded that the candidate column index schemes appearing in scan event 1, scan event 2, scan event 3 and scan event 4 include: single column index ID, single column index DATA, single column index TS, single column index DAT, double column index ID+DATA, double column index ID+TS, double column index DATA+TS, double column index DAT+TS, double column index ID+DAT and triple column index ID+DATA+TS.

[0167] Based on the number of occurrences of each candidate column index scheme in the candidate column index schemes corresponding to the four scanning events, it can be obtained that the number of scanning events hit by each candidate column index scheme is: 8 hits for single-column index ID, 4 hits for single-column index DATA, 9 hits for single-column index TS, 3 hits for single-column index DAT, 4 hits for double-column index ID+DATA, 6 hits for double-column index ID+TS, 4 hits for double-column index DATA+TS, 1 hit for double-column index DAT+TS, 1 hit for double-column index ID+DAT and 4 hits for triple-column index ID+DATA+TS.

[0168] From this we can get Fig.10As shown in the cluster analysis results, the single-column index with the largest number of hit scan events is TS, the double-column index is ID+TS, and the three-column index is ID+DATA+TS.

[0169] In one embodiment, when selecting the index scheme for the data lake table, it can be automatically selected according to the number of hit scan events, that is, the candidate column index scheme with the largest number of hit scan events is directly determined as the index scheme for the data lake table. Alternatively, the user can select a suitable index scheme from the four categories of candidate column index schemes based on the current calculation results and the actual business situation.

[0170] In one embodiment, Fig.11 As shown, after receiving multiple scan events for the data lake table, the server will store the multiple scan events, and generate candidate column index schemes corresponding to each of the multiple scan events through the calculation method of the candidate column index scheme involved in the above embodiment, and save the candidate column index schemes corresponding to each of the multiple scan events in the configuration storage (index configuration attributes of the data lake table). At the same time, users can also interact with the server to obtain these data-derived candidate column index schemes, select the index scheme of the data lake table from them, or specify a specific index scheme of the data lake table according to their own domain knowledge (i.e., user manual configuration) to achieve index optimization. Among them, users can achieve index optimization by interacting with the index optimization service in the server. After the index scheme of the data lake table is selected, it will also be set in the index configuration attributes of the data lake table. It should be noted that if you do not completely rely on the automatic generation of candidate column index schemes, the specified index scheme is compatible with HIVE (a data warehouse tool based on Hadoop, used for data extraction, transformation, and loading, which is a mechanism for storing, querying, and analyzing large-scale data stored in Hadoop). In other words, users can also use SQL (Structured Query Language serverdatabase) commands to specify the index scheme of the data lake table. This method is compatible with the traditional way of using indexes and supports the definition and reconstruction of indexes using SQL syntax.

[0171] In one embodiment, the user obtains each candidate column index scheme derived from these data by interacting with the server, and the specific method of selecting the index scheme of the data lake table from them can be: the server selects at least one recommended index field from each candidate column index scheme based on the number of scan events hit by each candidate column index scheme, generates an index optimization prompt based on the at least one recommended index field, and pushes the index optimization prompt to the display interface, responds to the index field adding event triggered by the user through the display interface based on the index optimization prompt, generates an addable index field option based on the at least one recommended index field, and responds to the selection event for at least one option in the addable index field options, determines the index field indicated by the selected event as the index scheme of the data table. Wherein, the addable index field option includes a custom option, and if the option indicated by the selected event is a custom option, the server will obtain the custom index field corresponding to the custom option, and determine the custom index field as the index scheme of the data table. Wherein, the display interface can be a display interface of a terminal connected to the server, or a display interface of a display device external to the server.

[0172] In one embodiment, the user interacts with the server to obtain the candidate column index schemes derived from these data. In the process of selecting the index scheme for the data lake table, the interface changes of the display interface can be as follows: Fig.12 As shown, the display interface first displays the total number of daily queries of the data table, partition columns, column filtering information, the total number of scanned files, index optimization tips, index field addition entries, etc. On this basis, in response to an index field addition event (specifically, a trigger event for adding an index field entry), the display interface will display options for adding index fields, including custom options and at least one recommended index field to be added (such as Fig.12 The figure shows index field 1, index field 2 and index field 3).

[0173] In one embodiment, Fig.13 As shown in the figure, after determining the index scheme of the data lake table, the server will trigger the incremental index construction task (incremental generation of index files) according to the submission status of the data lake table. This process is fully automatic and does not require human intervention. When the incremental index construction task is triggered, the server will call the incremental index construction stored procedure in the data lake engine. The incremental index construction stored procedure will incrementally build the index according to the index field specified by the user (that is, the index scheme of the data lake table). The generated index file will be saved in the storage of the data lake table (that is, Fig.13In the data lake storage in the data lake, the next time the data lake table is scanned, the data lake engine will decide whether to use the index file to skip unnecessary file scans based on the current file index status.

[0174] In this embodiment, the user obtains the candidate column index schemes derived from these data by interacting with the server, and selects the index scheme of the data lake table from them. This method allows the user to not decide the candidate column index scheme for the index by himself, but only needs to select the calculated candidate column index scheme in the display interface, or decide the index scheme of the data lake table based on his own experience and domain knowledge, which can reduce the complexity of operations on the basis of lightweight reading configuration. At the same time, it supports incremental reconstruction, and automatically builds incrementally when the data lake table changes, avoiding frequent and repeated reconstruction of the entire data lake table index, saving computing resources and storage resources.

[0175] The method for generating an index scheme for a data table provided by the present application can generate an index scheme for a data table by analyzing the usage and structural characteristics of the data table. The index scheme for the generated data table will be more in line with the data search requirements of the data table and can improve the efficiency of data search. At the same time, the scheme of the present application reduces the need for manual intervention and improves the efficiency of the use of the data storage system. In addition, the method for generating an index scheme for a data table provided by the present application has strong scalability and can support data tables of various formats.

[0176] In one embodiment, when the data table index scheme generation method provided by the present application is applied to a point query scenario applicable to a BLOOMFILTER (a binary vector data structure, which is space and time efficient and is used to detect whether an element is a member of a set) type of index, the amount of data table scanning can be greatly reduced to 10% of the original amount, thereby greatly improving the speed of data query and saving computing resources.

[0177] In one embodiment, the point query scenario is described by taking the index scheme as a single-column index TS as an example. If the index scheme is a single-column index TS, the data lake engine will sort the data files in the event table according to TS (assuming that each original data file can contain data of any TS time). It can be simplified to understand that each data file only contains 1 TS data. When the user executes TS-related indexes (for example, TS=20230505), because each file only contains one corresponding TS data, the data lake engine does not need to open each data file to retrieve TS-related data, and can skip most of the data files, and only open the file data containing TS=20230505, thus skipping a large number of files to achieve the purpose of scanning acceleration.

[0178] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0179] Based on the same inventive concept, the embodiment of the present application also provides a data table index scheme device for implementing the data table index scheme method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data table index scheme device embodiments provided below can refer to the limitations of the data table index scheme method above, and will not be repeated here.

[0180] In one embodiment, Fig.14 As shown, a data table indexing scheme device is provided, including: a receiving module 1402, an extracting module 1404, an enumerating module 1406, a statistical module 1408 and a selecting module 1410, wherein:

[0181] The receiving module 1402 is used to receive multiple scanning events for the data table; the scanning event is used to record the filtering conditions used when querying the data table;

[0182] An extraction module 1404 is used to extract filter column sets corresponding to the plurality of scanning events from the filter conditions corresponding to the plurality of scanning events respectively;

[0183] An enumeration module 1406 is used to enumerate, for each scanning event in the multiple scanning events, a combination of at least one element in a filter column set corresponding to the scanning event, to obtain a candidate column index scheme corresponding to the scanning event;

[0184] A statistics module 1408 is used to count the number of scan events hit by each candidate column index scheme according to the candidate column index schemes corresponding to the multiple scan events;

[0185] The selection module 1410 is used to select an index scheme for the data table from each candidate column index scheme based on the number of scan events hit by each candidate column index scheme.

[0186] The above-mentioned data table index scheme generating device can obtain the filter column set used by the data table when being queried by receiving multiple scan events for the data table, extracting the filter column set corresponding to each of the multiple scan events from the filter conditions corresponding to each of the multiple scan events, and obtaining the candidate column index scheme corresponding to the scan event by enumerating the combination of at least one element in the filter column set corresponding to the scan event for each of the multiple scan events, and can obtain the candidate column index scheme corresponding to each of the multiple scan events by analyzing the filter column set used by the data table when being queried, and then can count the number of scan events hit by each candidate column index scheme based on the candidate column index scheme corresponding to each of the multiple scan events, and select the index scheme of the data table from each candidate column index scheme based on the number of scan events hit by each candidate column index scheme. In the whole process, the index scheme of the data table can be generated by analyzing the usage and structural characteristics of the data table, and the generated index scheme of the data table will be more in line with the data search requirements of the data table, and can improve the data search efficiency.

[0187] In one embodiment, the enumeration module is also used to, for each scanning event in multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a single-column index element, take each single-column index element as a combination to obtain a candidate column index scheme corresponding to the scanning event.

[0188] In one embodiment, the enumeration module is also used to, for each scanning event in multiple scanning events, when at least one element in the filter column set corresponding to the targeted scanning event is a multi-column index element, obtain at least two index element subsequences corresponding to each multi-column index element by performing column splitting on each multi-column index element respectively, and for each multi-column index element, take the targeted multi-column index element and the at least two index element subsequences corresponding to the targeted multi-column index element as a combination to obtain a candidate column index scheme corresponding to the targeted scanning event.

[0189] In one embodiment, the enumeration module is also used to determine, for each multi-column index element, based on the number of columns of the multi-column index element, the number of extracted columns of the index element subsequence that can be constituted by each column index element in the multi-column index element, the number of extracted columns is smaller than the number of columns of the multi-column index element, and subsequence extraction processing is performed on the multi-column index element according to the number of extracted columns of the index element subsequence to obtain at least two index element subsequences corresponding to the multi-column index element.

[0190] In one embodiment, the enumeration module is also used to, when the number of extracted columns of the index element subsequence is single, take each column of index elements in the targeted multi-column index elements as an index element subsequence, to obtain at least two index element subsequences corresponding to the targeted multi-column index elements.

[0191] In one embodiment, the enumeration module is also used to, when there are multiple extracted column numbers of the index element subsequence, for each of the multiple extracted column numbers, enumerate the combinations of column index elements of the same number in the multiple column index elements according to the targeted extracted column number, to obtain the index element subsequence corresponding to the targeted extracted column number, and based on the index element subsequences corresponding to each of the multiple extracted column numbers, obtain at least two index element subsequences corresponding to the multiple column index elements.

[0192] In one embodiment, the selection module is also used to determine the candidate column index scheme with the largest number of hit scan events based on the number of hit scan events of each candidate column index scheme, and determine the candidate column index scheme with the largest number of hit scan events as the index scheme of the data table.

[0193] In one embodiment, the selection module is also used to screen out at least one suggested index field to be added from each candidate column index scheme based on the number of scan events hit by each candidate column index scheme, and generate an index optimization prompt based on the at least one suggested index field to be added; the index optimization prompt is used to indicate the addition of an index field from at least one suggested index field to be added, and in response to an index field addition event triggered based on the index optimization prompt, the added index field is determined as the index scheme of the data table.

[0194] In one embodiment, the selection module is also used to respond to an index field adding event triggered based on an index optimization prompt, generate addable index field options based on at least one suggested index field addition, and in response to a selection event for at least one option in the addable index field options, determine the index field indicated by the selection event as the index scheme for the data table.

[0195] In one embodiment, the addable index field options include custom options, and the selection module is further used to respond to a selection event for at least one option in the addable index field options. When the option indicated by the selection event is a custom option, the custom index field corresponding to the custom option is obtained, and the custom index field is determined as the index scheme of the data table.

[0196] In one embodiment, the data table index scheme device also includes a calling module, which is used to call the incremental index construction interface of the computing engine in the data storage system that stores the data table, so as to instruct the computing engine in the data storage system to incrementally construct the index according to the index scheme of the data table and generate an index file of the data table.

[0197] Each module in the above-mentioned data table index scheme generating device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0198] In one embodiment, a computer device is provided. The computer device may be a server or a terminal. Taking the computer device as a server as an example, its internal structure diagram may be as follows: Fig.15 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as index schemes of data tables. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for generating a data table index scheme is implemented.

[0199] Those skilled in the art will understand that Fig.15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0200] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0201] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0202] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0203] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0204] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0205] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating a data table index scheme, characterized in that: The method comprises: Receiving a plurality of scanning events for a data table; the scanning events are used to record filtering conditions used when querying the data table; Extracting, from the filter conditions corresponding to the plurality of scanning events, the filter column sets corresponding to the plurality of scanning events respectively; For each scanning event in the multiple scanning events, enumerate a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event; According to the candidate column index schemes corresponding to the multiple scan events, count the number of scan events hit by each candidate column index scheme; Based on the number of scan events hit by each of the candidate column indexing schemes, an indexing scheme for the data table is selected from each of the candidate column indexing schemes.

2. The method according to claim 1, characterized in that For each of the multiple scanning events, enumerating a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event includes: For each scanning event among the multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a single column index element, each of the single column index elements is taken as a combination to obtain a candidate column index scheme corresponding to the scanning event.

3. The method according to claim 1 or 2, characterized in that: For each of the multiple scanning events, enumerating a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event includes: For each scanning event in the multiple scanning events, when at least one element in the filter column set corresponding to the scanning event is a multi-column index element, at least two index element subsequences corresponding to each of the multi-column index elements are obtained by respectively performing column splitting on each of the multi-column index elements; For each of the multi-column index elements, the multi-column index element and at least two index element subsequences corresponding to the multi-column index element are respectively taken as a combination to obtain a candidate column index scheme corresponding to the scan event.

4. The method according to claim 3, characterized in that The step of performing column splitting on each of the multiple-column index elements to obtain at least two index element subsequences corresponding to each of the multiple-column index elements comprises: For each of the multi-column index elements, based on the number of columns of the multi-column index elements, determine the number of extracted columns of the index element subsequence that can be constituted by each column index element in the multi-column index elements; the number of extracted columns is less than the number of columns of the multi-column index elements; According to the number of extracted columns of the index element subsequence, subsequence extraction processing is performed on the targeted multiple columns of index elements to obtain at least two index element subsequences corresponding to the targeted multiple columns of index elements.

5. The method according to claim 4, characterized in that The step of performing subsequence extraction processing on the targeted multiple columns of index elements according to the number of extracted columns of the index element subsequence to obtain at least two index element subsequences corresponding to the targeted multiple columns of index elements includes: When the number of extracted columns of the index element subsequence is single, each column of index elements in the targeted multiple columns of index elements is taken as an index element subsequence to obtain at least two index element subsequences corresponding to the targeted multiple columns of index elements.

6. The method according to claim 4, characterized in that The step of performing subsequence extraction processing on the targeted multiple columns of index elements according to the number of extracted columns of the index element subsequence to obtain at least two index element subsequences corresponding to the targeted multiple columns of index elements includes: When there are multiple extracted column numbers in the index element subsequence, for each of the multiple extracted column numbers, according to the extracted column number, enumerate the combinations of column index elements with the same number in the multiple column index elements to obtain the index element subsequence corresponding to the extracted column number; Based on the index element subsequences corresponding to the multiple extracted column numbers, at least two index element subsequences corresponding to the targeted multiple column index elements are obtained.

7. The method according to any one of claims 1 to 6, characterized in that: The step of selecting the index scheme for the data table from the candidate column index schemes based on the number of scan events hit by the candidate column index schemes includes: Based on the number of scan events hit by each of the candidate column index schemes, a candidate column index scheme with the largest number of hit scan events is determined, and the candidate column index scheme with the largest number of hit scan events is determined as the index scheme of the data table.

8. The method according to any one of claims 1 to 6, characterized in that: The step of selecting the index scheme for the data table from the candidate column index schemes based on the number of scan events hit by the candidate column index schemes includes: Based on the number of scan events hit by each of the candidate column index schemes, at least one recommended index field to be added is selected from each of the candidate column index schemes; Generate an index optimization hint according to the at least one suggested index field addition; the index optimization hint is used to indicate adding an index field from the at least one suggested index field addition; In response to an index field adding event triggered based on the index optimization hint, the added index field is determined as an index scheme of the data table.

9. The method according to claim 8, characterized in that In response to the index field adding event triggered by the index optimization prompt, determining the added index field as the index scheme of the data table includes: In response to an index field adding event triggered based on the index optimization hint, generating an addable index field option according to the at least one suggested adding index field; In response to a selection event for at least one of the addable index field options, the index field indicated by the selection event is determined as an index scheme for the data table.

10. The method according to claim 9, characterized in that The addable index field options include a custom option; in response to a selection event for at least one of the addable index field options, determining the index field indicated by the selection event as the index scheme of the data table includes: In response to a selection event for at least one of the addable index field options, when the option indicated by the selection event is a custom option, obtaining a custom index field corresponding to the custom option; The user-defined index field is determined as the index scheme of the data table.

11. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The incremental index building interface of the computing engine in the data storage system storing the data table is called to instruct the computing engine in the data storage system to incrementally build the index according to the index scheme of the data table and generate an index file of the data table.

12. A data table index scheme generating device, characterized in that: The device comprises: A receiving module, used for receiving a plurality of scanning events for a data table; the scanning events are used for recording filtering conditions used when querying the data table; An extraction module, configured to extract, from the filter conditions corresponding to the plurality of scanning events, filter column sets corresponding to the plurality of scanning events respectively; An enumeration module is used to enumerate, for each scanning event in the multiple scanning events, a combination of at least one element in a filter column set corresponding to the scanning event to obtain a candidate column index scheme corresponding to the scanning event; A statistics module, configured to count the number of scan events hit by each candidate column index scheme according to the candidate column index schemes corresponding to the multiple scan events; A selection module is used to select an index scheme for the data table from each of the candidate column index schemes based on the number of scan events hit by each of the candidate column index schemes.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images