An information generation method, device, electronic device, computer-readable storage medium, and computer program product

Abstract

The application provides an information generation method and device, electronic equipment, computer readable storage medium and computer program product; the application embodiment can be applied to the processing scene of the database statement of the e-commerce platform, the financial system and the like; the method comprises the steps of: determining a first statistical item for the received database statement, wherein the first statistical item is used to describe the data distribution of the data table corresponding to the database statement; determining the latest second statistical value of the first statistical item based on the first statistical value of the first statistical item at the historical time; determining the first execution path of the database statement based on the second statistical value of the first statistical item, and the first execution path represents the operation steps adopted by the database statement during execution. Through the application, the efficiency of the execution path generation can be improved, and the resource consumption of the execution path generation is reduced.

Description

Technical Field

[0001] This application relates to database technology, and more particularly to an information generation method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Because database operations often involve repetitive use of the same database statements, electronic devices typically cache the execution paths of these statements to reduce the overhead of calculating execution plans. This allows them to directly retrieve the execution plan from the cache for the same statement, avoiding redundant calculations. However, these technologies suffer from inefficient execution path generation and high resource consumption. Summary of the Invention

[0003] This application provides an information generation method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the efficiency of execution path generation and reduce the resource consumption of execution path generation.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides an information generation method, the method comprising:

[0006] For the received database statement, a first statistical item is determined, wherein the first statistical item is used to describe the data distribution of the data table corresponding to the database statement;

[0007] Based on the first statistical value of the first statistical item in historical time, determine the latest second statistical value for the first statistical item;

[0008] Based on the second statistical value of the first statistical item, a first execution path for the database statement is determined, wherein the first execution path represents the operation steps adopted by the database statement during execution.

[0009] This application provides an information generation apparatus, including:

[0010] The statistical item determination module is used to determine a first statistical item for a received database statement, wherein the first statistical item is used to describe the data distribution of the data table corresponding to the database statement;

[0011] The statistical value determination module is used to determine the latest second statistical value for the first statistical item based on the first statistical value of the first statistical item in historical time.

[0012] The path determination module is used to determine a first execution path of the database statement based on the second statistical value of the first statistical item, wherein the first execution path represents the operation steps adopted by the database statement during execution.

[0013] In the above scheme, the statistical value determination module is further configured to generate a statistical value sequence using the first statistical value of the historical time, and determine the sequence type of the statistical value sequence; based on the sequence type and the statistical value sequence, determine the second statistical value of the first statistical item.

[0014] In the above scheme, the statistical value determination module is further used to determine the candidate prediction model that matches the sequence type among multiple candidate prediction models as the target prediction model; and to determine the second statistical value of the first statistical item based on the statistical value sequence through the target prediction model.

[0015] In the above scheme, the target prediction model includes: multiple sub-prediction models; the statistical value determination module is further configured to predict a third statistical value based on the statistical value sequence through each of the sub-prediction models; merge the multiple third statistical values ​​corresponding to the multiple sub-prediction models, and determine the fusion result as the second statistical value.

[0016] In the above scheme, the statistical value determination module is further used to filter multiple prediction inputs from the statistical value sequence based on the model order of the target prediction model, and determine multiple random errors based on the model order of the target prediction model; through the target prediction model, based on the multiple prediction inputs and the multiple random errors, to predict the statistical value of the first statistical item, and determine the prediction result as the second statistical value.

[0017] In the above scheme, the information generation device further includes: a model determination module, used to determine the degree of matching between each candidate prediction model and the sequence type based on the sample statistical value sequence of the sequence type; and to determine, based on the degree of matching, a candidate prediction model that matches the sequence type from among multiple candidate prediction models.

[0018] In the above scheme, the model determination module is further configured to sample a sub-sample sequence from the sample statistical value sequence of the sequence type, and determine the sample statistical value located after the sub-sample sequence in the sample statistical value sequence as the true value result; predict the sample statistical value of the sub-sample sequence through each candidate prediction model to obtain the prediction result of each candidate prediction model; and determine the matching degree between each candidate prediction model and the sequence type by using the difference between the true value result and the prediction result of each candidate prediction model.

[0019] In the above scheme, the statistical value determination module is further used to determine the change characteristics of the statistical value sequence; and based on the change characteristics, to determine the sequence type of the statistical value sequence.

[0020] In the above scheme, the statistical item determination module is further configured to decompose the database statement into multiple sub-operation statements; determine the associated second statistical item for each sub-operation statement; and determine the first statistical item of the database statement using the second statistical items of the multiple sub-operation statements.

[0021] In the above scheme, the statistical item determination module is further used to determine the operation type corresponding to the sub-operation statement; and to determine the statistical item required by the operation type as the second statistical item.

[0022] In the above scheme, the path determination module is further configured to generate multiple candidate second execution paths based on the second statistical value; calculate the first execution cost of each second execution path, wherein the first execution cost is used to describe the resources and time consumed by the second execution path during execution; and determine the second execution path that corresponds to the minimum first execution cost among the multiple second execution paths as the first execution path.

[0023] In the above scheme, the path determination module is further configured to determine a first expiration time and a first update time for the first execution path, wherein the first expiration time refers to the time when the first execution path becomes invalid, and the first update time refers to the time when the execution path is regenerated for the database statement; and to cache the first execution path, the first expiration time, and the first update time.

[0024] In the above scheme, the path determination module is further configured to predict the change time of the first statistical item based on the first statistical value and the second statistical value, wherein the change time refers to the time when the degree of change of the statistical value is greater than the degree threshold; determine the first expiration time based on the change time, and determine the first update time from the time before the first expiration time.

[0025] In the above scheme, the statistical item determination module is further configured to perform hash encoding on the database statement to obtain a hash value; based on the hash value, search the cache for the third execution path currently used by the database statement; when the second update time of the third execution path is reached, determine the first statistical item for the received database statement.

[0026] In the above scheme, the path determination module is further configured to switch the execution path of the database statement from the third execution path to the first execution path when the second expiration time of the third execution path is reached.

[0027] In the above scheme, the path determination module is further used to calculate the second execution cost of the first execution path and the third execution cost of the third execution path; when the third execution cost is greater than the second execution cost, the execution path of the database statement is switched from the third execution path to the first execution path.

[0028] This application provides an electronic device, the electronic device comprising:

[0029] Memory is used to store executable instructions or computer programs.

[0030] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the information generation method provided in the embodiments of this application.

[0031] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the information generation method provided in this application.

[0032] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the information generation method provided in this application.

[0033] The embodiments of this application have the following beneficial effects: First, the required first statistical item is determined for the database statement. Then, the first statistical value of the first statistical item in historical time is used to predict the statistical value of the first statistical item in the current time or future time, that is, the latest second statistical value. Then, the first execution path is generated based on the predicted second statistical value. In this way, by predicting the statistical value, the number of times the statistical value is collected can be reduced, thereby reducing the time and resource consumption caused by the collection of statistical values ​​in the execution path generation process, thus improving the efficiency of execution path generation and reducing the resource consumption of execution path generation. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the architecture of the information generation system provided in the embodiments of this application;

[0035] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0036] Figure 3This is a flowchart illustrating the information generation method provided in the embodiments of this application. Figure 1 ;

[0037] Figure 4 This is a flowchart illustrating the information generation method provided in the embodiments of this application. Figure 2 ;

[0038] Figure 5 This is a flowchart illustrating the information generation method provided in the embodiments of this application. Figure 3 ;

[0039] Figure 6 This is a flowchart illustrating the information generation method provided in the embodiments of this application. Figure 4 ;

[0040] Figure 7 This is a schematic diagram of the architecture of the database system provided in the embodiments of this application;

[0041] Figure 8 This is a schematic diagram of the search plan provided in the embodiments of this application;

[0042] Figure 9 This is a schematic diagram illustrating the change in the amount of data in the data table provided in the embodiments of this application. Figure 1 ;

[0043] Figure 10 This is a comparative diagram of the prediction results of linear regression and random forest provided in the embodiments of this application;

[0044] Figure 11 This is a schematic diagram illustrating the change in the amount of data in the data table provided in the embodiments of this application. Figure 2 ;

[0045] Figure 12 This is a schematic diagram of the statistical information prediction process provided in the embodiments of this application;

[0046] Figure 13 This is a schematic diagram of the cache of the query plan provided in the embodiments of this application;

[0047] Figure 14 This is a schematic diagram of the query plan elimination mechanism provided in the embodiments of this application;

[0048] Figure 15 This is a schematic diagram illustrating the switching of the query plan provided in the embodiments of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0051] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0052] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0053] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0054] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0055] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0056] 1) Database statements are commands or instructions used to manipulate a database. They direct the database management system to perform specific operations on the database. Common database statements include: SELECT statements, INSERT statements, DELETE statements, etc. Users can use these statements to interact with the database and manipulate the data within it.

[0057] 2) Statistical items refer to information items collected and maintained regarding the distribution of data in a data table. Statistical items may include: column distribution, index information, the number of unique values, etc. In this embodiment, each statistical item has its corresponding numerical value, such as the specific value of the average value, which is called the statistical value of the statistical item.

[0058] 3) Execution path refers to the specific methods or steps used when executing a database statement. Different execution paths result in different resource consumption and execution time. Taking a query statement as an example, the execution path can include: full table scan, index scan, etc., and these two scan methods require different resources and have different execution times.

[0059] 4) Execution cost refers to the cost incurred when executing database operations, such as resource consumption and time required. Execution cost is an important indicator for analyzing the quality of execution paths.

[0060] The execution path of a database statement refers to the specific method or strategy used when executing a database statement. Each database statement needs to be executed according to its execution path. Since the same database statements may be used repeatedly for operations, electronic devices typically cache the execution paths of database statements to reduce the overhead of calculating execution plans. This allows them to directly retrieve and use the execution plan for the same statement, avoiding redundant calculations. However, the data in a database is constantly changing, and over time, the execution plan may no longer be applicable to the current database, necessitating the timely generation of new execution paths.

[0061] In related technologies, there are two types of execution path generation methods: one type generates the execution path based on statistical information during program compilation and stores the generated execution path in a system table, and re-executes the path after each collection of statistical information; the other type first checks whether a corresponding execution plan exists in the cache when executing database statements, and if it does not exist, it generates the corresponding execution plan based on statistical information and inserts it into the cache.

[0062] However, both of the above methods require the collection of statistical data, i.e., the collection of statistical values, before execution path generation can proceed. However, collecting statistical data consumes time and computational resources, such as memory and I / O resources. Frequent collection of statistical values ​​can lead to significant time and system resource consumption, resulting in inefficient execution path generation and substantial resource consumption.

[0063] This application provides a method, apparatus, device, computer-readable storage medium, and computer program product for generating execution paths, which can improve the efficiency of execution path generation and reduce the resource consumption of execution path generation. The following describes exemplary applications of the electronic devices provided in this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. The following describes exemplary applications when the electronic device is implemented as a server.

[0064] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of an information generation system provided in this application embodiment. To support an information generation application, in the information generation system 100, a terminal 600 (terminals 600-1 and 600-2 are shown as examples) connects to a server 200 via a network 300. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. The information generation system 100 also includes a database 500 to provide data support to the server 200. The database 500 can be configured within the server 200 or can be independent of the server 200. Figure 1 This illustrates the scenario where database 500 is independent of server 200.

[0065] This application embodiment can be implemented collaboratively by a server and a terminal. For example, terminal 600 sends a database statement to server 200. Server 200 receives the database statement and processes it using the information generation method provided in this application embodiment to obtain the execution path of the database statement. This allows subsequent processing of the database statement based on the execution path to obtain the same processing result. Server 200 then returns a processing completion message to terminal 600 based on the processing result. Terminal 600 displays the processing completion message on a graphical interface (graphical interfaces 600-11 and 600-21 are shown as examples).

[0066] The information generation method provided in this application can be applied to various business scenarios that require the use of databases, such as order query scenarios on e-commerce websites and transaction data writing scenarios in financial systems. The following describes the business scenarios to which the information generation method provided in this application can be applied.

[0067] 1) In the order query scenario of an e-commerce website, for example, the terminal sends a query statement for querying orders to the server. Based on the information generation method provided in the embodiments of this application, the server generates an execution path for the query statement and caches the execution path so that when the same query statement is received in the future, the query statement can be processed directly based on the execution path.

[0068] 2) In the scenario of writing transaction data in a financial system, for example, the terminal sends an insert statement for writing transaction data to the server. The server generates an execution path for the insert statement based on the information generation method provided in the embodiments of this application, and immediately processes the insert statement based on the execution path.

[0069] It should be noted that in some embodiments, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, and this application embodiment does not impose any restrictions.

[0070] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 The illustrated electronic device 400 includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0071] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0072] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0073] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0074] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0075] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0076] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0077] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0078] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0079] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0080] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2An information generation device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a statistical item determination module 4551, a statistical value determination module 4552, a path determination module 4553, and a model determination module 4554. These modules are logically connected and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0081] In other embodiments, the information generation apparatus provided in this application can be implemented in hardware. As an example, the information generation apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the information generation method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0082] In some embodiments, an electronic device can implement the information generation method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as database management APPs; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0083] The information generation method provided in this application will now be described in conjunction with exemplary applications and implementations of the electronic devices provided in the embodiments of this application. As mentioned above, the electronic device implementing the information generation method of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0084] See Figure 3 , Figure 3 This is a flowchart illustrating the information generation method provided in the embodiments of this application. Figure 1 , will combine Figure 3 The steps shown are explained below. Figure 3 The main component of the process is electronic equipment.

[0085] Step 101: For the received database statement, determine the first statistical item, which is the statistical item used to determine the execution path of the database statement.

[0086] This application embodiment is implemented in the scenario of generating corresponding execution paths for database statements. First, the electronic device parses the statistical items of the received database statement to determine the statistical items used when generating the corresponding execution path for the database statement. These statistical items are recorded as the first statistical item. In other words, the obtained first statistical item is the statistical item required to generate the execution path for the database statement.

[0087] It should be noted that a database statement refers to a command or instruction used to manipulate a database, which instructs the database management system to perform specific operations on the database. In this embodiment, a database statement may include a SELECT statement, an INSERT statement, a DELETE statement, and so on. The execution path refers to the specific method or operation steps used when executing a database statement; it determines how the database statement is executed. Taking a query statement as an example, the execution path may include: a full table scan, an index scan, and so on.

[0088] A statistical item refers to an information item describing the data distribution in the collected and maintained database. Since a database statement can operate on at least one data table in the database, the first statistical item determined for the database statement is used to describe the data distribution of the data table corresponding to the database statement. The data table corresponding to the database statement can have multiple different statistical items. The first statistical item can include a portion of all statistical items of the data table, or it can include all statistical items of the data table. It should be noted that the first statistical item can include statistical information of the data table, such as the number of rows in the data table, the size of the data table, etc., and can also include column statistical information, such as the number of null values ​​in the column, the maximum value of the column, the average value of the column, etc., and can also include index information, such as the size of the index (i.e., the storage space occupied by the index), the depth of the index (i.e., the height of the index tree), etc. This embodiment of the application does not limit this.

[0089] It should be noted that, in this embodiment, the electronic device can determine the first statistical item and begin subsequent processing immediately after receiving a database statement, thereby enabling the generation of an execution path for the received database statement to begin immediately. Alternatively, the electronic device can determine the first statistical item and begin subsequent processing only after a specific time point has elapsed, thus enabling the generation of an execution path for the received database statement to begin only after the specific time point has elapsed. This specific time point can be a manually set time point, such as midnight every day, or it can be the update time of an existing execution path for the database statement.

[0090] If the execution path generated by the electronic device in response to a database statement is denoted as the first execution path, then the existing execution paths of the database statement can be denoted as the third execution path. In some embodiments of this application, Figure 3 Step 101, namely determining the first statistical item for the received database statement, can be achieved through the following processing: hashing the database statement to obtain a hash value; based on the hash value, searching the cache for the third execution path currently used by the database statement; when the second update time of the third execution path is reached, determining the first statistical item for the received database statement.

[0091] The electronic device first uses a hash algorithm to hash the database statement, converting it into a unique hash value. This hash value is then used as an index to check if an execution path corresponding to the database statement exists in the cache. If it does, the execution path is retrieved and recorded as the third execution path. Next, the electronic device obtains the update time of the third execution path and designates it as the second update time. The second update time refers to the time when the third execution path needs to be updated, i.e., the time when a new execution path needs to be generated for the database statement. The electronic device then compares the current time with the second update time to determine if the second update time has been reached. Only when the second update time has been reached will the electronic device perform statistical analysis on the database statement to obtain the first statistical item, which will then initiate the execution path generation process for the database statement, resulting in the latest first execution path. If the current time has not reached the second update time, the electronic device will not initiate execution path generation for the database statement but will continue processing the database statement based on the third execution path.

[0092] It should also be noted that in some embodiments, the execution path also has an expiration time, which represents the time when the execution path becomes invalid. After the electronic device finds the execution path from the cache based on the hash value, it will determine whether the execution path located by the hash value has become invalid based on the expiration time of the execution path. If it has not become invalid, the located execution path will be determined as the third execution path currently used by the database statement, and the database statement will continue to be processed based on the third execution path. When the second update time is reached, the corresponding first statistical item will be determined for the database statement to start the generation of a new execution path. If it has become invalid, the electronic device can immediately analyze the statistical item for the database statement to start the generation of the execution path for the database statement.

[0093] It is understood that, in this embodiment of the application, the electronic device can locate the third execution path currently used by the database statement from the cache based on the hash value of the database statement. Only when the second update time of the third execution path is reached will the determination of the first statistical item of the database statement begin, so as to start the generation process of the new execution path. In this way, the generation of the execution path for the database statement can be automatically started when a specific time is reached, so that the execution path of the database statement can be updated on a regular basis.

[0094] In some embodiments of this application, Figure 3 Step 101, which is to determine the first statistical item for the received database statement, can be achieved through the following process: decompose the database statement into multiple sub-operation statements; determine the second statistical item associated with each sub-operation statement based on the operation type of each sub-operation statement; and determine the first statistical item of the database statement using the second statistical items of multiple sub-operation statements.

[0095] The electronic device can first decompose the database statement into multiple sub-operation statements, each corresponding to a database operation type, such as query or delete. Next, the electronic device will determine the required statistical items for each sub-operation statement to obtain the second statistical item for each sub-operation statement. Then, the second statistical items of all sub-operation statements will be directly determined as the first statistical item, or the deduplication result of the second statistical items of all sub-operation statements will be determined as the first statistical item.

[0096] It should be noted that the second statistical item of each sub-operation statement is related to the operation type corresponding to that sub-operation statement. Therefore, the electronic device can determine the associated second statistical item by combining the operation type corresponding to the sub-operation statement.

[0097] In some embodiments of this application, determining the associated second statistical item for each of the sub-operation statements mentioned above can be achieved through the following process: determining the operation type corresponding to the sub-operation statement; and determining the statistical item required for the operation type as the second statistical item.

[0098] In other words, the electronic device can first analyze the operation type of each sub-operation statement, such as filtering, grouping, and sorting, and then directly use the statistical items required for these operation types as the second statistical items associated with the sub-operation statements. The statistical items required for each operation type can be manually preset.

[0099] For example, when the operation type corresponding to the sub-operation statement is filtering, the electronic device can determine the statistical item required by the filtering operation, namely the number of unique values ​​in the column, as the second statistical item of the sub-operation statement; when the operation type corresponding to the sub-operation statement is join, the electronic device can determine the statistical item required by the join operation, such as the distribution of columns, as the second statistical item of the sub-operation statement.

[0100] It is understood that, in the embodiments of this application, the electronic device can determine the second statistical item of the sub-operation statement based on the operation type of the sub-operation statement and the statistical items required by the operation type, thereby making the determination of the second statistical item simpler.

[0101] In some other embodiments of this application, the determination of the associated second statistical item for each sub-operation statement as described above can also be achieved by the following process: analyzing the sub-operation statement through a large language model to obtain the second statistical item of the sub-operation statement.

[0102] It is understood that in the embodiments of this application, the electronic device can first decompose the database statement, and then use the second statistical item of each sub-operation statement to determine the first statistical item of the database statement. In this way, the first statistical item can be determined more comprehensively, thereby ensuring the accuracy of the first statistical item.

[0103] In other embodiments of this application, Figure 3 Step 101, which is to determine the first statistical item for the received database statement, can also be achieved through the following process: by analyzing the statistical items of the database statement through a large language model, the first statistical item can be obtained.

[0104] In other words, electronic devices can directly utilize the analytical capabilities of large language models to analyze database statements and obtain the first statistical item.

[0105] Step 102: Based on the first statistical value of the first statistical item in historical time, determine the latest second statistical value for the first statistical item.

[0106] After determining the first statistical item, the electronic device obtains the historical statistical value of the first statistical item and sets the obtained statistical value as the first statistical value. Then, based on the historical first statistical value, the electronic device predicts the statistical value of the first statistical item at the current time or in the future, and uses the predicted statistical value as the second statistical value. It can be seen that the second statistical value is the latest statistical value predicted for the first statistical item.

[0107] It should be noted that statistical values ​​refer to the specific numerical values ​​of statistical items, such as the specific value of the average value, the specific value of the number of rows in the data table, etc. The first statistical value of historical time can be collected and stored in real time by the electronic device, which can then retrieve it directly from the storage space when needed; it can also be obtained by the electronic device during idle periods, i.e., periods with no database statements to process, or periods with few database statements to process, based on database logs; or it can be predicted by a machine learning model from statistical values ​​of times prior to historical time, i.e., even more distant times. This application embodiment does not limit this. Historical time refers to a time period preceding the current time in chronological order, and historical time can include multiple time points. It should be noted that the number of time points included in historical time, and the unit of time points, can be set according to actual needs. For example, historical time can include 90 days (in days) before the current time, or 100 seconds (in seconds) before the current time, or 30 hours before the current time, etc.

[0108] In this embodiment of the application, the determination of the latest second statistical value based on the first statistical value can be achieved through various processing methods.

[0109] See Figure 4 , Figure 4 This is a flowchart illustrating the information generation method provided in the embodiments of this application. Figure 2 In some embodiments of this application, Figure 3 Step 102, which is to determine the latest second statistical value for the first statistical item based on the first statistical value in historical time, can be achieved through the following processing:

[0110] Step 1021: Using the first statistical value from historical time, generate a statistical value sequence and determine the sequence type of the statistical value sequence.

[0111] It should be noted that historical time includes multiple points in time, each with its own first statistical value. Therefore, there are multiple first statistical values ​​for historical time. Electronic devices utilize these multiple first statistical values, arranging them in chronological order to form a sequence, resulting in a statistical value sequence. Then, type analysis is performed on this statistical value sequence to determine its sequence type. The sequence type characterizes the changes in the first statistical values ​​within the sequence.

[0112] In some embodiments of this application, Figure 4 Step 1021, namely determining the sequence type of the statistical value sequence, can be achieved through the following process: determining the variation characteristics of the statistical value sequence; and determining the sequence type of the statistical value sequence based on the variation characteristics.

[0113] Electronic devices can first extract the change features of a statistical value sequence. These change features are those that reflect the changing patterns of the statistical value sequence. Then, based on these change features, the electronic devices can analyze the type of the statistical value sequence and thus obtain the sequence type.

[0114] In this application embodiment, the sequence type may include: stable growth type, periodic random variation type, and steady fluctuation type, etc. Among them, stable growth type refers to a type that grows steadily at the same amplitude; even if there are some fluctuations in the data within the sequence, and some regular deletions, as long as it grows steadily overall, it can be considered a stable growth type sequence. Periodic random variation type refers to a sequence where the data is a random value within a period, but fluctuates periodically according to a certain period. For example, the data is a random value within a single period, but fluctuates periodically with a period of 100. Steady fluctuation type refers to a sequence without obvious trend or period, where the data fluctuations revolve around a fixed mean, i.e., the variance is constant, and the statistical characteristics do not change over time.

[0115] It should be noted that the change characteristics can include statistical characteristics, trend characteristics, periodic characteristics, etc., of the statistical value sequence. Among them, statistical characteristics are used to characterize the distribution characteristics of the statistical value sequence, which can include the mean, variance, skewness, etc.; trend characteristics are used to characterize whether the statistical value sequence has a long-term upward or downward trend, which can include the duration of the upward or downward trend and the degree of each rise or fall; periodic characteristics are used to characterize whether the statistical value sequence has fixed periodic fluctuations, which can include the period duration and the average fluctuation range of different periods.

[0116] In this embodiment of the application, the electronic device can judge the obtained change features according to the preset rules of different sequence types, and take the type corresponding to the preset rule hit by the change features as the sequence type of the statistical value sequence; the electronic device can also process the obtained change features through a trained machine learning model, such as a random forest model, and take the type of the output of the machine learning model as the sequence type of the statistical value sequence.

[0117] It is understood that in this embodiment of the application, the electronic device will determine the change characteristics of the statistical value sequence, and then determine the sequence type of the statistical value sequence based on the obtained change characteristics. In this way, when determining the sequence type of the statistical value sequence, the change law of the statistical value sequence can be fully considered, thereby improving the accuracy of the sequence type.

[0118] In other embodiments of this application, Figure 4 Step 1021, which is to determine the sequence type of the statistical value sequence, can also be achieved through the following process: read in the statistical value sequence through the sequence classification model, and determine the output of the sequence classification model as the sequence type of the statistical value sequence.

[0119] It should be noted that the sequence classification model can be a deep learning model, such as a Long Short-Term Memory network or a Recurrent Neural Network; it can also be a shallow machine learning model, such as a Random Forest model or a Gradient Boosting Machine. The embodiments in this application are not intended to limit this. The sequence classification model can acquire the ability to determine the type of a sequence through training. The training data for the sequence classification model can consist of sample sequences and the true types of the sample sequences. During training, the electronic device reads the sample sequences through an initial classification model, classifies them, and then calculates the difference between the predicted type output by the initial classification model and the true type of the sample sequence. This difference is used to determine the loss value, and the model parameters of the initial classification model are adjusted based on the loss value to obtain the sequence classification model.

[0120] This completes the determination of the sequence type.

[0121] Step 1022: Based on the sequence type and the statistical value sequence, determine the second statistical value of the first statistical term.

[0122] Since the sequence type affects the prediction method of statistical values, different methods are needed to process the statistical value sequence for different sequence types in order to predict the latest statistical value. Therefore, in this embodiment, the electronic device needs to combine the sequence type with the statistical value sequence to more accurately predict the statistical value of the first statistical item at the current time or in the future, and determine the predicted statistical value as the second statistical value.

[0123] See Figure 5 , Figure 5 This is a flowchart illustrating the information generation method provided in the embodiments of this application. Figure 3 In some embodiments of this application, Figure 4 Step 1022, namely, determining the second statistical value of the first statistical term based on the sequence type and the statistical value sequence, can be achieved through the following processing:

[0124] Step 10221: Select the candidate prediction model that matches the sequence type from among multiple candidate prediction models as the target prediction model.

[0125] The electronic device can first acquire multiple candidate prediction models, each of which is set for a specific sequence type. Then, the electronic device will directly determine the candidate prediction model set for the sequence type of the statistical value, that is, the candidate prediction model that matches the sequence type of the statistical value sequence, as the target prediction model.

[0126] For example, if the sequence type is a stable growth type, the candidate prediction model that matches this sequence type can be a random forest model or a linear regression model. Thus, the electronic device will directly use the random forest model or the linear regression model as the target prediction model. If the sequence type is a periodic random variation type, the candidate prediction model that matches this sequence type can be an autoregressive moving average model. Thus, the electronic device will directly use the autoregressive moving average model as the target prediction model.

[0127] It should be noted that the candidate prediction model matching the sequence type of the statistical value sequence can be determined before the first statistical term is determined for the database statement, that is, before the generation process of the first execution path of the database statement begins. This determination process will be described below.

[0128] Step 10222: Using the target prediction model, determine the second statistical value of the first statistical term based on the statistical value sequence.

[0129] The electronic device reads in the statistical value sequence through the determined target prediction model, and predicts the statistical value based on the read statistical value sequence to obtain the predicted statistical value. Then, the electronic device determines the predicted statistical value output by the target prediction model as the second statistical value.

[0130] It should be noted that, as shown above, the target prediction model will differ depending on the type of the statistical value sequence, thus requiring different processing methods for the statistical value sequence. Below, we will explain the specific process for determining the second statistical value in conjunction with the target prediction model.

[0131] In some embodiments of this application, the target prediction model includes multiple sub-prediction models, in which case, Figure 5 Step 10222, which is to determine the second statistical value of the first statistical term based on the statistical value sequence through the target prediction model, can be achieved by the following process: predicting the third statistical value based on the statistical value sequence through each sub-prediction model; fusing multiple third statistical values ​​corresponding to multiple sub-prediction models, and determining the fusion result as the second statistical value.

[0132] When the target prediction model includes multiple sub-prediction models, the electronic device sequentially uses each sub-prediction model to predict the latest statistical value based on the statistical value sequence, or simultaneously uses each sub-prediction model to predict the latest statistical value based on the statistical value sequence, thus obtaining multiple prediction results corresponding to each sub-prediction model, i.e., multiple third statistical values. Then, the electronic device weights and merges these multiple third statistical values, or averages them, or allows them to vote to obtain the final second statistical value.

[0133] For example, in this embodiment of the application, the target prediction model can be a random forest model. The electronic device can predict the latest statistical value based on the statistical value sequence through each decision tree in the random forest model to obtain a third statistical value, and then average the third statistical values ​​predicted by all decision trees to obtain a second statistical value.

[0134] It is understood that in the embodiments of this application, when the target prediction model includes multiple sub-prediction models, the electronic device can use the multiple sub-prediction models to predict the statistical values ​​separately, so as to make full use of the prediction capabilities of different sub-prediction models, capture the pattern changes at different time granularities, and then merge the third statistical values ​​predicted by the multiple sub-prediction models into the second statistical value, reduce the impact of extreme prediction values, and thus make the second statistical value more robust.

[0135] In other embodiments of this application, Figure 5 Step 10222, which is to determine the second statistical value of the first statistical term based on the statistical value sequence through the target prediction model, can also be achieved through the following processing: based on the model order of the target prediction model, multiple prediction inputs are selected from the statistical value sequence, and multiple random errors are determined based on the model order of the target prediction model; based on the multiple prediction inputs and multiple random errors, the statistical value of the first statistical term is predicted through the target prediction model, and the prediction result is determined as the second statistical value.

[0136] The electronic device can first utilize the model order of the target prediction model to select several historical time points as the first statistical values ​​from the statistical value sequence, which are then used as the inputs for the target prediction model during prediction. Simultaneously, based on the model order, the prediction errors of several historical time points are selected to obtain multiple random errors. Next, the electronic device reads these multiple prediction inputs and random errors through the target prediction model to predict the latest statistical value. The prediction output of the target prediction model is then determined as the second statistical value.

[0137] It should be noted that the model order of the target prediction model describes the autoregressive and moving average effects of the statistical value sequence, and this model order can be determined during the training phase of the target prediction model. In the embodiments of this application, the target prediction model may include an autoregressive moving average model. Therefore, the model order of the target prediction model may include the order of the autoregressive model and the order of the moving average model. Thus, the electronic device can filter the prediction input from the statistical value sequence according to the order of the autoregressive model, and filter multiple random errors from the prediction errors at different historical time points according to the order of the moving average model.

[0138] For example, the electronic device can obtain the first statistical values ​​of the order of the autoregressive model from the statistical value sequence in reverse order, and determine the obtained first statistical values ​​as the prediction input; the electronic device can obtain the prediction errors of the order of the moving average model before the current time, and treat these errors as random errors. Then, the electronic device can substitute the obtained prediction input and random errors into the calculation formula of the target prediction model to obtain the second statistical value.

[0139] It is understood that, in the embodiments of this application, the electronic device can determine the prediction input and random error of the target prediction model according to the model order, and then use the target prediction model to calculate the obtained prediction input and random error to obtain the second statistical value. In this way, the influence of error can be fully considered in the process of determining the second statistical value, thereby making the obtained second statistical value more accurate.

[0140] Thus, the second statistical value was determined based on the target prediction model.

[0141] It is understood that in the embodiments of this application, the electronic device can first select a suitable target prediction model according to the sequence type, and predict the second statistical value through the selected target prediction model. In this way, it can be ensured that the most suitable prediction method is selected for the statistical value sequence to predict the statistical value, thereby ensuring the accuracy of the second statistical value.

[0142] In other embodiments of this application, Figure 4Step 1022, which is to determine the second statistical value of the first statistical term based on the sequence type and the statistical value sequence, can also be achieved through the following process: read in the sequence type and the statistical value sequence through the statistical value prediction model, and determine the output of the statistical value prediction model as the second statistical value.

[0143] In other words, electronic devices can also directly provide the sequence type and statistical value sequence to the statistical value prediction model, so that the statistical value prediction model can predict the latest second statistical value for the first statistical term. The statistical value prediction model can be trained to acquire the ability to predict the second statistical value based on the sequence type and statistical value sequence; the training process will not be elaborated here.

[0144] It is understood that in the embodiments of this application, the electronic device can simultaneously combine the sequence type and the statistical value sequence to predict the latest second statistical value of the first statistical item, so as to achieve more accurate prediction of the statistical value based on the influence of the sequence type and ensure the accuracy of the second statistical value.

[0145] Step 103: Determine the first execution path of the database statement based on the second statistical value of the first statistical item.

[0146] After obtaining the second statistical value of the first statistical item, the electronic device generates an execution path for the database statement based on the second statistical value, and uses the generated execution path as the first execution path. Thus, the first execution path represents the operational steps adopted when the database statement is executed, and it provides corresponding processing strategies for the execution of the database statement.

[0147] In some embodiments of this application, Figure 3 Step 103, namely determining the first execution path of the database statement based on the second statistical value of the first statistical item, can be achieved through the following process: generating multiple candidate second execution paths based on the second statistical value; calculating the first execution cost of each second execution path; and determining the second execution path that corresponds to the minimum first execution cost among the multiple second execution paths as the first execution path.

[0148] In other words, the electronic device can first generate multiple different third execution paths based on the second statistical value, then calculate the cost of each second execution path, and select the second execution path with the lowest execution cost as the final first execution path for the database statement. This ensures that the first execution path of the database statement is always optimal.

[0149] It should be noted that the first execution cost describes the resources and time consumed by the third execution path during execution. Therefore, the resources can be computing resources, such as CPU resources, memory resources, etc., and the time can be the processing time, such as the time required to complete database statement processing based on the second execution path. The electronic device selects the optimal second execution path based on the first execution cost, which means selecting the optimal second execution path from both resource and time dimensions.

[0150] In this embodiment, the electronic device can first estimate the number of rows to be operated on by each operation node based on the second statistical value, and then select an access path based on the number of rows. Specifically, all feasible access paths can be determined based on the number of rows, and all feasible connection strategies can be determined based on the second statistical value. Finally, the access paths are combined according to the connection strategies to obtain multiple candidate second execution paths. Alternatively, the electronic device can directly output multiple candidate second execution paths based on the second statistical value using a large language model.

[0151] It should be noted that the electronic device can calculate the resource cost of each second execution path, which can be obtained by weighted summation of CPU utilization and memory utilization. Alternatively, it can calculate the time cost of each second execution path, and then weighted summation of the resource and time costs to obtain the final first execution cost. Of course, the electronic device can also input the second execution paths into a large language model, where the large language model will evaluate their execution costs to obtain the first execution cost.

[0152] It is understood that, in this embodiment of the application, the electronic device can first generate multiple different second execution paths based on the second statistical value, and then use the first execution cost to select the optimal second execution path from the multiple second execution paths as the first execution path. In this way, it can be ensured that the first execution path is the optimal execution path determined for the database statement, thereby ensuring the accuracy of the first execution path.

[0153] In other embodiments of this application, Figure 3 Step 103, namely determining the first execution path of the database statement based on the second statistical value of the first statistical item, can be achieved by the following process: comparing the second statistical value with the preset statistical values ​​of different preset execution paths, and determining the preset execution path corresponding to the preset statistical value with the smallest difference from the second statistical value as the first execution path.

[0154] In other words, the electronic device can select a suitable preset execution path from different preset execution paths based on the second statistical value, and use it as the first execution path. In this way, the first execution path can be determined more quickly.

[0155] Understandably, compared to related technologies, which suffer from low efficiency and high resource consumption due to the frequent collection of statistical values ​​for statistical items when generating execution paths for database statements, this embodiment of the application addresses this issue. The electronic device first determines the required first statistical item for the database statement, then uses the first statistical value of the first statistical item at a historical time to predict the statistical value of the first statistical item at the current or future time, i.e., the latest second statistical value. Finally, it generates the first execution path based on the predicted second statistical value. This reduces the number of times statistical values ​​are collected by predicting them, thereby reducing the time and resource consumption caused by collecting statistical values ​​during execution path generation, thus improving the efficiency of execution path generation and reducing resource consumption.

[0156] based on Figure 3 See Figure 6 , Figure 6 This is a flowchart illustrating the information generation method provided in the embodiments of this application. Figure 4 In some embodiments of this application, in Figure 3 After step 103, that is, after determining the first execution path of the database statement based on the second statistical value of the first statistical item, the method may further include the following processing:

[0157] Step 104: For the first execution path, determine the first expiration time and the first update time.

[0158] After determining the first execution path, the electronic device can further determine the expiration time and update time for the first execution path, and record the expiration time of the first execution path as the first expiration time and the update time of the first execution path as the first update time.

[0159] It should be noted that the first expiration time refers to the time when the first execution path becomes invalid. That is, when the first expiration time is reached, the electronic device can no longer process database statements based on the first execution path. The first update time refers to the time when the execution path is regenerated for the database statement. That is, when the first update time is reached, the electronic device needs to re-check whether the first execution path is reasonable and whether a new execution path needs to be regenerated.

[0160] In some embodiments of this application, Figure 6 Step 104, namely determining the first expiration time and the first update time for the first execution path, can be achieved through the following process: predicting the change time of the first statistical item based on the first statistical value and the second statistical value; determining the first expiration time based on the change time, and determining the first update time from the time before the first expiration time.

[0161] An electronic device can sort one or more first statistical values ​​and second statistical values ​​in chronological order to obtain a sequence, and based on this sequence, predict the change time for the first statistical item. The change time refers to the time when the degree of change of the statistical value is greater than the degree threshold. Therefore, the change time of the first statistical item can be understood as the time when the statistical value has undergone a huge change.

[0162] Since the change time refers to the time when a statistical value changes significantly, and when a significant change occurs, it indicates that the original execution path is no longer suitable and needs to be discarded, electronic devices can either directly determine the change time as the first expiration time, or they can use a point in time before the change time, such as one hour before the change time, as the first expiration time.

[0163] After determining the first expiration time, the electronic device can define the first update time as any point in time before the first expiration time, or a point in time before the first expiration time that is a preset time interval from the first expiration time. For example, three hours before the first expiration time can be used as the first update time. In this way, the first update time is before the first expiration time, thus ensuring that a new execution path is generated before the first execution path expires.

[0164] It is understood that in this embodiment of the application, the electronic device can determine the first expiration time based on the time when the first statistical item undergoes a significant change, thereby enabling the first execution path to be invalidated in a timely manner. At the same time, the first update time is determined from the time before the first expiration time, so that the electronic device can complete the generation of a new execution path before the first execution path expires, so that the database statement can be processed normally when the first expiration time is reached.

[0165] In other embodiments of this application, Figure 6 Step 104, which is to determine the first expiration time and the first update time for the first execution path, can also be achieved by the following processing: taking any time after the generation time of the first execution path as the first update time, and taking any time after the first update time as the first expiration time.

[0166] Step 105: Cache the first execution path, the first expiration time, and the first update time.

[0167] After determining the first expiration time and the first update time, the electronic device will store the first execution path, the first expiration time, and the first update time in memory or in a database used for data caching for later use.

[0168] It is understood that, in the embodiments of this application, the electronic device can determine the first expiration time and the first update data for the first execution path, and cache these data so that the first execution path can be processed accordingly based on these times in the future.

[0169] In some embodiments of this application, the first statistical item is determined for the database statement when the second update time of the third execution path is reached. That is, the first statistical item is determined for the received database statement only when the second update time of the third execution path is reached, in order to begin generating the execution path for the database statement. Figure 3 After step 103, that is, after determining the first execution path of the database statement based on the second statistical value of the first statistical item, the method may further include the following processing: when the second expiration time of the third execution path is reached, the execution path of the database statement is switched from the third execution path to the first execution path.

[0170] In other words, when a database statement is currently using the third execution path, and the second expiration time of the third execution path has not yet arrived, the electronic device will continue to process the database statement based on the third execution path until the second expiration time arrives. Only then will the third execution path be discarded, and the statement will be processed based on the first execution path. This is because the second statistical value may be a predicted value for a future time, and the first execution path generated based on the second statistical value may not be applicable to the current situation. Therefore, to ensure that the database statement is processed based on the optimal execution path, the execution path switch can wait until after the second expiration time.

[0171] It is understood that, in the embodiments of this application, the electronic device can switch the execution path for the database statement when the second expiration time of the third execution path is reached, so as to ensure that the database statement at the current time can be processed based on the third execution path that is most suitable for the current time.

[0172] In some embodiments of this application, in Figure 3 After step 103, that is, after determining the first execution path of the database statement based on the second statistical value of the first statistical item, the method may further include the following processing: calculating the second execution cost of the first execution path and calculating the third execution cost of the third execution path; when the third execution cost is greater than the second execution cost, switching the execution path of the database statement from the third execution path to the first execution path.

[0173] In other words, the electronic device can compare the second execution cost of the first execution path with the third execution cost of the third execution path. If the third execution cost is greater than the second execution cost, it means that the first execution path is better than the third execution path, and the electronic device can immediately switch to the third execution path. If the third execution cost is less than or equal to the second execution cost, it means that the third execution path is better than the first execution path or has the same degree of superiority or inferiority as the first execution path. In this case, the first execution path can be deleted, or the third execution path can still be used for processing.

[0174] It is understood that in the embodiments of this application, the electronic device determines the minimum execution cost by comparing the third execution cost with the second execution cost, and then always ensures that the database statement is processed based on the execution path with the minimum execution cost.

[0175] The process of determining candidate prediction models that match the sequence type in the above content will be explained below.

[0176] In some embodiments of this application, in Figure 5 Before step 10221, that is, before determining the candidate prediction model that matches the sequence type among multiple candidate prediction models as the target prediction model, the method may further include the following processing: determining the degree of matching between each candidate prediction model and the sequence type based on the sample statistical value sequence of the sequence type; and determining the candidate prediction model that matches the sequence type from multiple candidate prediction models based on the degree of matching.

[0177] The electronic device first obtains the corresponding sample statistical value sequence for the sequence type of the statistical value sequence. Then, based on the sample statistical value sequence, it analyzes whether each candidate prediction model is suitable for processing the statistical value sequence of the sequence type, thereby obtaining the matching degree. Then, the candidate prediction model with the highest matching degree, or the matching degree exceeding the degree threshold, is taken as the candidate prediction model that matches the sequence type of the statistical value sequence.

[0178] In some embodiments of this application, the determination of the matching degree between each candidate prediction model and the sequence type based on the sample statistical value sequence of the sequence type mentioned above can be achieved through the following process: sampling a sub-sample sequence from the sample statistical value sequence of the sequence type, and determining the sample statistical values ​​located after the sub-sample sequence in the sample statistical value sequence as the true value results; predicting the sample statistical values ​​of the sub-sample sequence using each candidate prediction model to obtain the prediction result of each candidate prediction model; and determining the matching degree between each candidate prediction model and the sequence type by using the difference between the true value and the prediction result of each candidate prediction model.

[0179] The electronic device first randomly samples a sub-sample sequence from the sample statistical value sequence. Then, it can directly use the statistical value of the next sample after the last statistical value of the sub-sample sequence as the true value, or it can use the statistical value of a sample at a time interval after the last statistical value of the sub-sample sequence as the true value. Next, the electronic device reads the sub-sample sequence through each candidate prediction model to predict subsequent statistical values ​​for the sub-sample sequence. Here, the candidate prediction model needs to predict the statistical value for the time point corresponding to the true value. The output of the candidate prediction model is the predicted statistical value. Afterwards, the electronic device can calculate the difference between the true value and the prediction results of each candidate prediction model. It can then directly determine the degree of matching between the candidate prediction model and the sequence type based on the difference, or it can use a deep learning model to predict the degree of matching based on this difference to determine the degree of matching between the candidate prediction model and the sequence type.

[0180] In this embodiment of the application, the electronic device can determine the difference between the true result and the predicted result by subtraction, or it can map the true result and the predicted result to the feature space and determine the difference between the true result and the predicted result by the reciprocal of the cosine similarity between the features of the true result and the features of the predicted result, or by the difference between the cosine similarity and 1.

[0181] It is understood that, in the embodiments of this application, the electronic device can compare the prediction results of the candidate prediction model with the true results to determine whether the candidate prediction model is suitable for processing the statistical value sequence of the sequence type, thereby accurately determining the degree of matching between the candidate prediction model and the sequence type.

[0182] In other embodiments of this application, the determination of the matching degree between each candidate prediction model and the sequence type based on the sample statistical value sequence described above can also be achieved through the following process: using a large model, a model suitable for processing the sample statistical value sequence is determined, and the model determined by the large model is compared with the candidate prediction models. Based on the comparison results, a candidate prediction model that matches the sequence type is determined from multiple candidate prediction models.

[0183] Electronic devices can read in sample statistical value sequences through a large model, so that the large model can analyze which models are suitable for processing the sample statistical value sequences. Then, the models that the large model has determined are suitable for processing the sample statistical value sequences, and are compared with candidate prediction models to determine whether they are similar. Candidate prediction models with a similarity greater than a threshold are selected as candidate prediction models that match the sequence type.

[0184] It is understood that in this embodiment of the application, the electronic device will analyze the degree of matching between the candidate prediction model and the sequence type based on the sample statistical value sequence of the sequence type, and then combine the degree of matching to more accurately select the candidate prediction model that matches the sequence type.

[0185] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0186] This application embodiment is implemented in a scenario where a query plan (called an execution path) is generated from a database query statement (called a database statement) and cached.

[0187] A database query statement can be executed through the following steps: 1. The user sends a query statement to the database system (running on an electronic device); 2. The database system's optimizer calculates the execution cost and generates a query plan based on statistical information (called the first statistic); 3. The database system executes the query plan.

[0188] In this embodiment, the focus is on step 2 above, namely, the process by which the database system's optimizer module calculates the execution cost (referred to as the first execution cost) based on statistical information and generates a query plan. This process is illustrated below.

[0189] Figure 7 This is a schematic diagram of the database system architecture provided in an embodiment of this application. See also... Figure 7The database system includes an optimizer (7-1), a statistics module (7-2), a machine learning control module (7-3), a query plan update module (7-4), a query plan cache module (7-5), and a system table (7-6). First, for a user-sent query, the optimizer (7-1) checks the cached system table (7-6) for a corresponding query plan (referred to as the third execution path) and determines if the plan is within its validity period (referred to as the second expiration time). If valid, the plan is used; otherwise, the statistics module (7-2) retrieves historical statistics (referred to as the first statistical value) and provides it to the machine learning control module (7-3). The machine learning control module (7-3) selects a suitable prediction model (referred to as the target prediction model) based on the statistics to predict current or future statistics (referred to as the second statistical value) and returns the predicted statistics to the statistics module (7-2). Simultaneously, the machine learning control module (7-3) trains the prediction model based on the historical statistics to facilitate statistical prediction. The statistics module 7-2 provides the predicted statistics to the optimizer 7-1. The optimizer calculates the query plan based on the returned statistics and provides the query plan to the query plan caching module 7-5 for caching. At the same time, the query plan update module 7-4 predicts the expiration time and update time of the query plan based on the predicted statistics and caches them.

[0190] As can be seen from the above, the embodiments of this application mainly involve the following steps:

[0191] 1. The optimizer checks the cache. If a query plan that has not expired exists in the cache, it is used directly.

[0192] In the query cache, each query plan has an expiration time and an update time. Figure 8 This is a schematic diagram illustrating the lookup of query plans provided in this application embodiment. Each query plan in the query cache, namely query plan 8-21 to query plan 8-24, has a corresponding expiration time and update time, namely expiration time 8-31 to expiration time 8-34 and update time 8-41 to update time 8-44. When the database system receives a new query statement, it hashes and encodes it, and uses hash table 8-1 to locate the query plan required by the query statement from different query plans, such as query plans 8-21 to query plan 8-24, and determines whether the query plan has expired based on the process time of that query plan.

[0193] 2. When the optimizer obtains statistical information for calculation, the machine learning control module needs to select an appropriate prediction model.

[0194] Different types of statistical information have different characteristics. Based on these characteristics, statistical information can be categorized as follows:

[0195] 1) Stable growth with consistent increments and data changes including insertions and deletions (referred to as stable growth). For generally stable business operations, the data volume in the tables will grow steadily. Data fluctuations may occur, but overall, the growth will be stable. Besides insertion operations, regular deletion operations are also necessary. For example, Figure 9 This is a schematic diagram illustrating the change in the amount of data in the data table provided in the embodiments of this application. Figure 1 .from Figure 9 As can be seen, although the amount of data in the data table 9-1 has decreased somewhat, the overall trend is one of steady growth.

[0196] For this type of statistical information, although there may be some changes, that is, the amount of data may fluctuate due to the addition or deletion of data, the overall trend is stable growth. In this embodiment of the application, linear regression or random forest can be used for prediction.

[0197] Figure 10 This is a schematic diagram comparing the prediction results of linear regression and random forest provided in the embodiments of this application. From Figure 10 As can be seen from the comparison of the actual data volume and the predicted data volume, the prediction results of linear regression 10-1 will fluctuate, while the prediction results of random forest 10-2 are basically close to the diagonal, indicating that random forest 10-2 has a better fitting effect.

[0198] 2) Random changes with periodicity. For some business processes, although the amount of data is random, its changes exhibit certain periodic patterns. Figure 11 This is a schematic diagram illustrating the change in the amount of data in the data table provided in the embodiments of this application. Figure 2 .from Figure 11 As can be seen, the data volume of the data table is 11-1, which is random data within a single period, but fluctuates with a period of 100.

[0199] For these types of statistical information—that is, statistical information that is approximately random but still exhibits a certain periodicity—this embodiment of the application uses an autoregressive moving average model for prediction. The autoregressive moving average model is generated by combining an autoregressive model and a moving average model, and can effectively capture the autocorrelation and random fluctuations in time series data.

[0200] 3. The machine learning control module predicts current or future statistical information based on the selected prediction model.

[0201] Database systems collect historical time series data of statistical information (called statistical value series) and use prediction models determined for the statistical information to predict current or future statistical information from the historical time series data.

[0202] Figure 12 This is a schematic diagram of the statistical information prediction process provided in the embodiments of this application. First, the database system collects historical data of statistical information to obtain historical data time series 12-1. Then, a machine learning model is established. Through the established model 12-2, the statistical information is predicted to generate prediction results and obtain statistical information 12-3. Then, the predicted statistical information is returned to the optimizer to generate a query plan 12-4, and the prediction model is fine-tuned using real statistical information.

[0203] 4. The optimizer calculates the query plan based on the statistical information predicted by the machine learning control module.

[0204] 5. Determine the expiration time and update time based on the predicted statistical information. When the expiration time is reached, the generated query plan will be invalidated, and when the update time is reached, a new query plan will be generated.

[0205] The database system compares the generated query plan with the current query plan. If they are inconsistent, the generated query plan is stored and waits for the query plan in the query cache to expire before replacing it, updating the expiration and update times of the current query plan. Specifically, the query plan update module in the database system sets an expiration time for the query plan based on changes in statistical information and selects an update time before the expiration time, so that a new query plan can be generated and updated before the current query plan expires. Due to errors in the prediction model, the query plan obtained based on predicted statistical information may not be better than the current query plan. Therefore, a query plan eviction mechanism is needed to prevent query plan degradation.

[0206] based on Figure 8 See Figure 13 , Figure 13 This is a schematic diagram of the query plan cache provided in this application embodiment. The newly generated query plan 13-1 by the optimizer, as well as the expiration time 13-2 and update time 13-3 generated by the machine learning control module, will be inserted into the query cache for storage.

[0207] Figure 14 This is a schematic diagram illustrating the query plan elimination mechanism provided in an embodiment of this application. See also... Figure 13 The elimination mechanism is implemented through the following processes: 1. The optimizer first generates a new query plan based on the predicted statistical information; 2. The current query plan is compared with the new query plan. If the new query plan is better, it is replaced; otherwise, it is rolled back to the current query plan; 3. Expired query plans are deleted from the cache to avoid excessive cleanup resources.

[0208] In this embodiment of the application, the query plan needs to be updated regularly. That is, when the query plan update time is reached, the query plan in the cache needs to be updated. Expired query plans will be replaced in a timely manner and cleaned up periodically.

[0209] Figure 15 This is a schematic diagram of the query plan switching provided in the embodiments of this application. When the expiration time 15-1 of query plan 1 is reached, the query plan of the query statement will be switched from query plan 1 to query plan 2. When the expiration time 15-2 of query plan 2 is reached, the query plan of the query statement will be switched from query plan 2 to query plan 3.

[0210] The above approach effectively reduces the number of times statistical information is collected, thereby reducing the consumption of system resources and improving efficiency. At the same time, setting an expiration time for query plans makes it more likely that the optimizer will select a more efficient query plan.

[0211] It is understood that in the embodiments of this application, user information, such as data in the database, is involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0212] The following description continues to illustrate the exemplary structure of the information generation device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software module stored in the information generation device 455 of the memory 450 may include:

[0213] The statistical item determination module 4551 is used to determine a first statistical item for a received database statement, wherein the first statistical item is used to describe the data distribution of the data table corresponding to the database statement;

[0214] The statistical value determination module 4552 is used to determine the latest second statistical value for the first statistical item based on the first statistical value of the first statistical item in historical time.

[0215] The path determination module 4553 is used to determine the first execution path of the database statement based on the second statistical value of the first statistical item, wherein the first execution path represents the operation steps adopted by the database statement during execution.

[0216] In the above scheme, the statistical value determination module 4552 is further configured to generate a statistical value sequence using the first statistical value of the historical time, and determine the sequence type of the statistical value sequence; and determine the second statistical value of the first statistical item based on the sequence type and the statistical value sequence.

[0217] In the above scheme, the statistical value determination module 4552 is further used to determine the candidate prediction model that matches the sequence type among multiple candidate prediction models as the target prediction model; and to determine the second statistical value of the first statistical item based on the statistical value sequence through the target prediction model.

[0218] In the above scheme, the target prediction model includes: multiple sub-prediction models; the statistical value determination module 4552 is further configured to predict a third statistical value based on the statistical value sequence through each of the sub-prediction models; merge the multiple third statistical values ​​corresponding to the multiple sub-prediction models, and determine the fusion result as the second statistical value.

[0219] In the above scheme, the statistical value determination module 4552 is further configured to filter multiple prediction inputs from the statistical value sequence based on the model order of the target prediction model, and determine multiple random errors based on the model order of the target prediction model; through the target prediction model, based on the multiple prediction inputs and the multiple random errors, predict the statistical value of the first statistical item, and determine the prediction result as the second statistical value.

[0220] In the above scheme, the information generation device 455 further includes: a model determination module 4554, used to determine the degree of matching between each candidate prediction model and the sequence type based on the sample statistical value sequence of the sequence type; and to determine the candidate prediction model that matches the sequence type from multiple candidate prediction models based on the degree of matching.

[0221] In the above scheme, the model determination module 4554 is further configured to sample a sub-sample sequence from the sample statistical value sequence of the sequence type, and determine the sample statistical value located after the sub-sample sequence in the sample statistical value sequence as the true value result; predict the sample statistical value of the sub-sample sequence through each candidate prediction model to obtain the prediction result of each candidate prediction model; and determine the matching degree between each candidate prediction model and the sequence type by using the difference between the true value result and the prediction result of each candidate prediction model.

[0222] In the above scheme, the statistical value determination module 4552 is further used to determine the change characteristics of the statistical value sequence; and based on the change characteristics, to determine the sequence type of the statistical value sequence.

[0223] In the above scheme, the statistical item determination module 4551 is further configured to decompose the database statement into multiple sub-operation statements; determine the associated second statistical item for each sub-operation statement; and determine the first statistical item of the database statement using the second statistical items of the multiple sub-operation statements.

[0224] In the above scheme, the statistical item determination module 4551 is further used to determine the operation type corresponding to the sub-operation statement; and to determine the statistical item required by the operation type as the second statistical item.

[0225] In the above scheme, the path determination module 4553 is further configured to generate multiple candidate second execution paths based on the second statistical value; calculate the first execution cost of each second execution path, wherein the first execution cost is used to describe the resources and time consumed by the second execution path during execution; and determine the second execution path that corresponds to the minimum first execution cost among the multiple second execution paths as the first execution path.

[0226] In the above scheme, the path determination module 4553 is further configured to determine a first expiration time and a first update time for the first execution path, wherein the first expiration time refers to the time when the first execution path becomes invalid, and the first update time refers to the time when the execution path is regenerated for the database statement; and to cache the first execution path, the first expiration time, and the first update time.

[0227] In the above scheme, the path determination module 4553 is further configured to predict the change time of the first statistical item based on the first statistical value and the second statistical value, wherein the change time refers to the time when the degree of change of the statistical value is greater than the degree threshold; determine the first expiration time based on the change time, and determine the first update time from the time before the first expiration time.

[0228] In the above scheme, the statistical item determination module 4551 is further configured to perform hash encoding on the database statement to obtain a hash value; based on the hash value, search the cache for the third execution path currently used by the database statement; when the second update time of the third execution path is reached, determine the first statistical item for the received database statement.

[0229] In the above scheme, the path determination module 4553 is further configured to switch the execution path of the database statement from the third execution path to the first execution path when the second expiration time of the third execution path is reached.

[0230] In the above scheme, the path determination module 4553 is further used to calculate the second execution cost of the first execution path and the third execution cost of the third execution path; when the third execution cost is greater than the second execution cost, the execution path of the database statement is switched from the third execution path to the first execution path.

[0231] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the information generation method described in this application.

[0232] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the information generation method provided in this application, for example... Figure 3 The information generation method is shown.

[0233] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0234] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0235] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0236] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0237] In summary, through the embodiments of this application, the required first statistical item is first determined for the database statement. Then, using the first statistical value of the first statistical item at a historical time, the statistical value of the first statistical item at the current time or a future time is predicted, i.e., the latest second statistical value. Then, a first execution path is generated based on the predicted second statistical value. In this way, by predicting the statistical value, the number of times the statistical value is collected can be reduced, thereby reducing the time and resource consumption caused by the collection of statistical values ​​during the execution path generation process, thus improving the efficiency of execution path generation and reducing the resource consumption of execution path generation.

[0238] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An information generation method, characterized in that, The method includes: For the received database statement, a first statistical item is determined, wherein the first statistical item is used to describe the data distribution of the data table corresponding to the database statement; Based on the first statistical value of the first statistical item in historical time, determine the latest second statistical value for the first statistical item; Based on the second statistical value of the first statistical item, a first execution path for the database statement is determined, wherein the first execution path represents the operation steps adopted by the database statement during execution.

2. The method according to claim 1, characterized in that, The step of determining the latest second statistical value for the first statistical item based on the first statistical value over historical time includes: Using the first statistical value from the historical time period, a statistical value sequence is generated, and the sequence type of the statistical value sequence is determined; Based on the sequence type and the sequence of statistical values, the second statistical value of the first statistical item is determined.

3. The method according to claim 2, characterized in that, Determining the second statistical value of the first statistical item based on the sequence type and the statistical value sequence includes: The candidate prediction model that matches the sequence type among multiple candidate prediction models is determined as the target prediction model; Based on the statistical value sequence, the second statistical value of the first statistical term is determined using the target prediction model.

4. The method according to claim 3, characterized in that, The target prediction model includes: multiple sub-prediction models; determining the second statistical value of the first statistical term based on the statistical value sequence using the target prediction model includes: Based on the statistical value sequence, a third statistical value is predicted using each of the sub-prediction models; The third statistical values ​​corresponding to the multiple sub-prediction models are fused together, and the fusion result is determined as the second statistical value.

5. The method according to claim 3, characterized in that, The step of determining the second statistical value of the first statistical term based on the statistical value sequence using the target prediction model includes: Based on the model order of the target prediction model, multiple prediction inputs are selected from the statistical value sequence, and multiple random errors are determined based on the model order of the target prediction model. Using the target prediction model, based on multiple prediction inputs and multiple random errors, a statistical value is predicted for the first statistical term, and the prediction result is determined as the second statistical value.

6. The method according to any one of claims 3 to 5, characterized in that, Before determining the candidate prediction model that matches the sequence type from among multiple candidate prediction models as the target prediction model, the method further includes: Based on the sample statistical value sequence of the sequence type, determine the degree of matching between each candidate prediction model and the sequence type; Based on the degree of matching, a candidate prediction model that matches the sequence type is determined from among the multiple candidate prediction models.

7. The method according to claim 6, characterized in that, The determination of the matching degree between each candidate prediction model and the sequence type based on the sample statistical value sequence of the sequence type includes: From the sequence of sample statistics values ​​of the sequence type, a sub-sample sequence is obtained, and the sample statistics values ​​located after the sub-sample sequence in the sequence of sample statistics values ​​are determined as the true values. By using each of the candidate prediction models, the sample statistics of the sub-sample sequence are predicted, and the prediction results of each candidate prediction model are obtained. The degree of matching between each candidate prediction model and the sequence type is determined by using the difference between the true value result and the prediction result of each candidate prediction model.

8. The method according to any one of claims 2 to 5, characterized in that, Determining the sequence type of the statistical value sequence includes: Determine the variation characteristics of the statistical value sequence; Based on the aforementioned change characteristics, the sequence type of the statistical value sequence is determined.

9. The method according to any one of claims 1 to 5, characterized in that, The determination of the first statistical item for the received database statement includes: The database statement is broken down into multiple sub-operation statements; For each of the sub-operation statements, determine the associated second statistical item; The first statistical item of the database statement is determined by using the second statistical items of the multiple sub-operation statements.

10. The method according to claim 9, characterized in that, The determination of the associated second statistical item for each of the sub-operation statements includes: Determine the operation type corresponding to the sub-operation statement; The statistical item required by the operation type is determined as the second statistical item.

11. The method according to any one of claims 1 to 5, characterized in that, Determining the first execution path of the database statement based on the second statistical value of the first statistical item includes: Based on the second statistical value, multiple candidate second execution paths are generated; Calculate the first execution cost for each of the second execution paths, wherein the first execution cost is used to describe the resources and time consumed by the second execution path during execution; The first execution path is determined from among the multiple second execution paths that corresponds to the second execution path with the smallest first execution cost.

12. The method according to any one of claims 1 to 5, characterized in that, After determining the first execution path of the database statement based on the second statistical value of the first statistical item, the method further includes: For the first execution path, a first expiration time and a first update time are determined, wherein the first expiration time refers to the time when the first execution path becomes invalid, and the first update time refers to the time when the execution path is regenerated for the database statement; The first execution path, the first expiration time, and the first update time are cached.

13. The method according to claim 12, characterized in that, Determining the first expiration time and the first update time for the first execution path includes: Based on the first statistical value and the second statistical value, predict the change time of the first statistical item, wherein the change time refers to the time during which the degree of change of the statistical value is greater than a degree threshold; Based on the change time, the first expiration time is determined, and the first update time is determined from the time before the first expiration time.

14. The method according to any one of claims 1 to 5, characterized in that, The determination of the first statistical item for the received database statement includes: The database statement is hashed to obtain a hash value; Based on the hash value, retrieve the third execution path currently used by the database statement from the cache; When the second update time of the third execution path is reached, a first statistical item is determined for the received database statement.

15. The method according to claim 14, characterized in that, After determining the first execution path of the database statement based on the second statistical value of the first statistical item, the method further includes: When the second expiration time of the third execution path is reached, the execution path of the database statement is switched from the third execution path to the first execution path.

16. The method according to any one of claims 1 to 5, characterized in that, After determining the first execution path of the database statement based on the second statistical value of the first statistical item, the method further includes: Calculate the second execution cost of the first execution path, and calculate the third execution cost of the third execution path; When the third execution cost is greater than the second execution cost, the execution path of the database statement is switched from the third execution path to the first execution path.

17. An information generation device, characterized in that, The device includes: The statistical item determination module is used to determine a first statistical item for a received database statement, wherein the first statistical item is used to describe the data distribution of the data table corresponding to the database statement; The statistical value determination module is used to determine the latest second statistical value for the first statistical item based on the first statistical value of the first statistical item in historical time. The path determination module is used to determine a first execution path of the database statement based on the second statistical value of the first statistical item, wherein the first execution path represents the operation steps adopted by the database statement during execution.

18. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 16.

19. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method according to any one of claims 1 to 16.

20. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method according to any one of claims 1 to 16.

Images