Anticipating pre-execution data queries

Abstract

A method (301) for anticipating pre-execution queries (100) includes calculating an execution cost of each of a plurality of different queries (260) to a data source (150) scheduled at a particular time on a particular date. The method also includes monitoring a query process schedule (150) and detecting from the monitoring a non-scheduled time (110) on a particular date prior to the particular time on the particular date. Finally, the method includes responding to the detection by first selecting the most expensive one of the different queries (260) based on execution cost and then executing the most expensive one of the selected different queries (260) during the non-scheduled time (110) on the particular date prior to the particular time on the particular date.

Description

Technical Field

[0001] This disclosure relates to the field of query scheduling, and more specifically, to the pre-scheduling of executing queries before the requested query execution time. Background Technology

[0002] A query is a request for information from an information retrieval system. There are three common methods for formulating queries: menu-driven, instance-based, and query language-based. In the first example, the query is formulated and published based on parameter selections from a menu. In the second example, the information retrieval system provides blank records, allowing end-users to specifically define the fields and values ​​of the query. In the third example, end-users formulate queries using a stylized query language. The latter is the most complex method because it requires the use of a specialized language, but it is also the most powerful because it represents the least restrictive mode for querying information retrieval systems.

[0003] Queries are typically published on demand through query interfaces or programmatically when a computer program is executed. However, queries can also be published in batch mode. That is, queries can be specified simultaneously, but their execution for an information retrieval system can be postponed to a later time. In this respect, it is common for multiple users to submit queries to the database simultaneously for execution in information retrieval systems. Therefore, if the information retrieval system lacks sufficient computing resources to execute all submitted queries concurrently, it must postpone the execution of one or more of these queries, and only a subset of the queries can be processed immediately. The process of determining which queries to postpone and when to execute the postponed queries is called query scheduling.

[0004] One way to schedule queries is to execute incoming queries in the order they arrive, known as the "first-come, first-served" (FFS) method. However, the FFS method cannot distinguish between queries with different response time requests, some of which are more time-sensitive than others. If queries are simply scheduled according to their arrival order, some time-sensitive queries may be forced to wait behind time-insensitive queries, which can negatively impact the availability and responsiveness of the information retrieval system.

[0005] Query scheduling can also be based on fixed priority. In fixed-priority scheduling, each query is assigned a priority based on one or more attributes known upon arrival, such as the identity or type of the query requester. Subsequently, each query can be scheduled according to its assigned priority. It can be seen that fixed-priority scheduling avoids the problems of the first-come, first-served approach, as time-sensitive queries can take precedence over time-insensitive queries. However, fixed-priority scheduling cannot account for "heavy" queries that require relatively long execution times and "light" queries that require relatively short execution times, such as in milliseconds or seconds. Summary of the Invention

[0006] Embodiments of this disclosure address the shortcomings of the present technology in query scheduling and provide a novel and non-obvious method, system, and computer program product for anticipating pre-execution queries. In one embodiment of this disclosure, a method for anticipating pre-execution queries includes calculating the execution cost of each of a plurality of different queries to a data source scheduled at a specific time on a specific date. The method also includes monitoring query process scheduling and detecting, from the monitoring, a non-scheduled period on a particular date preceding the specific time on the specific date. Finally, the method includes responding to the detection by first selecting the most expensive of the different queries in terms of execution cost, such as the amount of computational resources consumed in executing the corresponding one of the different queries, and then executing the selected most expensive of the different queries during the non-scheduled period on the particular date preceding the specific time on the specific date.

[0007] In one aspect of the invention, different queries can be filtered to include only queries that do not depend on the underlying data, which is expected to be updated after the unscheduled time of a particular date and before a specific time on a particular date. In another aspect of the invention, the selection of the most expensive of the different queries includes not only selecting the most expensive of the different queries but also selecting one of the different queries least dependent on the freshness of the underlying data. In yet another aspect of the invention, the process step of monitoring query process scheduling can detect duplicate instances during unscheduled times on multiple dates without scheduled queries, so as to detect the unscheduled time of a particular date before a specific time on a particular date. Even in another aspect of the invention, a manually specified one of the different queries can be selected to perform the query instead of the most expensive of the different queries during the unscheduled time of a particular date before a specific time on a particular date.

[0008] In another embodiment of this disclosure, a query scheduling data processing system is configured to anticipate and pre-execute queries. The system includes a host computing platform comprising one or more computers, each having memory and at least one processor, a data source coupled to the host computing platform, and a query processor adapted to execute queries based on data set in the data source. The system also includes an anticipate and pre-execute query module. This module includes computer program instructions executed in the memory of the host computing platform to calculate the execution cost of each of a plurality of different queries scheduled to be performed on the data source at a specific time on a specific date, monitor query process scheduling, detect from the monitoring, during a non-scheduled period on a particular date preceding the specific time on the specific date, select the most expensive of the different queries based on execution cost in response to the detection, and execute the selected most expensive of the different queries during the non-scheduled period on the particular date preceding the specific time on the specific date.

[0009] Other aspects of this disclosure will be set forth in part in the description which follows, and will be apparent in part from the description, or may be learned by practice of this disclosure. Various aspects of this disclosure will be realized and achieved by the elements and combinations particularly pointed out in the appended claims. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the scope of the claimed disclosure. Attached Figure Description

[0010] Details of one or more embodiments of this disclosure are set forth in the accompanying drawings and the following description. Other aspects, features, and advantages will become apparent from the description, the drawings, and the claims.

[0011] Figure 1 This is a diagram illustrating the expected pre-execution query process.

[0012] Figure 2 This is a diagram of a data processing system configured to perform queries as expected.

[0013] Figure 3 This is a flowchart of an exemplary arrangement for operations that are intended to be performed in advance of a query method. Detailed Implementation

[0014] Embodiments of this disclosure provide a method for pre-executing queries. According to one embodiment of this disclosure, the execution cost of each of several different queries to a data source scheduled for a specific time on a specific date can be calculated. The query process scheduling can then be monitored to detect unscheduled time on a particular date preceding the specific time on the specific date. Subsequently, in response to the detection of unscheduled time, the most expensive of the different queries can be selected first based on execution cost, and then executed during the unscheduled time on the particular date preceding the specific time on the specific date. In this way, the most expensive of the different queries can be pre-executed within the expected time, which would otherwise be wasted, thereby improving the efficiency of the total execution cost of all queries planned to be processed within a specific time on a specific date.

[0015] In further explanation, Figure 1 The procedure 101 for the anticipated pre-execution of a query is shown. For example... Figure 1As shown, query processor 170 executes query 100 against data source 140 (e.g., a database or data model of data in one or more databases). Query processor 170 executes one or more queries 100 at different scheduling times 110 according to query process scheduling 150, in which different queries are scheduled to be executed against data source 140 at different scheduling times 110. Importantly, when a corresponding one of the queries 100 is scheduled in query process scheduling 150 for processing by query processor 170, each query 100 is associated with metadata defined at the scheduling time 110 on a specific date. Similarly, each query 100 is associated with metadata that specifies a processing cost 120 in the form of resource costs (e.g., processor and memory resources) and optionally a sensitivity 130 to data freshness values. The cost of process 120 can be manually specified directly in the metadata, or the cost of process 120 can be calculated as a mathematical cost function taking into account several input parameters, such as runtime, freshness sensitivity / resilience, and the likelihood that the query will be required to be re-run due to updates to observed relevant records. In fact, in one aspect of the invention, these parameters can be weighted according to different weights determined by the administrator.

[0016] A sensitivity of 130 to data freshness indicates the dependency of the corresponding query in query 100 on the underlying data, which must be as up-to-date as possible to provide accurate query results. For example, a dependency on addressing data may be less sensitive than a dependency on daily sales information. In the former case, if the corresponding query in 100 is pre-executed before the scheduled time 110, it may not have any impact because the addressing information is unlikely to change during the time interval between the pre-execution time and date and the scheduled time 110. However, in the latter case, if the corresponding query in 100 depends on data that may change during the time interval between the corresponding scheduled time 110 and the execution time prior to the scheduled time 110, such as sales data, then pre-executing the corresponding query in 100 may produce inaccurate results.

[0017] Monitor 160 monitors query process scheduling 150 to identify different time periods and whether any time slots are not scheduled for query processing. Upon detecting an available time 180, monitor 160 examines the queries 100 pending processing in the future time starting from available time 180 to identify those queries 100 with the highest corresponding processing cost 120. Monitor 160 then selects one of the identified queries 100 with the highest corresponding processing cost 120 and reschedules one of the selected queries 100 for processing by query processor 170 against data source 140 during the detected available time 180. Among multiple different queries 100 with a range of similar corresponding costs 120 determined to be the highest among all queries 100, one of the queries with similar corresponding costs 120 is selected for pre-execution based on the minimum corresponding data freshness sensitivity 130.

[0018] and Figure 1 The relevant description process can be implemented in the data processing system 200. In further detail, Figure 2 A data processing system 200 configured to perform queries is illustrated. The system 200 includes a host computing system 210, which comprises one or more computers, each having memory and at least one processor. The host computing system 210 is communicatively coupled to a database 230 via a data communication network 220 and supports the operation of a query processor 240 adapted to execute different queries 260 against the database 230 based on a query schedule 250, which specifies the date and time on which the query processor 240 executes the corresponding query 260 against the database 230.

[0019] Importantly, the anticipated pre-scheduling module 300 is executed in the memory of the host computing system 210. The anticipated pre-scheduling module 300 includes computer program instructions that, during execution, allow one of the queries 260 to be manually or automatically rescheduled to an earlier time slot known to be available based on monitoring of the query schedule 250. Regarding automatic rescheduling, the program instructions monitor the query schedule 250 to identify unscheduled time slots where no query processor 240 is scheduled to process a query. To this end, the program instructions detect repeating instances of unscheduled time slots existing across multiple dates on which no queries are scheduled, in order to detect unscheduled time slots on a specific date preceding a specific time on a specific date.

[0020] Upon detecting an unscheduled period, the program instructions are further able to select one of the queries 260 scheduled for execution at a later scheduled time after the unscheduled period, the query having the highest corresponding execution cost among all queries 260, or in another case exceeding a threshold. Optionally, the program code selects a query 260 from a plurality of queries 260 with corresponding costs exceeding the threshold, the query 260 having the lowest record sensitivity to changes in the underlying data upon which the query depends. The program instructions then pre-schedule one of the selected queries 260 for execution within the available time.

[0021] To further illustrate the operation of the expected pre-scheduling module 300 Figure 3 This is an exemplary flowchart illustrating the operations of method 301 for anticipating the execution of a query. Starting at block 310, the module connects to the query process scheduler. In block 320, the module reads the query schedule to be processed in the next time period. In decision block 330, the module determines whether there is available time within that time period. If not, in block 340, the module selects the next time period; in block 320, the module reads the schedule for the next time period. The process then returns to decision block 330.

[0022] In decision block 330, if available time exists within the time period, in block 350, the module selects an available time slot within that time period, and in block 360, the module retrieves queries scheduled for future time periods beyond the selected time period. In block 370, the retrieved queries are filtered to include only those that do not depend on underlying data expected to be updated after the available time or before a specific time. In block 380, the filtered queries are sorted by execution cost. Then, in block 390, starting with the highest-cost queries, the module selects queries for pre-execution based on the sorting least sensitive to the freshness of the underlying data involved in the queries. Finally, in block 400, the module reschedules the pre-execution of the selected queries within the available time.

[0023] This disclosure may be embodied in systems, methods, computer program products, or any combination thereof. A computer program product may include a computer-readable storage medium or a medium having computer-readable program instructions thereon for causing a processor to perform various aspects of this disclosure. A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0024] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to their respective computing / processing devices, or downloaded over a network to an external computer or external storage device. The computer-readable program instructions can execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. Hereinafter, aspects of this disclosure are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products, according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0025] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute through the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / behaviors specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other apparatus to operate in a particular manner, such that a computer-readable storage medium having instructions stored therein includes an article of manufacture comprising instructions for implementing aspects of specific functions / behaviors in the flowchart and / or block diagram blocks.

[0026] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device, thereby producing a computer-implemented process, such that the instructions that execute on the computer, other programmable apparatus or other device implement specific functions / behaviors in flowcharts and / or block diagram blocks.

[0027] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible embodiments of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specific logical function. In some alternative embodiments, the functions indicated in a block may not appear in the order shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or, depending on the functions involved, these blocks may sometimes be executed in reverse order. It will also be noted that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a system based on special-purpose hardware that performs a specific function or behavior, or by a combination of special-purpose hardware and computer instructions.

[0028] Finally, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprising" and / or "including," when used in this specification, specifically indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] All means or steps plus functional elements in the following claims are intended to include any structure, material, behavior, and equivalent for performing the function in combination with elements of other specific claims. The description of this disclosure is for illustrative and descriptive purposes only and is not intended to be exhaustive or limited to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. The embodiments were chosen and described in order to best explain the principles and practical application of this disclosure and to enable those skilled in the art to understand the various embodiments of this disclosure and to make various modifications suitable for the particular purpose contemplated.

[0030] Having described the disclosure of this application in such detail and with reference to its embodiments, it is apparent that modifications and variations may be made without departing from the scope of the disclosure as defined in the appended claims, as follows.

Claims

1. A method (301) for anticipating a pre-execution query (100), characterized in that, The method (301) includes: Calculate the execution cost of each of multiple different queries (260) to a data source (140) scheduled at a specific time on a specific date; Monitor and query process scheduling (150); Detect the unscheduled time (110) of a special date preceding the specific time on the specific date from the monitoring; and In response to the detection, the most expensive of the different queries (260) is selected based on execution cost, and the most expensive of the selected different queries (260) is executed during the non-scheduled time (110) of the special date preceding the specific time of the specific date. This further includes filtering the different queries (260) to include only queries (100) that do not depend on the underlying data, which is expected to be updated after the non-scheduled time (110) of the particular date and before the specific time of the particular date. Among them, selecting the most expensive of the different queries (260) includes not only selecting the most expensive of the different queries (260), but also selecting the one of the different queries (260) that is least dependent on the freshness of the underlying data.

2. The method (301) according to claim 1, characterized in that, The monitoring of the query process The scheduling (150) includes detecting duplicate instances of the unscheduled time (110) in a plurality of dates for which no query (100) is scheduled, so as to detect the unscheduled time (110) on a particular date prior to the particular time of the particular date.

3. The method (301) according to claim 1, characterized in that, The execution cost includes the amount of computing resources consumed in executing one of the different queries (260).

4. The method (301) according to any one of claims 1-3, characterized in that, Further, it includes selecting a manually specified one of the different queries (260) instead of the most expensive one of the different queries (260) executed during the non-scheduled time (110) of the special date prior to the specific time of the specific date.

5. A query scheduling data processing system (200) configured to perform anticipated queries (100), characterized in that, The system (200) includes: A host computing platform, comprising one or more computers, each computer having memory and at least one processor; The data source (140) is coupled to the host computing platform; A query processor (170, 240) adapted to execute a query (100) based on data set in the data source (140); and The expected query pre-execution module includes computer program instructions that execute in the memory of the host computing platform, the instructions executing: Calculate the execution cost of each of multiple different queries (260) to a data source (140) scheduled at a specific time on a specific date; Monitor and query process scheduling (150); Detect the unscheduled time (110) of a special date preceding the specific time on the specific date from the monitoring; and The detection is responded to by selecting the most expensive of the different queries (260) based on execution cost, and executing the most expensive of the selected different queries (260) during the non-scheduled time (110) of the special date preceding the specific time of the specific date. This further includes filtering the different queries (260) to include only queries (100) that do not depend on the underlying data, which is expected to be updated after the non-scheduled time (110) of the particular date and before the specific time of the particular date. Among them, selecting the most expensive of the different queries (260) includes not only selecting the most expensive of the different queries (260), but also selecting the one of the different queries (260) that is least dependent on the freshness of the underlying data.

6. The system (200) according to claim 5, characterized in that, Monitoring the query process scheduling (150) includes detecting duplicate instances of the non-scheduled time (110) in multiple dates on which no query (100) is scheduled, so as to detect the non-scheduled time (110) on a particular date prior to the specific time on the specific date.

7. The system (200) according to claim 6, characterized in that, The execution cost includes the amount of computing resources consumed in executing one of the different queries (260).

8. The system (200) according to claim 7, characterized in that, The program instructions further execute the selection of a manually specified one of the different queries (260) instead of the most expensive one of the different queries (260) executed during the non-scheduled time (110) of the special date prior to the specific time of the specific date.

9. A computer program product for anticipating a pre-execution query (100), the computer program product comprising a computer-readable storage medium having program instructions embedded therein, the program instructions being executable by a device to cause the device to perform a method (301), characterized in that, include: Calculate the execution cost of each of multiple different queries (260) to a data source (140) scheduled at a specific time on a specific date; Monitor and query process scheduling (150); Detect the non-scheduled time (110) of a special date prior to the specific time of the specific date from the monitoring. as well as The detection is responded to by selecting the most expensive of the different queries (260) based on execution cost, and executing the most expensive of the selected different queries (260) during the non-scheduled time (110) of the special date preceding the specific time of the specific date. This further includes filtering the different queries (260) to include only queries (100) that do not depend on the underlying data, which is expected to be updated after the non-scheduled time (110) of the particular date and before the specific time of the particular date. Among them, selecting the most expensive of the different queries (260) includes not only selecting the most expensive of the different queries (260), but also selecting the one of the different queries (260) that is least dependent on the freshness of the underlying data.

10. The computer program product according to claim 9, characterized in that, Monitoring the query process scheduling (150) includes detecting repeated instances of the non-scheduled time (110) on multiple dates when no query (100) is scheduled, so as to detect the non-scheduled time (110) on a particular date prior to the specific time on the specific date.

11. The computer program product according to claim 10, characterized in that, The execution cost includes the amount of computing resources consumed in executing one of the different queries (260).

12. The computer program product according to claim 11, characterized in that, The method (301) further includes selecting a manually specified one of the different queries (260) to replace the most expensive one of the different queries (260) executed during the non-scheduled time (110) of the special date prior to the specific time of the specific date.

Images