Dual Engine Technique Based-On Open-Source RDBMS

By classifying queries and adapting engine usage in DBMS systems, the method addresses inefficiencies in query processing, enhancing speed and resource utilization.

KR103046250B1Active Publication Date: 2026-11-10DATAGRIT CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
KR1020240111833
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2026-11-10
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

Existing DBMS systems face challenges in efficiently processing and executing queries, particularly in terms of speed and resource utilization, especially in RDBMS environments.

Method used

A method that classifies queries as transactional or analytical based on predetermined criteria and adapts the query processing engine accordingly, using either an OLTP or OLAP engine, involving pipeline construction, logical and physical optimizations, and task scheduling across worker threads.

Benefits of technology

Enhances the speed and efficiency of query processing in DBMS systems by optimizing resource utilization and engine selection based on query type, thereby improving performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112024091060770-PAT00002_ABST
    Figure 112024091060770-PAT00002_ABST
Patent Text Reader

Abstract

According to one embodiment of the present disclosure, a method for processing a query performed by a computing device capable of operating based on a Database Management System (DBMS) is disclosed. The method may include: receiving a query requesting an execution result in the DBMS; determining the type of the query as a transactional query or an analytic query based on a predetermined criterion for classifying the query; and generating an execution result corresponding to the query by adaptively determining an engine for processing the query among a plurality of different engines or by processing the query in a different way according to the determined type of the query.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present disclosure relates to a Database Management System (DBMS), and more specifically, to a technique for processing a query executed by a computing device capable of operating based on a DBMS. Background Technology

[0002] A DBMS stands for Database Management System, representing a software system for managing and manipulating databases. A DBMS can provide functions for efficiently storing, modifying, searching, and managing data. Upon receiving a query written in SQL statements, the DBMS interprets, optimizes, and executes the query, thereby returning a result corresponding to the query. A DBMS may include an RDBMS, which structures and manages data.

[0003] RDBMS stands for Relational Database Management System, which refers to a software system designed to manage and manipulate databases by structuring data. RDBMS can store and manage data by structuring it according to a relational model.

[0004] RDBMS technology is evolving to meet the demands for data processing speed, the possibility of real-time analysis, and large-scale data processing. Prior art literature

[0005] Republic of Korea Registered Patent No. 10-1966815 (Registered on April 2, 2019) The problem to be solved

[0006] A technique according to one embodiment of the present disclosure is intended to increase the speed and efficiency of processing and / or executing queries in a DBMS system. A technique according to one embodiment of the present disclosure is intended to efficiently use computing resources for processing and / or executing queries in a DBMS system.

[0007] A technique according to one embodiment of the present disclosure is intended to increase the speed and efficiency of processing and / or executing queries in an RDBMS system. A technique according to one embodiment of the present disclosure is intended to efficiently use computing resources for processing and / or executing queries in an RDBMS system. means of solving the problem

[0008] According to one embodiment of the present disclosure, a method for processing a query performed by a computing device capable of operating based on a Database Management System (DBMS) is disclosed. The method may include: receiving a query requesting an execution result in the DBMS; determining the type of the query as a transactional query or an analytic query based on a predetermined criterion for classifying the query; and generating an execution result corresponding to the query by adaptively determining an engine for processing the query among a plurality of different engines or by processing the query in a different manner according to the determined type of the query.

[0009] In one embodiment, the predetermined criterion includes a criterion related to the number of tuples to be processed to generate a result of the query, the transaction query is a query in which the number of tuples in the query is less than a predetermined first threshold, and the analysis query may be a query in which the number of tuples in the query is greater than or equal to the predetermined first threshold.

[0010] In one embodiment, the predetermined criteria include criteria related to the type of the operator of the query, and the transaction query includes at least one of a query including a Database Manipulation Language (DML), a query including a Database Definition Language (DDL), a query for searching specific data, or a query where an index exists when searching for data, and the analysis query may be a query including at least one of an order by, a group by, a join, an aggregation function, or a window function.

[0011] In one embodiment, the predetermined criteria include criteria related to the processing method for the transaction of the query, the transaction query is a query used in OLTP (Online Transaction Processing), and the analysis query may be a query used in OLAP (Online Analytical Processing).

[0012] In one embodiment, the step of generating the execution result corresponding to the query may include: a step of generating a first execution result corresponding to the query by processing the query using a first engine for processing the analysis query when the type of the query is determined to be the analysis query; and a step of generating a second execution result corresponding to the query by processing the query using a second engine for processing the transaction query when the type of the query is determined to be the transaction query.

[0013] In one embodiment, the first engine includes an OLAP (Online Analytical Processing) based query execution engine, and the second engine may include an OLTP (Online Transaction Processing) based query execution engine.

[0014] In one embodiment, the step of generating the first execution result corresponding to the query comprises: parsing the received query; performing a logical optimization on the parsed query to determine an order for processing requests included in the parsed query; performing a physical optimization on the logically optimized query to determine an operation method required to process the requests according to the determined order; building a plurality of pipelines for processing the query and dependencies between the plurality of pipelines based on the physically optimized query, wherein a pipeline is a set of one or more operators and represents a processing unit of the query; building at least one schedule for processing the query based on the dependencies between the plurality of pipelines; building a plurality of tasks based on the at least one schedule; and adding the plurality of tasks to a task queue. The method may include: a step of processing the plurality of tasks by assigning the plurality of tasks added to the task queue to each of the plurality of worker threads included in the worker thread pool; and a step of generating the first execution result including information about the result of processing the plurality of tasks.

[0015] In one embodiment, the step of establishing the plurality of pipelines and the dependencies between the plurality of pipelines includes the step of establishing the plurality of pipelines based on an operator corresponding to a pipeline breaker; wherein the pipeline breaker is an operator that is the start or end of each pipeline and may include at least one of a sorting operator, a grouping operator, a join operator, an aggregate function, or a window function.

[0016] In one embodiment, the step of establishing dependencies between the plurality of pipelines and the plurality of pipelines includes the step of determining the type of the plurality of operators included in each of the plurality of pipelines as one of the origin role type, the on-the-fly role type, and the destination role type, wherein the origin role type acts as the start operator of the pipeline and performs the creation, loading, or pre-processing of a chunk, the on-the-fly role type receives a chunk and processes it in memory, and the destination role type processes the chunk to create an output chunk.

[0017] In one embodiment, the step of processing the plurality of tasks may include: a step of assigning the first-1 task to the first worker thread to process the first-1 task among the plurality of first tasks corresponding to the first schedule in the first worker thread; and a step of processing the first-1 task using the first worker thread.

[0018] In one embodiment, the step of processing the first-1 task using the first worker thread may include: processing the first-1 task by executing at least one operator included in a first pipeline corresponding to the first-1 task using the first worker thread; determining whether a second schedule different from the first schedule exists when the type of the processed first-1 task is a shared task type shared with other worker threads and the number of completed shared tasks corresponds to a predetermined total number of shared tasks; when it is determined that the second schedule exists, constructing a plurality of second tasks corresponding to the second schedule using the first worker thread; and adding the plurality of second tasks to the task queue.

[0019] In one embodiment, the step of processing the 1-1 task by executing at least one operator included in a first pipeline corresponding to the 1-1 task using the first worker thread comprises: a step of assigning the 1-1 operator, which is the origin operator among at least one first operator included in the first pipeline, as the current operator currently being processed, if the type of the 1-1 task is an isolated task not shared with other worker threads; a step of checking the type of the 1-1 operator if the role type of the 1-1 operator is the origin role type; a step of attempting to obtain a tuple from a data file of the DBMS if the type of the 1-1 operator is a scan operator; and, if the acquisition of the tuple is successful, a step of converting the acquired tuple into a vector by column and selecting a vector necessary for processing the query to create a first chunk. If the acquisition of the above tuple fails, the method may include the step of increasing the number of completed independent tasks by a predetermined number; and the step of scheduling a shared task if the number of completed independent tasks corresponds to a predetermined total number of independent tasks.

[0020] In one embodiment, the method may further include: a step of, after checking the type of the first-1 operator, attempting to acquire a second chunk if the type of the first-1 operator is not a scan operator; a step of executing the first-1 operator if the acquisition of the second chunk is successful; and a step of increasing the number of completed independent tasks by a predetermined number if the acquisition of the second chunk is unsuccessful.

[0021] In one embodiment, the step of processing the first-1 task by executing at least one operator included in the first pipeline corresponding to the first-1 task using the first worker thread may include: if the type of the first-1 task is a shared task, assigning the first-2 operator, which is a destination operator among at least one first operator included in the first pipeline, as the current operator currently being processed; attempting to acquire a third chunk generated from an independent task; if the acquisition of the third chunk is successful, executing the first-2 operator; and if the acquisition of the third chunk fails, increasing the number of completed shared tasks by a predetermined number.

[0022] In one embodiment, a computer program stored on a computer-readable storage medium is disclosed. The computer program enables a processor of a computing device capable of operating based on a Database Management System (DBMS) to perform a method for processing a query, wherein the method may include: receiving a query requesting an execution result in the DBMS; determining, based on a predetermined criterion, that the type of the query is a transactional query or an analytic query; and generating an execution result corresponding to the query by adaptively determining, among a plurality of different engines, an engine for processing the query or by processing the query in a different manner according to the determined type of the query.

[0023] In one embodiment, a computing device capable of operating based on a Database Management System (DBMS) is disclosed. The computing device comprises: a processor; memory; and a network unit; and the processor may perform: an operation of receiving a query requesting an execution result in the DBMS; an operation of determining the type of the query as a transactional query or an analytic query based on a predetermined criterion; and an operation of generating an execution result corresponding to the query by adaptively determining an engine among a plurality of different engines to process the query or by processing the query in a different way according to the determined type of the query. Effects of the invention

[0024] A technique according to one embodiment of the present disclosure can increase the speed and efficiency of processing and / or executing queries in a DBMS system. A technique according to one embodiment of the present disclosure can efficiently use computing resources to process and / or execute queries in a DBMS system.

[0025] A technique according to one embodiment of the present disclosure can increase the speed and efficiency of processing and / or executing queries in an RDBMS system. A technique according to one embodiment of the present disclosure can efficiently use computing resources to process and / or execute queries in an RDBMS system. Brief explanation of the drawing

[0026] FIG. 1 illustrates an exemplary block diagram of a computing device according to one embodiment of the present disclosure. FIG. 2 illustrates an exemplary flowchart for generating the execution result of a query according to one embodiment of the present disclosure. FIG. 3 illustrates an exemplary flowchart of processing a task using an engine according to one embodiment of the present disclosure. FIG. 4 illustrates an exemplary block diagram of a DBMS according to one embodiment of the present disclosure. FIG. 5 illustrates an exemplary pipeline corresponding to an input query according to one embodiment of the present disclosure. FIG. 6 illustrates an exemplary scan according to one embodiment of the present disclosure. FIG. 7 illustrates an exemplary process for processing a plurality of tasks according to one embodiment of the present disclosure. FIG. 8 illustrates an exemplary process for processing a plurality of tasks according to another embodiment of the present disclosure. FIG. 9 exemplarily illustrates the process of executing an operator included in a pipeline according to one embodiment of the present disclosure. FIG. 10 exemplarily illustrates the process of executing an operator included in a pipeline according to another embodiment of the present disclosure. FIG. 11 illustrates a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented. Specific details for implementing the invention

[0027] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions.

[0028] As used herein, terms such as “component,” “module,” “system,” etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and / or a computer. For example, both an application executed on a computing device and the computing device itself may be a component. One or more components may reside within a processor and / or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from a component interacting with another component in a local system or distributed system, and / or data transmitted through signals to other systems and networks such as the Internet).

[0029] Furthermore, the term "or" is intended to mean an implicit "or" rather than an exclusive "or." That is, unless otherwise specified or evident from the context, "X uses A or B" is intended to mean one of the natural implicit substitutions. In other words, if X uses A; if X uses B; or if X uses both A and B, "X uses A or B" may apply to any of these cases. Additionally, the terms "or" and "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the enumerated related items.

[0030] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that such features and / or components are present. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Furthermore, unless otherwise specified or clearly evident from the context to indicate a singular form, the singular in this specification and claims should generally be interpreted to mean “one or more.”

[0031] And, the term "at least one of A or B" should be interpreted to mean "a case including only A," "a case including only B," or "a combination of A and B."

[0032] Those skilled in the art should recognize that the various exemplary logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or a combination of both.

[0033] The description of the presented embodiments is provided to enable those skilled in the art to use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein. The present disclosure should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

[0034] Terms expressed as N, such as the first, second, or third in the present disclosure, are used to distinguish at least one entity. For example, the entities expressed as the first and second may be identical or different from each other.

[0035] Terms expressed as NMs, such as 1-1 or 1-2 in the present disclosure, are used to distinguish M entities included in the upper group N. For example, the entities expressed as 1-1 and 1-2 may be identical or different from each other.

[0036] The term "~ etc." as in "A, B, etc." in the present disclosure should be interpreted to mean "cases including only A," "cases including only B," or "cases combined with the composition of A and B."

[0037] FIG. 1 illustrates an exemplary block diagram of a computing device according to one embodiment of the present disclosure.

[0038] The configuration of the computing device (100) illustrated in FIG. 1 is merely a simplified example. In one embodiment of the present disclosure, the computing device (100) may include other configurations for performing the computing environment of the computing device (100), and only some of the disclosed configurations may constitute the computing device (100). For example, if the aforementioned computing device (100) includes a user terminal, an output unit (not shown) and an input unit (not shown) may be included within the scope of the computing device (100).

[0039] In one embodiment, the computing device (100) may represent an entity capable of operating based on a Database Management System (DBMS), and may represent a database server or a client. In another embodiment, the computing device (100) may represent an entity capable of operating based on a Relational Database Management System (RDBMS).

[0040] In the present disclosure, the computing device (100) may be used to encompass any type of server and any type of terminal.

[0041] The computing device (100) of the present disclosure may be intended to include a client and / or a database server. The computing device (100) may refer to node(s) in a database system having a mechanism for communicating over a network. For example, the computing device (100) may include a PC, a laptop computer, a workstation, a terminal, and / or any electronic device having network connectivity. Additionally, the computing device (100) may include any server implemented by at least one of an agent, an API (Application Programming Interface), and a plug-in.

[0042] For example, the computing device (100) in FIG. 1 may be associated with a user who uses a database server (e.g., a DBA (Database Administration)). In this example, the computing device (100) may issue query statements to the database server. For example, the computing device (100) in FIG. 1 may represent a database server.

[0043] The computing device (100) in the present disclosure may mean any type of component constituting a system for implementing embodiments of the present disclosure.

[0044] In one embodiment, the computing device (100) may include a processor (110) for executing a DBMS and / or an RDBMS, a memory (130) for storing and managing data related to the DBMS and / or an RDBMS, and a network unit (150) for performing communication related to the DBMS and / or an RDBMS. In another embodiment, the computing device (100) may include a permanent storage medium for storing and managing data related to the DBMS and / or an RDBMS. Depending on the implementation mode, the term memory (130) may be used to encompass the permanent storage medium.

[0045] In one embodiment, the processor (110) can perform the overall operation of the computing device (100). For example, the processor (110) can provide appropriate information or functions to the user by processing data, information, or signals, etc., that are input or output through components included in the computing device (100), or by running an application stored in a storage unit.

[0046] The processor (110) may be composed of at least one core. The processor (110) may include a device for data analysis and / or processing, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of the computing device (100).

[0047] The processor (110) can read a computer program stored in memory (130) and, according to one embodiment of the present disclosure, generate an execution plan corresponding to a query and generate an execution result corresponding to the query.

[0048] According to one embodiment of the present disclosure, memory (130) may store any form of information generated or determined by the processor (110) and any form of information received by the computing device (100). According to one embodiment of the present disclosure, memory (130) may be a storage medium that stores computer software that enables the processor (110) to perform operations according to embodiments of the present disclosure. Accordingly, memory (130) may mean computer read media for storing software code necessary to perform embodiments of the present disclosure, data to be executed by the code, and results of code execution.

[0049] According to one embodiment of the present disclosure, memory (130) may mean any type of storage medium. For example, memory (130) may include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk. The computing device (100) may operate in connection with web storage that performs the storage function of the memory (130) on the internet. The description of memory described above is merely an example, and the memory (130) used in the present disclosure is not limited to the examples described above.

[0050] The network unit (150) in the present disclosure may be configured regardless of the mode of communication, such as wired or wireless, and may be configured as various communication networks, such as a Personal Area Network (PAN) or a Wide Area Network (WAN). Additionally, the network unit (150) may operate based on the known World Wide Web (WWW) and may utilize wireless transmission technologies used for short-range communication, such as Infrared Data Association (IrDA) or Bluetooth. For example, the network unit may include a database link (dblink), and accordingly, multiple computing devices may communicate with each other through this database link to retrieve data from other computing devices. The technologies described in this specification may be used not only in the networks mentioned above but also in other networks.

[0051] The computing device (100) in the present disclosure may include any type of user terminal and / or any type of server. Accordingly, embodiments of the present disclosure may be performed by a server and / or a user terminal.

[0052] In one embodiment, the user terminal may include any type of terminal capable of interacting with a server or other computing device. The user terminal may include, for example, a mobile phone, a smartphone, a laptop computer, a PDA (personal digital assistant), a slate PC, a tablet PC, and an ultrabook. In one embodiment, the user terminal may refer to a device including a camera installed in a vehicle.

[0053] In one embodiment, the server may include any type of computing system or computing device, such as, for example, a microprocessor, a mainframe computer, a digital processor, a portable device, and a device controller.

[0054] In one embodiment, the server may include a storage unit (not shown) for storing data and / or information used in the present disclosure. Such a storage unit may be included within the server or exist under the management of the server. As another example, the storage unit may be implemented in a form that exists outside the server and is capable of communicating with the server. In this case, the storage unit may be managed and controlled by an external server different from the server. As another example, the storage unit may be implemented in a form that exists outside the server and is capable of communicating with the server. In this case, the storage unit may be managed and controlled by an external server different from the server.

[0055] In the present disclosure, the DBMS or RDBMS may be operated by a processor (110) on memory (130).

[0056] FIG. 2 illustrates an exemplary flowchart for generating the execution result of a query according to one embodiment of the present disclosure. The steps illustrated in FIG. 2 are exemplary steps. Accordingly, it will be apparent to those skilled in the art that some of the steps in FIG. 2 may be omitted or additional steps may exist without departing from the spirit of the present disclosure. Furthermore, specific details regarding the configurations described in FIG. 2 (e.g., computing device (100), etc.) may be replaced by the details previously described through FIG. 1. In one embodiment, the examples illustrated in FIG. 2 may be performed by a computing device (100) (e.g., a processor (110) of the computing device (100)).

[0057] In one embodiment, the computing device (100) can process queries using a dual engine technique. For example, the computing device (100) can process queries using a dual engine technique based on an open-source RDBMS. An RDBMS stands for Relational Database Management System and may refer to a software system for managing and manipulating databases by structuring data. An RDBMS can store and manage data by structuring it according to a relational model.

[0058] In one embodiment, the dual-engine technique may be a technique for determining an engine to process a query based on the type of query. In one embodiment, the dual-engine technique may utilize an OLAP (Online Analytical Processing) based query execution engine and an OLTP (Online Transaction Processing) based query execution engine. For example, if the type of the query is an analytical query, the dual-engine technique may process the query using an OLAP-based query execution engine. As another example, if the type of the query is a transaction query, the dual-engine technique may process the query using an OLTP-based query execution engine.

[0059] Below, a specific method for processing queries using the dual-engine technique will be described.

[0060] In one embodiment, the computing device (100) can receive a query requesting an execution result in the DBMS (210).

[0061] For example, a query may refer to a request statement written in SQL (Structured Query Language) statements. For example, a query may be written by a user. Here, "user" may be used interchangeably with "client" or "DBA". Also, here, "query" and "SQL statement" may be used interchangeably. For example, an SQL statement to generate a query may include Data Definition Language (DDL) and Data Manipulation Language (DML).

[0062] In one embodiment, the computing device (100) may perform any database operation corresponding to the query in response to the reception of the query. For example, the computing device (100) may perform a database operation corresponding to the entire query and / or subquery. Additionally, the computing device (100) may perform any database operation that occurs in generating output information for the query. Furthermore, the computing device (100) may perform any database operation that is performed according to a determined storage method.

[0063] In one embodiment, the computing device (100) can determine the type of the query as a transactional query or an analytic query based on predetermined criteria for classifying the query (220).

[0064] In one embodiment, the predetermined criteria may include criteria related to the number of tuples in the query. Accordingly, the computing device (100) may determine the type of the query as a transaction query or an analysis query based on the number of tuples in the query. For example, a transaction query may be a query in which the number of tuples in the query is less than a predetermined first threshold (e.g., 5, etc.). An analysis query may be a query in which the number of tuples in the query is greater than or equal to a predetermined first threshold.

[0065] In one embodiment, the predetermined criteria may include criteria related to the type of the operator of the query. Accordingly, the computing device (100) may determine the type of the query as a transactional query or an analytical query based on the type of the operator of the query. For example, a transactional query may include at least one of a query including Database Manipulation Language (DML), a query including Database Definition Language (DDL), a query for searching specific data, or a query where an index exists when searching for data. An analytical query may include at least one of an order by operator, a group by operator, a join operator, an aggregation function, and / or a window function.

[0066] In one embodiment, a join operator may be used to combine data between tables in a database. For example, a join operator may be used to combine specific rows and / or columns between tables in a database. In one embodiment, an analytical query may include a join operator when no index associated with the query exists.

[0067] In one embodiment, a sort operator may be used to sort a result set in a database. For example, the sort operator may be used to adjust the order of retrieved data or to sort retrieved results according to desired criteria.

[0068] In one embodiment, a grouping operator can be used to group results in a database. For example, the grouping operator can group data based on the values ​​of a specific column.

[0069] In one embodiment, aggregate functions may be used to perform specific operations in a database. For example, aggregate functions may include a COUNT function that calculates the number of rows in a specific column, a SUM function that calculates the sum of a specific column, an AVG function that calculates the average value of a specific column, and the like. In one embodiment, an analytical query may include an aggregate function whose selectivity is greater than or equal to a predetermined second threshold (e.g., 80%). In one embodiment, selectivity may refer to the proportion of data satisfying a specific condition. For example, in a WHERE clause or JOIN condition, selectivity may refer to the proportion of rows satisfying the condition.

[0070] In one embodiment, a window function may be used to partition or sort data in a database, perform specific calculations on a window that includes a range before or after the current row, and add the calculated values ​​to each row. For example, the window function may include a RANK function that ranks rows in the window, a MAX function that calculates the maximum value of a specific column in the window, a FIRST_VALUE function that calculates the value of the first row when sorted by a specific condition in the window, and a CUME_DIST function that calculates the cumulative distribution of the current row in the window.

[0071] In one embodiment, the predetermined criteria may include criteria related to the processing method for the transaction of the query. A transaction of a query may refer to one or more operations that must be executed at once in the DBMS. Accordingly, the computing device (100) may determine the type of the query as a transaction query or an analytical query depending on the processing method for the transaction of the query. For example, a transaction query may be a query used in OLTP (Online Transaction Processing). An analytical query may be a query used in OLAP (Online Analytical Processing).

[0072] In one embodiment, the computing device (100) may generate an execution result corresponding to the query by adaptively determining an engine to process the query among a plurality of different engines or by processing the query in a different way, depending on the type of the query determined above (230).

[0073] In the present disclosure, the engine may refer to software responsible for storing and managing data in a database. The engine implements the internal operating mechanism of the database and can perform functions such as storing, searching, updating, and deleting data.

[0074] In one embodiment, when the type of the query is determined to be an analysis query, a first execution result corresponding to the query can be generated by processing the query using a first engine for processing analysis queries.

[0075] In one embodiment, the first engine may include an OLAP (Online Analytical Processing) based query execution engine. In one embodiment, the OLAP may be a management system that enables processing a large amount of data from a database. The OLAP may use historical and aggregated data from multiple sources. Accordingly, the OLAP may be specialized in processing analytical queries with large data sizes rather than transaction queries with relatively small data sizes.

[0076] In one embodiment, when the type of the query is determined to be a transaction query, a second execution result corresponding to the query can be generated by processing the query using a second engine for processing transaction queries.

[0077] In one embodiment, the second engine may include an OLTP (Online Transaction Processing) based query execution engine. In one embodiment, the OLTP may be a management system that enables the processing of transactions in a database. The OLTP may use real-time and transactional data from a single source. Accordingly, the OLTP may be specialized in processing transactional queries with relatively small data sizes rather than analytical queries with large data sizes.

[0078] FIG. 3 illustrates an exemplary flowchart for processing a task using an engine according to an embodiment of the present disclosure. In one embodiment, FIG. 3 may be a flowchart illustrating a process of generating a first execution result corresponding to a query by processing a query determined as an analysis query using a first engine for processing an analysis query. The steps illustrated in FIG. 3 are exemplary steps. Accordingly, it will be apparent to those skilled in the art that some of the steps in FIG. 3 may be omitted or additional steps may exist without departing from the spirit of the present disclosure. Furthermore, specific details regarding the configurations described in FIG. 3 (e.g., computing device (100), etc.) may be replaced by the details previously described through FIG. 1 and 2. In one embodiment, the examples illustrated in FIG. 3 may be performed by a computing device (100) (e.g., a processor (110) of the computing device (100)).

[0079] Referring to FIG. 3, the computing device (100) can parse the received query (310).

[0080] In one embodiment, the computing device (100) can determine a processing method and / or a storage method for the received query by parsing the received query. In one embodiment, the processing method may include determining a pipeline for the query, determining an optimization method for the query, or determining operators corresponding to each of the pipelines for the query. In one embodiment, the storage method may include an internal storage method or an external storage method. For example, the internal storage method may include a storage method that utilizes storage within the database (e.g., memory, etc.), such as output in the form of a table. Additionally, the external storage method may include a storage method that utilizes storage outside the database (e.g., a permanent storage medium such as a disk), such as output in the form of a file.

[0081] The computing device (100) can perform logical optimization on the parsed query to determine the order for processing requests included in the parsed query (320).

[0082] In one embodiment, logical optimization may be a process of analyzing the logical structure of a query. Logical optimization may mean performing at least one of query analysis, determining join order, selecting indexes, optimizing filtering conditions, optimizing aggregate functions, and / or optimizing window functions. In one embodiment, query analysis is a process of analyzing a query to determine the intent of the query, and may mean determining the tables to be queried by the query, join conditions, filtering conditions, etc. In one embodiment, determining join order may mean determining the optimal join order when joining multiple tables. In one embodiment, index selection may mean determining the indexes that the query can use and selecting the optimal index. Optimizing filtering conditions may mean analyzing filtering conditions in the WHERE clause to apply conditions in the optimal order or, if possible, reconstructing conditions to improve performance. In one embodiment, in the case of a query containing aggregate functions, aggregate function optimization may mean determining the order of operations, etc., to perform aggregation in the optimal way. In one embodiment, window function optimization may mean determining the order of operations, etc., so that the window function can be calculated in an optimal way for a query containing a window function.

[0083] The computing device (100) can perform physical optimization on a logically optimized query to determine the operation method required to process requests according to a determined order (330).

[0084] In one embodiment, physical optimization is performed after logical optimization is completed and may refer to maximizing performance by considering the characteristics of the actual database system. Physical optimization may include index selection and construction, table partitioning, query tuning, etc. In one embodiment, index selection and construction may refer to building actual indexes or changing the indexes to be used based on the index selection determined in logical optimization. Physical optimization can improve data access speed by selecting optimal indexes that match the filtering and join conditions of a query. In one embodiment, table partitioning may refer to physically dividing a table to improve the performance of large-capacity tables. In one embodiment, query tuning may refer to optimizing query execution plans by considering the physical environment of the actual database system, such as hardware resources, the operating system, and storage devices.

[0085] The computing device (100) can build a plurality of pipelines for processing queries and dependencies between the plurality of pipelines based on a physically optimized query (340).

[0086] In one embodiment, the pipeline may be a set of one or more operators. The pipeline may represent a unit of processing for a query.

[0087] In one embodiment, an operator may be a symbol or keyword representing a function used to process and manipulate data in a database. Such operators may be used to perform various operations such as filtering, sorting, grouping, or combining data.

[0088] In one embodiment, the computing device (100) can construct a plurality of pipelines based on an operator corresponding to a pipeline breaker.

[0089] In one embodiment, the pipeline breaker is an operator that is the start or end of each pipeline and may include at least one of a sorting operator, a grouping operator, a join operator, an aggregate function, or a window function. The operations of these operators may be operations that require all tuples. In one embodiment, the next pipeline can be executed only after all outputs for the corresponding pipeline have been generated. For example, if there is a dependency between the first pipeline and the second pipeline, the second pipeline can be executed only after the first pipeline has finished.

[0090] In one embodiment, the computing device (100) can determine the type of a plurality of operators included in each of a plurality of pipelines as one of an origin role type, an on-the-fly role type, or a destination role type.

[0091] In one embodiment, the origin role type acts as the start operator of the pipeline and can create, load, or preprocess chunks. For example, the origin role type may include a scan operator, a sort operator, etc.

[0092] In one embodiment, the on-the-fly role type can receive a single chunk and process it in memory. For example, the on-the-fly role type may include a filter operator.

[0093] In one embodiment, the destination role type can process a chunk to generate an output chunk. For example, the destination role type may include a sorting operator, an aggregate function, a window function, etc.

[0094] In one embodiment, operators that overlap in multiple pipelines may have different types depending on the pipeline. For example, if a sorting operator overlaps in a first pipeline and a second pipeline, the sorting operator in the first pipeline may be of the destination role type, and the sorting operator in the second pipeline may be of the origin role type.

[0095] The computing device (100) can establish at least one schedule for processing queries based on dependencies between multiple pipelines (350).

[0096] In one embodiment, at least one schedule may mean a time sequence of execution based on dependencies between multiple pipelines. For example, if there is a dependency between a first pipeline and a second pipeline, the computing device (100) may establish a schedule to execute the first pipeline first and then the second pipeline.

[0097] The computing device (100) can build multiple tasks based on at least one schedule (360).

[0098] In the present disclosure, work may refer to a unit of work that can be processed at once in a worker thread.

[0099] The computing device (100) can add multiple tasks to a task queue (370).

[0100] In the present disclosure, a work queue may refer to a queue that holds control information for all unit tasks within a system at a given time. The work queue can coordinate and manage tasks so that they are executed in order when multiple tasks are requested simultaneously.

[0101] The computing device (100) can process multiple tasks by assigning multiple tasks added to the task queue to each of the multiple worker threads included in the worker thread pool (380).

[0102] In the present disclosure, a worker thread pool may refer to software that manages a plurality of worker threads to be used in parallel. In the present disclosure, a worker thread may refer to a thread allocated to process a specific task.

[0103] In one embodiment, the computing device (100) may assign a first-1 task to a first worker thread to process a first-1 task among a plurality of first tasks corresponding to a first schedule in a first worker thread.

[0104] In one embodiment, the computing device (100) can process the first-1 task using a first worker thread. In one embodiment, the computing device (100) can process the first-1 task based on the type of the first-1 task.

[0105] For example, if the type of the 1-1 task is a shared task shared with other worker threads, the computing device (100) can assign the 1-2 operator, which is a destination operator among at least one 1 operator included in the 1 pipeline, as the current operator currently being processed.

[0106] The computing device (100) may attempt to acquire a third chunk. For example, the computing device (100) may attempt to acquire a third chunk generated from an independent task (e.g., an independent task performed in a previous pipeline) using a first-2 operator. The computing device (100) may perform different actions depending on whether the acquisition of the third chunk is successful. For example, if the computing device (100) succeeds in acquiring the third chunk, it may execute the first-2 operator. As another example, if the computing device (100) fails to acquire the third chunk, it may increase the number of completed shared tasks by a predetermined number (e.g., 1). The computing device (100) may perform a return to complete the process of handling the first-1 task. In one embodiment, the third chunk may be a chunk generated through a scan operator in an independent task. For example, the third chunk may be a chunk generated through a single scan operator. As another example, the third chunk may be a single merged chunk created by combining chunks generated respectively through two scan operators when the pipeline ends with a Join probe. In one embodiment, the third chunk may be the first chunk generated using the first-1 operator, which is a scan operator.

[0107] In one embodiment, failure to acquire a chunk may include cases where all chunks have been acquired and there are no more chunks to acquire. That is, failure to acquire a chunk may be a case where the entire chunk containing said chunk has been successfully acquired. Accordingly, when the computing device (100) has successfully acquired the entire chunk, it may increase the number of completed shared tasks (or completed independent tasks) by a predetermined number.

[0108] In another example, if the type of the 1-1 task is an isolated task that is not shared with other worker threads, the computing device (100) can assign the 1-1 operator, which is the origin operator among at least one 1 operator included in the 1 pipeline, as the current operator currently being processed.

[0109] The computing device (100) can check the type of the first-1 operator if the assigned first-1 operator's role type is the origin role type.

[0110] The computing device (100) may attempt to acquire a second chunk if the type of the first-1 operator is not a scan operator. The computing device (100) may perform different actions depending on whether the acquisition of the second chunk is successful. For example, if the computing device (100) succeeds in acquiring the second chunk, it may execute the first-1 operator. As another example, if the computing device (100) fails to acquire the second chunk, it may increase the number of completed independent tasks by a predetermined number (e.g., 1). In one embodiment, the second chunk may be a chunk generated through a scan operator in an independent task. For example, the second chunk may be a chunk generated through a single scan operator in a previous pipeline. As another example, if the previous pipeline ends with a Join probe, the second chunk may be a single merged chunk created by combining chunks generated through two scan operators in the previous pipeline.

[0111] The computing device (100) may attempt to obtain a tuple from a data file of a DBMS when the type of the first-1 operator is a scan operator. In one embodiment, the computing device (100) may read a page from the buffer pool of the first engine by reading the data file through the second engine by executing a scan operator (e.g., a scan operator generated by the first engine).

[0112] In one embodiment, if the computing device (100) fails to acquire a tuple, it may increase the number of completed independent tasks by a predetermined number (e.g., 1). In one embodiment, failure to acquire a tuple may include cases where all tuples of the read page have been acquired and there are no more tuples to acquire. That is, failure to acquire a tuple may be a case where the entire tuple containing the tuple has been successfully acquired. Accordingly, if the computing device (100) has successfully acquired the entire tuple, it may increase the number of completed independent tasks by a predetermined number.

[0113] In one embodiment, when the number of completed isolated tasks corresponds to a predetermined total number of isolated tasks, the computing device (100) can schedule shared tasks. The computing device (100) can assign the type of the first-1 task to the shared task type. For example, the computing device (100) can build the first-1 task to correspond to the shared task type. The computing device (100) can add the first-1 task built to correspond to the shared task type to a task queue. The computing device (100) can perform a return to complete the process of handling the first-1 task.

[0114] In one embodiment, if the number of completed independent tasks does not correspond to a predetermined total number of independent tasks, the computing device (100) may perform a return that completes the process of processing the first-1 task.

[0115] When the computing device (100) succeeds in obtaining a tuple, it can convert the obtained tuple into a vector by column and select the vectors necessary for processing the query to create a first chunk. In one embodiment, the computing device (100) can vectorize data in a row-oriented format within a page to create a single chunk.

[0116] In the present disclosure, a chunk may mean a set of vectors. For example, a chunk may be a set of vectors required to process a query. A chunk may include at least one vector. In one embodiment, the first chunk, the second chunk, and the third chunk may be different from each other or at least partially corresponding. For example, the first chunk, the second chunk, and the third chunk may be corresponding to each other or identical.

[0117] In one embodiment, if the vector size of a generated chunk is smaller than a predetermined vector size, the computing device (100) may repeatedly perform the step of attempting to obtain tuples from a data file of the DBMS. During the process of repeating, the number of vectors included in a chunk may increase, thereby increasing the vector size of a chunk. The computing device (100) may repeatedly perform the step of attempting to obtain tuples from a data file of the DBMS until the vector size of a generated chunk corresponds to a predetermined vector size.

[0118] In one embodiment, if the vector size of a generated chunk is larger than the size of a predetermined vector, the computing device (100) can perform at least one of compression and / or filtering on the chunk by executing a scan operator.

[0119] In one embodiment, compression may mean performing space optimization related to storage space by changing the method of vectorization for data. For example, compression may mean performing space optimization related to storage space by changing the method of compressing data to one of dictionary-based compression, Run-Length-Encoding (RLE), bitmap encoding, or delta encoding. In one embodiment, dictionary-based compression may be a method that saves storage space by reducing data redundancy by storing frequently used terms, patterns, etc., in a dictionary. In one embodiment, RLE may be a method that compresses sections where the same value appears consecutively in a single column. For example, RLE may compress sections where the same value appears consecutively in a single column using triplets. A triplet may be the attribute value, the starting position (offset) in the column interval, and the number of elements (length) of the interval. In one embodiment, bitmap encoding may be a method that stores a separate bitmap for each unique value of an attribute. The offset of a vector may correspond to a tuple. For example, the i-th position of a bitmap (e.g., i is a natural number) may correspond to the i-th tuple of a table. In one embodiment, the bitmap encoding method may be used to represent binary states (e.g., male / female, present / non-present, true / false, etc.). For example, the bitmap encoding method may enable fast searching for male or female by compressing by converting male to 1 and female to 0. In one embodiment, the difference encoding method may be a method of recording the difference between values ​​that appear consecutively in the same column.Information about the difference or the default value, which is the first element of the data (e.g., the first value of a column), can be stored directly in the column or in a separate lookup table.

[0120] In one embodiment, filtering may mean selecting items from data that meet the required conditions. The computing device (100) may add an index to the selection vector determined by performing filtering.

[0121] After the step of assigning the first-1 operator, which is the origin operator among at least one first operator included in the first pipeline, as the current operator, the computing device (100) can perform the first-1 operator if the role type of the first-1 operator is not the origin role type.

[0122] After the step of assigning the first-1 operator, which is the origin operator among at least one first operator included in the first pipeline, as the current operator, the computing device (100) can execute the first-1 operator if the role type of the first-1 operator is not the origin role type.

[0123] The computing device (100) can assign the role type of the first-1 operator to the origin role type if the role type of the first-1 operator is the destination role type. And the computing device (100) can re-execute the first-1 operator if the type of the first-1 operator is not the scan operator.

[0124] If the role type of the first-1 operator is not a destination role type, the computing device (100) may assign the first-2 operator, which corresponds to the next order of the first-1 operator, as the current operator. And if the role type of the first-2 operator is not an origin role type or the type of the first-2 operator is not a scan operator, the computing device (100) may execute the first-2 operator.

[0125] In one embodiment, the computing device (100) can process the first-1 task by using a first worker thread to execute at least one operator included in a first pipeline corresponding to the first-1 task.

[0126] In one embodiment, the computing device (100) may assign a new task to the first worker thread if the type of the processed first-1 task is not a shared task type that is shared with other worker threads. For example, the computing device (100) may assign one of a plurality of first tasks corresponding to a first schedule.

[0127] In one embodiment, the computing device (100) may assign a new task to a first worker thread if the number of completed shared tasks does not correspond to a predetermined total number of shared tasks. For example, the computing device (100) may assign one of a plurality of first tasks corresponding to a first schedule to the first worker thread.

[0128] In one embodiment, the computing device (100) can determine whether there exists a second schedule different from the first schedule if the type of the processed first-1 task is a shared task type that is shared with other worker threads, and the number of completed shared tasks corresponds to the number of predetermined total shared tasks.

[0129] In one embodiment, the computing device (100) may assign a new task to the first worker thread when it is determined that the second schedule does not exist. For example, the computing device (100) may assign one of a plurality of first tasks corresponding to the first schedule to the first worker thread.

[0130] In one embodiment, when it is determined that a second schedule exists, the computing device (100) can construct a plurality of second tasks corresponding to the second schedule using the first worker thread.

[0131] In one embodiment, the computing device (100) can add a plurality of second tasks to a task queue.

[0132] The computing device (100) can generate a first execution result including information about the result of processing a plurality of tasks (390).

[0133] Information regarding the result of processing a plurality of operations in the present disclosure may be data including specific values, specific tables, specific charts, etc. obtained by executing the query.

[0134] In one embodiment, the computing device (100) can generate an execution result corresponding to a query by executing an operator included in each of the plurality of pipelines. In one embodiment, the computing device (100) can generate an execution result corresponding to a query by executing an operator included in each of the plurality of pipelines according to the order assigned to each of the plurality of pipelines.

[0135] FIG. 4 illustrates an exemplary block diagram of a DBMS according to one embodiment of the present disclosure. Specific details regarding redundant configurations among the configurations shown in FIG. 4 may be omitted and replaced with the details described in FIG. 1 to 3.

[0136] Referring to FIG. 4, the DBMS (400) may include a first engine (410), a second engine (420), a data file (430), etc.

[0137] In one embodiment, the first engine (410) may include an OLAP-based engine. In one embodiment, the first engine (410) may be a management system capable of processing a large amount of data in a database. The first engine (410) may use recorded and aggregated data from multiple sources. Accordingly, the first engine (410) may be specialized in processing analytical queries with a large amount of data rather than transaction queries with a relatively small amount of data.

[0138] In one embodiment, the second engine (420) may include an OLTP-based engine. In one embodiment, the second engine (420) may be a management system that enables processing transactions in a database. The second engine (420) may use real-time and transactional data from a single source. Accordingly, the second engine (420) may be specialized in processing transactional queries with relatively small data sizes rather than analytical queries with large data sizes.

[0139] In one embodiment, the data file (430) may be data stored in the memory (130) of the computing device (100). In one embodiment, the data file (430) may be data stored on an OLTP basis. Accordingly, the first engine (410) may only perform reading of the data file (430). The second engine (420) may perform writing and reading of the data file (430).

[0140] FIG. 5 illustrates an exemplary pipeline corresponding to an input query according to one embodiment of the present disclosure. Specific details regarding redundant configurations among the configurations shown in FIG. 5 may be omitted and replaced with the details described in FIG. 1 to 4.

[0141] Referring to FIG. 5, in one embodiment, the query (510) may include a SELECT MEDIAN(A) statement (511), a FROM T clause (512), a WHERE C!=E clause (513), an ORDER BY C, E clause (514), etc.

[0142] In one embodiment, the SELECT MEDIAN(A) statement (511) can be used to select the median of A.

[0143] In one embodiment, the FROM T clause (512) may be used to select a table to be used to select the median value of A.

[0144] In one embodiment, the WHERE C!=E clause (513) can be used to select rows in a selected table where the values ​​of column C and column E are different.

[0145] In one embodiment, the ORDER BY C, E clause (514) can be used to sort the selected results based on column C and column E.

[0146] In one embodiment, the first pipeline (520) may include a scan operator (521), a filter operator (522), a sort operator (531), etc.

[0147] In one embodiment, the second pipeline (530) may include a sorting operator (531), an aggregate function (532), etc.

[0148] In one embodiment, the scan operator (521) may be an operator that scans A, C, and E in a selected table.

[0149] In one embodiment, the filter operator (522) is configured to correspond to the WHERE C!=E clause (513) and may be an operator that selects rows in the scanned data where the values ​​of column C and column E are different.

[0150] In one embodiment, the sorting operator (531) is configured to correspond to the ORDER BY C, E clause (514) and may be a scanning operator that sorts the selected results based on column C and column E. The sorting operator (531) may be included in duplicate in the first pipeline (520) and the second pipeline (530).

[0151] As described above, the computing device (100) can construct a plurality of pipelines (520, 530) to process the query (510).

[0152] FIG. 6 illustrates an exemplary scan according to one embodiment of the present disclosure. Specific details regarding redundant configurations among the configurations shown in FIG. 6 may be omitted and replaced with the details described above in FIG. 1 to 5.

[0153] Referring to FIG. 6, a computing device (100) can scan columns A, C, and E in a table (620) using a scan operator (610) that scans columns A, C, and E in a table, and convert them into vectors row by row. For example, the computing device (100) can scan the row of column A of the table (620) and convert it into a first vector (631). The computing device (100) can scan the row of column C of the table (620) and convert it into a second vector (632). The computing device (100) can scan the row of column E of the table (620) and convert it into a third vector (633).

[0154] In one embodiment, the computing device (100) can generate a chunk (630) including a first vector (631), a second vector (632), and a third vector (633).

[0155] FIG. 7 illustrates an exemplary process for processing a plurality of operations according to one embodiment of the present disclosure. Specific details regarding redundant configurations among the configurations shown in FIG. 7 may be omitted and replaced with the details described above in FIG. 1 to 6.

[0156] In one embodiment, the computing device (100) may establish at least one schedule for processing a query based on dependencies between a plurality of pipelines. For example, the computing device (100) may establish a first schedule (730) for processing a query based on dependencies between a first pipeline (710) and a second pipeline (720).

[0157] In one embodiment, the computing device (100) can construct a plurality of tasks (741) based on a first schedule (730).

[0158] In one embodiment, the computing device (100) can add a plurality of tasks (741) to a task queue (740).

[0159] In one embodiment, the computing device (100) can process a plurality of tasks (741) by assigning a plurality of tasks (741) added to a task queue (740) to a plurality of worker threads (751) included in a worker thread pool (750).

[0160] FIG. 8 illustrates an exemplary process for processing a plurality of operations according to another embodiment of the present disclosure. Specific details regarding redundant configurations among the configurations shown in FIG. 8 may be omitted and replaced with the details described above in FIG. 1 to 7.

[0161] Referring to FIG. 8, the computing device (100) can establish at least one schedule for processing a query based on dependencies between multiple pipelines. For example, the computing device (100) can establish a first schedule (850) and a second schedule (860) for processing a query based on dependencies between a first pipeline (810) and a second pipeline (820).

[0162] In one embodiment, the computing device (100) may add a task to the task queue (830) by taking into account the dependency between the first schedule (850) and the second schedule (860). For example, the computing device (100) may add the first task (832) of the second schedule (860) to the task queue (830) after the last task (831) of the first schedule (850) so that the first task (832) of the second schedule (860) is assigned to each of the multiple worker threads (841) included in the worker thread pool (840).

[0163] FIG. 9 exemplarily illustrates the process of executing an operator included in a pipeline according to one embodiment of the present disclosure. Specific details regarding redundant configurations among the configurations shown in FIG. 9 may be omitted and replaced with the details described in FIG. 1 to 8.

[0164] Referring to FIG. 9, the computing device (100) can assign a first task (941) and a second task (951), which are independent tasks of the task type, to a first worker thread (920). The computing device (100) can assign a third task (961) and a fourth task (971), which are independent tasks, to a second worker thread (930). Here, the tasks may be in chunks.

[0165] In one embodiment, the computing device (100) can execute operators (911, 912, 913) included in the pipeline (910) until each of the independent tasks (first task (941), second task (951), third task (961), fourth task (971)) in each of the plurality of worker threads (920, 930) becomes shared tasks (first task (942), second task (952), third task (962), fourth task (972)).

[0166] FIG. 10 exemplarily illustrates the process of executing an operator included in a pipeline according to another embodiment of the present disclosure. Specific details regarding redundant configurations among the configurations shown in FIG. 10 may be omitted and replaced with the details described in FIG. 1 through 9.

[0167] Referring to FIG. 10, the computing device (100) can obtain a first task (1021), a second task (1022), a third task (1023), and a fourth task (1024) of which are types of shared tasks by executing operators (1011, 1012, 1013) included in a pipeline (1010) until each of the independent tasks in each of the plurality of worker threads becomes a shared task.

[0168] In one embodiment, the computing device (100) can improve the processing speed of tasks by utilizing the maximum number of available worker threads depending on the situation, by assigning shared tasks to new multiple worker threads (1031, 1032, 1033, 1034) instead of assigning them as they are to existing worker threads that have processed independent tasks. For example, the tasks that the computing device (100) assigned to two worker threads (920, 930) in FIG. 9 can be newly assigned to four worker threads (1031, 1032, 1033, 1034) while remaining as shared tasks.

[0169] FIG. 11 illustrates a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0170] Although the present disclosure has been described as generally being implementable by a computing device, those skilled in the art will understand that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that can be executed on one or more computers, and / or as a combination of hardware and software.

[0171] Generally, a program module includes routines, programs, components, data structures, etc., that perform a specific task or implement a specific abstract data type. Furthermore, those skilled in the art will be well aware that the method of the present disclosure may be implemented in other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, etc. (each of which may be connected to and operated with one or more associated devices).

[0172] The embodiments described in this disclosure may also be implemented in a distributed computing environment in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0173] Computers typically include various computer-readable media. Any medium accessible by a computer may be a computer-readable medium, and such computer-readable media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media. By example, but not limiting, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD (digital video disk) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be accessed by a computer and used to store desired information.

[0174] Computer-readable transmission media typically include all information transmission media that implement computer-readable instructions, data structures, program modules, or other data, etc., on a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal in which one or more of the characteristics of the signal are set or modified to encode information within the signal. By example, not limiting, computer-readable transmission media include wired media, such as wired networks or direct-wired connections, and wireless media, such as acoustic, RF, infrared, and other wireless media. Any combination of the media described above is also considered to be within the scope of computer-readable transmission media.

[0175] An exemplary environment for implementing various aspects of the present disclosure, including a computer (1102), is shown, wherein the computer (1102) includes a processing unit (1104), system memory (1106), and a system bus (1108). The system bus (1108) connects system components, including system memory (1106) (but not limited thereto), to the processing unit (1104). The processing unit (1104) may be any processor among various commercial processors. Dual processors and other multiprocessor architectures may also be used as the processing unit (1104).

[0176] The system bus (1108) may be any of several types of bus structures that can be additionally interconnected to a local bus using any of the memory bus, peripheral bus, and various commercial bus architectures. System memory (1106) includes read-only memory (ROM) (1110) and random access memory (RAM) (1112). The basic input / output system (BIOS) is stored in non-volatile memory (1110), such as ROM, EPROM, EEPROM, etc., and this BIOS includes basic routines that help transfer information between components within the computer (1102) at times such as during startup. The RAM (1112) may also include high-speed RAM, such as static RAM, for caching data.

[0177] The computer (1102) also includes an internal hard disk drive (HDD) (1114) (e.g., EIDE, SATA)—this internal hard disk drive (1114) may also be configured for external use within a suitable chassis (not shown)—a magnetic floppy disk drive (FDD) (1116) (e.g., for reading from or writing to a removable diskette (1118)), and an optical disk drive (1120) (e.g., for reading from a CD-ROM disk (1122) or reading from or writing to other high-capacity optical media such as a DVD). The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) may each be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128). The interface (1124) for implementing an external drive includes at least one or both of USB (Universal Serial Bus) and IEEE 1394 interface technologies.

[0178] These drives and associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, etc. In the case of a computer (1102), the drives and media correspond to storing any data in a suitable digital format. Although the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will know that other types of computer-readable media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, etc., may also be used in exemplary operating environments and that any of these media may contain computer-executable instructions for performing the methods of the present disclosure.

[0179] A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), may be stored in the drive and RAM (1112). All or part of the operating system, application, module and / or data may also be cached in RAM (1112). It will be well known that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

[0180] The user can input commands and information into the computer (1102) through one or more wired / wireless input devices, such as a pointing device like a keyboard (1138) and a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, etc. These and other input devices are often connected to the processing unit (1104) via an input device interface (1142) connected to the system bus (1108), but may also be connected via other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, etc.

[0181] A monitor (1144) or other type of display device is also connected to the system bus (1108) via an interface such as a video adapter (1146). In addition to the monitor (1144), the computer generally includes other peripheral output devices (not shown), such as speakers, a printer, and so on.

[0182] The computer (1102) may operate in a networked environment using a logical connection to one or more remote computers, such as remote computer(s) (1148), via wired and / or wireless communication. The remote computer(s) (1148) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and generally include many or all of the components described for the computer (1102), but for brevity, only the memory storage device (1150) is illustrated. The illustrated logical connection includes a wired / wireless connection to a local area network (LAN) (1152) and / or a larger network, e.g., a wide area network (WAN) (1154). Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to a global computer network, e.g., the Internet.

[0183] When used in a LAN networking environment, the computer (1102) is connected to a local network (1152) via a wired and / or wireless communication network interface or adapter (1156). The adapter (1156) may facilitate wired or wireless communication to the LAN (1152), and the LAN (1152) may also include a wireless access point installed therein to communicate with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) may include a modem (1158), be connected to a communication computing device on the WAN (1154), or have other means to establish communication through the WAN (1154), such as through the Internet. The modem (1158), which may be an internal or external and a wired or wireless device, is connected to the system bus (1108) via a serial port interface (1142). In a networked environment, the program modules described for the computer (1102) or parts thereof may be stored in a remote memory / storage device (1150). It will be well known that the illustrated network connection is exemplary and that other means of establishing a communication link between computers may be used.

[0184] The computer (1102) operates to communicate with any wireless device or object that is deployed and operated by wireless communication, for example, a printer, scanner, desktop and / or portable computer, PDA (portable data assistant), communication satellite, any equipment or place associated with a wireless detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network, or simply ad hoc communication between at least two devices.

[0185] Wi-Fi (Wireless Fidelity) enables connectivity to the Internet and other sources without wires. Wi-Fi is a wireless technology, similar to a cell phone, that allows devices, such as computers, to transmit and receive data indoors and outdoors—that is, anywhere within the coverage area of ​​a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz wireless bands, for example, at data rates of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual band).

[0186] Those skilled in the art of the present disclosure will understand that information and signals may be represented using any various different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0187] Those skilled in the art will understand that the various exemplary logic blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented by electronic hardware, various forms of programs or design code (referred to herein as software for convenience), or a combination of all such. To clearly illustrate this interoperability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in relation to their functions. Whether such functions are implemented as hardware or software depends on the design constraints imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementation decisions should not be interpreted as being outside the scope of this disclosure.

[0188] The various embodiments presented herein may be implemented as methods, devices, or articles manufactured using standard programming and / or engineering techniques. The term "article manufactured" includes a computer program, a carrier, or a medium accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical discs (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Additionally, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0189] It should be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of this disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but do not imply being limited to the specific order or hierarchy presented.

[0190] Description of the presented embodiments is provided so that a person skilled in the art may use or practice the present disclosure. Various modifications to these embodiments will be apparent to a person skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein, but should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

Claims

Claim 1 A method for processing a query executed by a computing device capable of operating based on a DBMS (Database Management System), comprising: receiving a query requesting an execution result in said DBMS; and determining the type of said query as a transactional query or an analytic query based on predetermined criteria for classifying said query. The method comprises the step of generating an execution result corresponding to the query by adaptively determining, among a plurality of different engines, an engine to process the query according to the determined type of the query, or by processing the query in a different way; wherein the step of generating the execution result corresponding to the query comprises, when the type of the query is determined to be the analysis query, building a plurality of pipelines for processing the query and dependencies between the plurality of pipelines based on the query - wherein a pipeline is a set of one or more operators and represents a processing unit of the query -; building at least one schedule for processing the query based on the dependencies between the plurality of pipelines; building a plurality of tasks based on the at least one schedule and adding them to a task queue; processing the plurality of tasks by assigning the plurality of tasks added to the task queue to a plurality of worker threads included in a worker thread pool, respectively; and generating the execution result including information on the result of processing the plurality of tasks.A method comprising, and the step of processing the plurality of tasks, wherein the step of assigning the first-1 task to the first worker thread in order to process the first-1 task among the plurality of first tasks corresponding to the first schedule in the first worker thread; and the step of processing the first-1 task using the first worker thread; Claim 2 A method according to claim 1, wherein the predetermined criterion includes a criterion related to the number of tuples to be processed to generate a result of a query, wherein the transaction query is a query in which the number of tuples of the query is less than a predetermined first threshold, and the analysis query is a query in which the number of tuples of the query is greater than or equal to the predetermined first threshold. Claim 3 A method according to claim 1, wherein the predetermined criteria include criteria related to the type of operator of the query, and the transaction query includes at least one of a query including a Database Manipulation Language (DML), a query including a Database Definition Language (DDL), a query for searching specific data, or a query where an index exists when searching for data, and the analysis query includes at least one of an order by, a group by, a join, an aggregation, or a window function. Claim 4 A method according to claim 1, wherein the predetermined criteria include criteria related to the processing method for the transaction of the query, the transaction query is a query used in OLTP (Online Transaction Processing), and the analysis query is a query used in OLAP (Online Analytical Processing). Claim 5 A method comprising: a step of generating an execution result corresponding to the query in claim 1, wherein, when the type of the query is determined to be the analysis query, a step of generating a first execution result corresponding to the query by processing the query using a first engine for processing the analysis query; and a step of generating a second execution result corresponding to the query by processing the query using a second engine for processing the transaction query when the type of the query is determined to be the transaction query. Claim 6 In claim 5, the first engine includes an OLAP (Online Analytical Processing) based query execution engine, and the second engine includes an OLTP (Online Transaction Processing) based query execution engine. Claim 7 The method according to claim 1, wherein the step of generating the execution result corresponding to the query further comprises: a step of parsing the received query prior to the step of establishing the plurality of pipelines and the dependencies between the plurality of pipelines; a step of performing a logical optimization on the parsed query to determine the order for processing requests included in the parsed query; and a step of performing a physical optimization on the logically optimized query to determine the operation method required to process the requests according to the determined order. Claim 8 The method according to claim 1, wherein the step of establishing the plurality of pipelines and the dependencies between the plurality of pipelines comprises the step of establishing the plurality of pipelines based on an operator corresponding to a pipeline breaker; wherein the pipeline breaker is an operator that is the start or end of each pipeline and includes at least one of a sorting operator, a grouping operator, a join operator, an aggregate function, or a window function. Claim 9 The method according to claim 1, wherein the step of establishing dependencies between the plurality of pipelines and the plurality of pipelines comprises: determining the type of a plurality of operators included in each of the plurality of pipelines as one of an origin role type, an on-the-fly role type, and a destination role type, wherein the origin role type acts as the start operator of the pipeline and performs the creation, loading, or pre-processing of a chunk, the on-the-fly role type receives a chunk and processes it in memory, and the destination role type processes the chunk to create an output chunk. Claim 10 delete Claim 11 A method according to claim 1, wherein the step of processing the first-1 task using the first worker thread comprises: processing the first-1 task by executing at least one operator included in a first pipeline corresponding to the first-1 task using the first worker thread; determining whether a second schedule different from the first schedule exists when the type of the processed first-1 task is a shared task type shared with other worker threads and the number of completed shared tasks corresponds to a predetermined total number of shared tasks; when it is determined that the second schedule exists, constructing a plurality of second tasks corresponding to the second schedule using the first worker thread; and adding the plurality of second tasks to the task queue. Claim 12 In claim 11, the step of processing the 1-1 task by executing at least one operator included in the first pipeline corresponding to the 1-1 task using the first worker thread comprises: a step of assigning the 1-1 operator, which is the origin operator among at least one first operator included in the first pipeline, as the current operator currently being processed, if the type of the 1-1 task is an isolated task not shared with other worker threads; a step of checking the type of the 1-1 operator if the role type of the 1-1 operator is the origin role type; and a step of attempting to obtain a tuple from the data file of the DBMS if the type of the 1-1 operator is a scan operator. A method comprising: a step of, if successful in obtaining the above tuple, converting the obtained tuple into a vector by column and selecting a vector necessary to process the query to create a first chunk; a step of, if unsuccessful in obtaining the above tuple, increasing the number of completed independent tasks by a predetermined number; and a step of scheduling a shared task if the number of completed independent tasks corresponds to a predetermined total number of independent tasks. Claim 13 A method according to claim 12, further comprising: a step of attempting to acquire a second chunk if, after the step of checking the type of the first-1 operator, the type of the first-1 operator is not a scan operator; a step of executing the first-1 operator if the acquisition of the second chunk is successful; and a step of increasing the number of completed independent tasks by the predetermined number if the acquisition of the second chunk is unsuccessful. Claim 14 A method according to claim 11, wherein the step of processing the first-1 task by executing at least one operator included in the first pipeline corresponding to the first-1 task using the first worker thread comprises: a step of, if the type of the first-1 task is a shared task, assigning a first-2 operator, which is a destination operator among at least one first operator included in the first pipeline, as the current operator currently being processed; a step of attempting to acquire a third chunk generated from an independent task; a step of executing the first-2 operator if the acquisition of the third chunk is successful; and a step of increasing the number of completed shared tasks by a predetermined number if the acquisition of the third chunk is unsuccessful. Claim 15 A computer program stored on a computer-readable storage medium, wherein the computer program causes a processor of a computing device capable of operating based on a DBMS (Database Management System) to perform a method for processing a query, the method comprising: receiving a query requesting an execution result in the DBMS; and determining the type of the query as a transactional query or an analytic query based on predetermined criteria. The method comprises the step of generating an execution result corresponding to the query by adaptively determining, among a plurality of different engines, an engine to process the query according to the determined type of the query, or by processing the query in a different way; wherein the step of generating the execution result corresponding to the query comprises, when the type of the query is determined to be the analysis query, building a plurality of pipelines for processing the query and dependencies between the plurality of pipelines based on the query - wherein a pipeline is a set of one or more operators and represents a processing unit of the query -; building at least one schedule for processing the query based on the dependencies between the plurality of pipelines; building a plurality of tasks based on the at least one schedule and adding them to a task queue; processing the plurality of tasks by assigning the plurality of tasks added to the task queue to a plurality of worker threads included in a worker thread pool, respectively; and generating the execution result including information on the result of processing the plurality of tasks.A computer program stored on a computer-readable storage medium, comprising: a step of processing a plurality of tasks, wherein the step of processing a plurality of tasks comprises: a step of assigning a first-1 task to a first worker thread to process a first-1 task among a plurality of first tasks corresponding to a first schedule in a first worker thread; and a step of processing the first-1 task using the first worker thread. Claim 16 A computing device capable of operating based on a DBMS (Database Management System) comprises: a processor; memory; and a network unit; wherein the processor performs the operation of receiving a query requesting an execution result in the DBMS; and the operation of determining the type of the query as a transactional query or an analytic query based on predetermined criteria. and, depending on the determined type of the above query, an operation to generate an execution result corresponding to the above query by adaptively determining an engine to process the above query among a plurality of different engines or by processing the above query in a different way; and the operation to generate the execution result corresponding to the above query comprises, when the type of the above query is determined to be the analysis query, an operation to build a plurality of pipelines for processing the above query and dependencies between the plurality of pipelines based on the above query - where a pipeline is a set of one or more operators and represents a processing unit of the above query -; an operation to build at least one schedule for processing the above query based on the dependencies between the plurality of pipelines; an operation to build a plurality of tasks based on the at least one schedule and add them to a task queue; an operation to process the plurality of tasks by assigning the plurality of tasks added to the task queue to a plurality of worker threads included in a worker thread pool, respectively; and an operation to generate the execution result including information about the result of processing the plurality of tasks.A computing device comprising, and wherein the operation of processing the plurality of tasks comprises: an operation of assigning the first-1 task to the first worker thread in order to process the first-1 task among the plurality of first tasks corresponding to the first schedule in the first worker thread; and an operation of processing the first-1 task using the first worker thread.

Citation Information

Patent Citations

  • Query intermediate language method and system

    KR1020050000348A

  • Query execution apparatus and method, and system for processing data employing the same

    KR1020160018918A

  • Systems and methods for managing highly available distributed hybrid transactional and analytical databases.

    KR1020220077108A