Enhanced database query supporting debug output, apparatus, device, and medium

Abstract

The application provides an enhanced database query supporting debugging output, an apparatus, a device and a medium, relates to the technical field of databases, and comprises the following steps: receiving a query statement to be processed; performing lexical analysis, syntax analysis and semantic analysis on the query statement through a query parser to construct a semantic tree structure corresponding to the query statement, wherein the semantic tree structure comprises semantic nodes corresponding to clauses, and a semantic node corresponding to a clause in which a non-destructive observation instruction is embedded is associated with an observation node; generating a query plan tree carrying an observation hook based on the semantic tree structure through a query optimizer; and executing the query plan tree through an execution engine to obtain a response result corresponding to the query statement and intermediate observation snapshot data corresponding to the non-destructive observation instruction. The application can effectively solve the black-box debugging problem of a traditional CTE while maintaining transaction consistency and execution efficiency.

Description

Technical Field

[0001] This invention relates to the field of database query technology, and in particular to an enhanced database query, apparatus, device, and medium that supports debug output. Background Technology

[0002] With the continuous development of database technology, Common Table Expressions (CTEs) have become an important tool for constructing temporary result sets in modern SQL (Structured Query Language) queries. In existing implementations, the execution of a CTE is entirely controlled by the database engine; the entire workflow is a black box to the user, with the system only returning the final result set without exposing any intermediate states. For regular CTEs, this mechanism does not cause obvious problems; however, when dealing with recursive or multi-level nested CTEs, especially when the query logic involves complex condition filtering, multi-table joins, or business rule operations, if developers find the final result does not meet expectations, existing solutions can only indirectly infer the root cause of the problem by adding debug logs, disassembling the query statement, or simulating the execution environment. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide an enhanced database query, apparatus, device and medium that supports debug output, which can effectively solve the black-box debugging problem of traditional CTE while maintaining transaction consistency and execution efficiency.

[0004] In a first aspect, the present invention provides an enhanced database query method that supports debug output, comprising:

[0005] Receive a query statement to be processed, wherein at least one clause in the query statement contains a non-destructive observation instruction;

[0006] The query parser performs lexical, syntactic, and semantic parsing on the query statement to construct a semantic tree structure corresponding to the query statement. The semantic tree structure contains semantic nodes corresponding to clauses, and the semantic nodes corresponding to clauses that embed non-destructive observation instructions are associated with observation nodes.

[0007] The query optimizer generates a query plan tree carrying observation hooks based on the semantic tree structure. The observation hooks are the execution operators corresponding to the observation nodes. The observation hooks are used to trigger the execution engine to generate intermediate observation snapshot data corresponding to non-destructive observation instructions.

[0008] The execution engine executes the query plan tree to obtain the response results corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instructions.

[0009] In one implementation, a query plan tree carrying observation hooks is generated based on a semantic tree structure by a query optimizer, including:

[0010] The query optimizer generates a query plan tree based on the semantic nodes contained in the semantic tree structure, generates observation hooks based on the observation nodes contained in the semantic tree structure, and then implants the observation hooks into the query plan tree.

[0011] In one implementation, the execution engine includes a query executor and an enhanced debug output executor; the execution engine executes the query plan tree to obtain the response results corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instructions, including:

[0012] The query executor executes the execution operators contained in the query plan tree. During execution, if the currently executing execution operator is associated with an observation hook, the enhanced debug output executor is triggered to execute the observation hook in parallel to obtain intermediate observation snapshot data corresponding to the non-destructive observation instructions, until the query plan tree is completed and the response result corresponding to the query statement is obtained.

[0013] In one implementation, the non-destructive observation command is a single observation command; the enhanced debug output executor is triggered to execute the observation hook in parallel to obtain intermediate observation snapshot data corresponding to the non-destructive observation command, including:

[0014] Enhance the debug output executor to execute single observation instructions in parallel, and take snapshots of the intermediate results obtained by the execution operators associated with the query executor's execution observation hooks to obtain intermediate observation snapshot data.

[0015] In one implementation, the non-destructive observation command is a recursive observation command; triggering the enhanced debug output executor to execute the observation hook to obtain intermediate observation snapshot data corresponding to the non-destructive observation command, further includes:

[0016] Enhance the debug output executor to execute recursive observation instructions in parallel, so as to listen to the current recursion depth of the execution operator associated with the query executor's execution observation hook. When the current recursion depth reaches the preset recursion depth carried by the recursive observation instruction, take a snapshot of the intermediate results at the current recursion depth to obtain intermediate observation snapshot data.

[0017] In one implementation, after obtaining the intermediate observation snapshot data corresponding to the non-destructive observation command, the method further includes:

[0018] Record the attribute information of intermediate observation snapshot data. The attribute information should include at least the version number and the data source.

[0019] Based on attribute information, intermediate observation snapshot data between two adjacent observations are determined, and incremental capture is performed on the two intermediate observation snapshot data to obtain the data differences between the two intermediate observation snapshot data.

[0020] In one implementation, the method further includes:

[0021] The data differences between two intermediate observation snapshots are marked using specified effects, and the data differences are displayed according to the marks through the graphical user interface of the associated terminal.

[0022] Secondly, the present invention also provides an enhanced database query apparatus that supports debug output, comprising:

[0023] The receiving module is used to receive a query statement to be processed, wherein at least one clause in the query statement contains a non-destructive observation instruction.

[0024] The parsing module is used to perform lexical, syntactic and semantic parsing on the query statement through the query parser in order to construct the semantic tree structure corresponding to the query statement. The semantic tree structure contains semantic nodes corresponding to clauses, and the semantic nodes corresponding to clauses that embed non-destructive observation instructions are associated with observation nodes.

[0025] The optimization module is used to generate a query plan tree carrying observation hooks based on the semantic tree structure through the query optimizer. The observation hooks are the execution operators corresponding to the observation nodes. The observation hooks are used to trigger the execution engine to generate intermediate observation snapshot data corresponding to non-destructive observation instructions.

[0026] The execution module is used to execute the query plan tree through the execution engine to obtain the response results corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instructions.

[0027] Thirdly, the present invention also provides an electronic device including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement any of the methods provided in the first aspect.

[0028] Fourthly, the present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement any of the methods provided in the first aspect.

[0029] This invention provides an enhanced database query, apparatus, device, and medium supporting debug output. First, it receives a query statement to be processed, where at least one clause embeds a non-destructive observation instruction. Then, a query parser performs lexical, syntactic, and semantic parsing on the query statement to construct a semantic tree structure corresponding to the query statement. The semantic tree structure contains semantic nodes corresponding to the clauses, and the semantic nodes corresponding to the clauses embedding non-destructive observation instructions are associated with observation nodes. Next, a query optimizer generates a query plan tree carrying observation hooks based on the semantic tree structure. The observation hooks are execution operators corresponding to the observation nodes, used to trigger the execution engine to generate intermediate observation snapshot data corresponding to the non-destructive observation instructions. Finally, the execution engine executes the query plan tree to obtain the response result corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instructions. The above method introduces non-destructive observation instructions into the query statement and improves the processing of the query parser, query optimizer, and execution engine. The query parser generates a semantic tree structure containing semantic nodes and observation nodes, the query optimizer generates a query plan tree carrying observation hooks, and the execution engine executes the query plan tree to obtain the corresponding response results and intermediate observation snapshot data. Therefore, this invention can effectively solve the black-box debugging problem of traditional CTE while maintaining transaction consistency and execution efficiency.

[0030] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0031] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0032] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 A flowchart illustrating an enhanced database query method supporting debug output provided in an embodiment of the present invention;

[0034] Figure 2 A flowchart for observing intermediate results provided in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the structure of an enhanced database query device that supports debug output, provided in an embodiment of the present invention.

[0036] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Currently, the limitations of existing CTEs include: (1) Invisible temporary results: The execution process of CTEs (including recursive CTEs) is transparent to users, and intermediate results, such as the intermediate state of recursive iteration or the intermediate table of nested CTEs, cannot be directly observed; (2) High debugging cost: When there are too many nesting levels or complex recursive logic, developers can only deduce the problem through logs or final results, which is inefficient; (3) Transaction consistency constraints: In order to ensure ACID properties, traditional databases prohibit the direct output of results in the execution of CTEs to avoid destroying atomicity.

[0039] Based on the limitations mentioned above, typical scenarios present the following pain points:

[0040] In recursive CTE debugging scenarios, particularly in typical applications such as social network analysis and organizational hierarchy queries, recursive CTEs need to handle computational processes with an indeterminate number of iterations. For example, in a query to "find all N-degree friend relationships of a user," the intermediate result set generated by each recursive iteration directly affects the data input of subsequent iterations, but current technologies cannot provide any visibility into the intermediate states during the iteration process. Similarly, in multi-level nested queries containing CTE1→CTE2→CTE3, when the final output is abnormal, developers must manually reconstruct the query structure and locate the root cause of the problem by gradually isolating each CTE module. This debugging method is not only inefficient but also prone to introducing new errors when handling large queries containing complex business logic.

[0041] Furthermore, to achieve ACID (Atomicity, Consistency, Isolation, and Durability) properties, database systems typically strictly limit the output of intermediate results during CTE execution. While this design ensures transaction integrity, it also completely prevents developers from observing the query execution process. When a recursive CTE results in an infinite loop due to improperly set termination conditions, or when a nested CTE causes abnormal results due to an incorrect filtering condition at a certain level, developers lack effective real-time diagnostic tools and can only rely on post-event analysis for troubleshooting, significantly increasing the debugging cost and maintenance difficulty of complex queries.

[0042] Based on this, the present invention provides an enhanced database query, device, equipment and medium that supports debug output. Through an interruptible and observable CTE execution framework, it allows for the dynamic output of intermediate observation snapshot data during recursive or nested CTE execution, which can effectively solve the black-box debugging problem of traditional CTE while maintaining transaction consistency and execution efficiency.

[0043] To facilitate understanding of this embodiment, a detailed description of an enhanced database query method supporting debug output disclosed in this embodiment of the invention will be provided first. (See [link to relevant documentation]). Figure 1 The diagram shows an enhanced database query method that supports debug output. This method mainly includes the following steps S102 to S108:

[0044] Step S102: Receive the query statement to be processed.

[0045] In this query, at least one clause contains a non-destructive observation instruction. This instruction declares that the execution result (i.e., intermediate result) of the embedded clause will be observed. In one example, the non-destructive observation instruction can be a single-observation instruction or a recursive observation instruction. A single-observation instruction is used to snapshot the intermediate result in a non-recursive scenario, while a recursive observation instruction is used to snapshot the intermediate result when a specified recursion depth is reached in a recursive scenario.

[0046] Step S104: The query statement is processed by a query parser through lexical analysis, syntax analysis and semantic analysis to construct the semantic tree structure corresponding to the query statement.

[0047] The semantic tree structure contains semantic nodes corresponding to clauses. Semantic nodes corresponding to clauses embedding non-destructive observation instructions are associated with observation nodes. Observation nodes are semantic nodes obtained by the query parser through lexical, syntactic, and semantic parsing of the non-destructive observation instructions. In one example, for ordinary clauses in the query statement (i.e., clauses without embedded non-destructive observation instructions), the query parser performs regular lexical, syntactic, and semantic parsing to obtain the corresponding semantic nodes. For clauses in the query statement embedding non-destructive observation instructions, lexical, syntactic, and semantic parsing is performed on the non-destructive observation instructions and other instructions in the clause, respectively, to obtain the observation nodes corresponding to the non-destructive observation instructions and the semantic nodes corresponding to the other instructions. The observation nodes and semantic nodes are then associated to finally construct the semantic tree structure.

[0048] Step S106: The query optimizer generates a query plan tree carrying observation hooks based on the semantic tree structure.

[0049] Here, the observation hook is the execution operator corresponding to the observation node. The observation hook is used to trigger the execution engine to generate intermediate observation snapshot data corresponding to the non-destructive observation instructions. The intermediate observation snapshot data is the data obtained by taking a snapshot of the intermediate results. In one example, the query optimizer generates a query plan tree based on the semantic nodes contained in the semantic tree structure, generates observation hooks based on the observation nodes contained in the semantic tree structure, and inserts the observation hooks into the query plan tree.

[0050] Step S108: Execute the query plan tree through the execution engine to obtain the response results corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instructions.

[0051] The execution engine includes a query executor and an enhanced debug output executor. In one example, the query engine executes the query plan tree normally, and when it reaches an execution operator associated with an observation hook, it triggers the enhanced debug output executor to execute the observation hook in parallel to obtain the corresponding intermediate observation snapshot data. The query engine continues to execute the query plan tree normally, repeating this process until the end to obtain the final response result and intermediate observation snapshot data.

[0052] The enhanced database query method supporting debug output provided in this invention introduces non-destructive observation instructions into the query statement and improves the processing of the query parser, query optimizer, and execution engine. The query parser generates a semantic tree structure containing semantic nodes and observation nodes, the query optimizer generates a query plan tree carrying observation hooks, and the execution engine executes the query plan tree to obtain the corresponding response results and intermediate observation snapshot data. Therefore, this invention can effectively solve the black-box debugging problem of traditional CTE while maintaining transaction consistency and execution efficiency.

[0053] To facilitate understanding, this embodiment of the invention first explains the database architecture, including a query processor, an execution engine, and a storage engine. The query processor is divided into a query parser and a query optimizer. The query parser performs lexical analysis, syntax analysis, and semantic analysis to generate a semantic structure tree. The query optimizer converts the semantic structure tree into a query plan tree. The execution engine includes a query executor and an enhanced debug output executor. The query executor, also known as the regular executor, executes the regular execution operators in the query plan tree. The enhanced debug output executor is a newly added executor that executes in parallel with the query executor. Specifically, when the execution operator currently being executed by the query executor is associated with an observation hook, the enhanced debug output executor executes that observation hook in parallel to generate corresponding intermediate observation snapshot data.

[0054] Compared to traditional databases, the dynamic observation point mechanism proposed in this embodiment of the invention is implemented based on a three-layer architecture model: (1) Syntax layer: Extend SQL syntax to introduce the OBSERVE instruction family: OBSERVE [INMEDIA|CONTRAL [n|expression]] [OUTPUT table_name]; (2) Execution layer: Inject observation hooks into the query plan tree and create lightweight data snapshots when the observation point is triggered, that is, introduce observation hooks to trigger the enhanced debug output executor, where the observation point is the node corresponding to the clause embedded with non-destructive observation instructions; Add an enhanced debug output executor, whose main operations are: data snapshot generation, version chain generator, incremental snapshot storage, etc.; (3) Storage layer: Use incremental snapshot storage to save storage space consumption.

[0055] Based on the foregoing embodiments, this invention provides a specific implementation of an enhanced database query method that supports debug output, including:

[0056] (i) Receive a query statement to be processed, wherein at least one clause in the query statement embeds a non-destructive observation instruction, wherein the non-destructive observation instruction is a single observation instruction or a recursive observation instruction. For ease of understanding, embodiments of the present invention provide examples of single observation instructions and recursive observation instructions respectively:

[0057] Example 1, Single observation command (OBSERVE INMEDIA command):

[0058] This allows inserting single-observation instructions into CTE queries to generate intermediate observation snapshot data, enabling the viewing of intermediate results without compromising transaction consistency. The OBSERVE INMEDIA instruction is introduced into the query statement as a non-destructive observation point marker.

[0059] like:

[0060] WITH RECURSIVE

[0061] Q1(id) as (select 1 union all select id+1 from Q1 where id<100),

[0062] Q2(id) as (select from (

[0063] With RECURSIVE

[0064] Q3(id) as (select from Q1 union all select id+1 from Q3 where id<200) observe inmedia output Q3_temp_data

[0065] Select from Q3 ) )

[0068] Select from Q1 union all select from Q2;

[0069] / / Observe the intermediate result set Q3

[0070] The above query statement includes multiple recursive queries (Q1, Q2, and Q3), which are nested together. Q1 is a recursive query that starts with an initial value of 1 and increments by 1 each time until `id` reaches 100. Q3 is another recursive query that generates a result set based on Q1 and continues to increment `id` until it reaches 200. Q3 also includes an `OBSERVEINMEDIA` instruction to observe intermediate results. Q2 is a wrapper around Q3. Finally, the main query merges the results of Q1 and Q2 to form the final response.

[0071] Example 2: Recursive observation command (OBSERVE CONTRAL command):

[0072] It provides control functions such as recursion depth limits and recursion pause. It allows the generation of intermediate recursion result snapshot tables during the recursion process, enabling the viewing of intermediate results without compromising transaction consistency. The OBSERVE CONTRAL directive is introduced into the SQL syntax as a non-destructive observation point marker.

[0073] like:

[0074] WITH RECURSIVE

[0075] Q1(id) as (select 1 union all select id+1 from Q1 where id<100),

[0076] Q2(id) as (select from (

[0077] With RECURSIVE

[0078] Q3(id) as (select from Q1 union all select id+1 from Q3 where id<200) observe control 10 output Q3_temp_data

[0079] Select from Q3 ) )

[0082] Select from Q1 union all select from Q2;

[0083] / / Observe the results of Q3 iteration 10 times.

[0084] The above query statement includes multiple recursive queries (Q1, Q2, and Q3). The difference from Example 1 is that the OBSERVE CONTRAL instruction embedded in Example 2 provides control functions such as recursion depth limit and recursion pause, and can record or output intermediate results at the 10th recursion.

[0085] Traditional solutions require splitting the CTE into multiple temporary tables and rewriting the query. In this embodiment of the invention, only an OBSERVE INMEDIA or OBSERVE CONTRAL instruction needs to be embedded in the query statement to maintain the integrity of the original query structure. This ensures that the correctness of the final response result is not affected when outputting intermediate results, and helps SQL writers maintain SQL use cases.

[0086] (ii) The query parser performs lexical, syntactic, and semantic analysis on the query statement to construct the semantic tree structure corresponding to the query statement. The specific process is as follows:

[0087] (2.1) Lexical Analysis Layer Extension: During the lexical analysis stage, all vocabularies supported by the SQL language need to be added to form a vocabulary table. In the lexical analyzer, lexical identification is performed by recognizing keywords. Specifically, keyword recognition is added to the lexical analyzer: OBSERVE, IMNEDIA, CONTRAL; and a new lexical table extension is added.

[0088] (2.2) Expansion of the syntax parsing layer: In the syntax parsing stage, the syntax analysis of lexical combinations is performed, and the corresponding syntax is formed according to the rules of syntax (i.e., the coherence of word combinations). Here, since new syntax rules have been added, the syntax part needs to be expanded. Specifically, the CTE definition rules are expanded in the syntax parser, an OBSERVE instruction processing module is added, and the observation point information is stored in the corresponding parser structure.

[0089] (2.3) Semantic Parsing Extension: In the semantic stage, it is necessary to specifically identify whether the semantics of each grammar are correct. Since this embodiment of the invention adds this function, it is necessary to check and define the meaning and operation of this part of the grammar based on the grammar itself. In the semantic stage, the syntax tree structure is transformed into a semantic tree structure. Due to the addition of grammatical functions, these syntax tree nodes need to be converted into semantic tree nodes. Specifically, observation nodes are added in the semantic analysis stage, thereby converting non-destructive observation instructions into observation nodes in the semantic tree structure.

[0090] (iii) Generate a query plan tree with observation hooks based on the semantic tree structure through the query optimizer.

[0091] In this embodiment of the invention, observation operators are seamlessly integrated into the query plan tree, i.e., observation hooks are inserted. Considering that traditional executor operators execute one operator at a time before moving on to the next in the tree, this embodiment, while introducing a new operator, does not insert it as in the traditional way, but rather acts as a hook, providing an entry point without affecting the execution of the original execution plan. Therefore, it is named an observation hook. When an observation hook is triggered, it is introduced into the enhanced debug output executor, executing in parallel with the original execution plan. An original execution plan can have multiple observation hooks, which are added in the query optimizer. Each observation hook is the execution of a parallel execution operator, and data snapshots are marked and stored according to the dynamic data stream marking algorithm.

[0092] Based on this, the process of generating the query plan tree is as follows: The query plan tree is generated based on the semantic nodes contained in the semantic tree structure; observation hooks are generated based on the observation nodes contained in the semantic tree structure; and these observation hooks are then implanted into the query plan tree. Since the semantic tree structure includes semantic nodes, and some semantic nodes are associated with observation nodes, within the query optimizer, the observation hooks generated based on the observation nodes are associated with the execution operators generated based on the semantic nodes associated with those observation nodes, thereby implanting the observation hooks into the query plan tree. This embodiment of the invention can maintain the optimization space of the original execution plan and minimize the performance overhead caused by observation.

[0093] (iv) The query plan tree is executed through the execution engine to obtain the response results corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instructions.

[0094] For specific implementation details, please refer to [link / reference]. Figure 2 The flowchart shown illustrates an intermediate result observation process. Through the query executor, the execution operators contained in the query plan tree are executed. During the execution process, if the currently executed execution operator is associated with an observation hook, the enhanced debug output executor is triggered to execute the observation hook in parallel to obtain intermediate observation snapshot data corresponding to the non-destructive observation instruction. If the currently executed execution operator is not associated with an observation hook, the original plan is maintained until the query plan tree is completed and the response result corresponding to the query statement is obtained.

[0095] For ease of understanding, the embodiments of the present invention provide the specific observation processes for single observation commands and recursive observation commands respectively:

[0096] Scenario 1: The non-destructive observation instruction is a single-observation instruction (OBSERVE INMEDIA instruction): The enhanced debug output executor executes the single-observation instruction in parallel to take a snapshot of the intermediate results obtained by the execution operators associated with the observation hooks of the query executor, thus obtaining intermediate observation snapshot data. Continuing with Example 1 above, when the query executor executes Q3, it triggers the enhanced debug output executor to execute the observation hook corresponding to the OBSERVE INMEDIA instruction in parallel to take a snapshot of the intermediate results obtained after the recursion of Q3, thus obtaining the intermediate observation snapshot data corresponding to Q3.

[0097] Scenario 2: The non-destructive observation instruction is a recursive observation instruction (OBSERVE CONTRAL instruction): The enhanced debug output executor executes the recursive observation instruction in parallel to monitor the current recursion depth of the execution operator associated with the query executor's observation hook. When the current recursion depth reaches the preset recursion depth carried by the recursive observation instruction, a snapshot of the intermediate results at the current recursion depth is taken to obtain intermediate observation snapshot data. Continuing with Example 2 above, when the query executor executes Q3, it triggers the enhanced debug output executor to execute the observation hook corresponding to the OBSERVE CONTRAL instruction in parallel to take a snapshot of the intermediate results obtained when Q3 recurses 10 times, obtaining the intermediate observation snapshot data corresponding to Q3.

[0098] (v) Obtain intermediate observation snapshot data and record the current version number and data source information transmitted by the observation hook. Pass the data to the incremental capture algorithm for incremental capture and store it in memory. Finally, replay the intermediate observation snapshot data and its attribute information from memory and print it to the log for user viewing.

[0099] In one implementation, attribute information of intermediate observation snapshot data can be recorded, including at least a version number and data source; intermediate observation snapshot data of two adjacent observations can be determined based on the attribute information, and incremental capture can be performed on the two intermediate observation snapshot data to obtain the data difference between the two intermediate observation snapshot data; the data difference between the two intermediate observation snapshot data can be marked using a specified effect, so as to display the data difference according to the mark through the graphical user interface of the associated terminal.

[0100] In this embodiment of the invention, during CTE execution, data lineage is automatically identified, and versioned markers are established for intermediate results to track the data evolution process and record the generation path of each row of intermediate observation snapshot data. Specifically, a lightweight marker injector is designed, with the basic elements being: data snapshot, data source, and version number. In the execution engine, the query executor and the enhanced debug output executor designed in this invention run in parallel. This function inserts an observation hook into the optimizer, which triggers the enhanced debug output executor to generate a data snapshot and generate an MD5 version number, which is used as the data source for the next version (it should be noted that the first data snapshot has no data source).

[0101] In this embodiment of the invention, FastCDC (Change Data Capture) technology is used to store only the data differences between two adjacent observations, and two-color marking is used to visualize the differences. Specifically, in the enhanced debug output executor, not a complete data snapshot is stored, but the difference data between two adjacent observations (two consecutive observations include: data from two consecutive observation points, and difference data from two consecutive iterations, based on the kinship graph formed by the version source), which can effectively reduce memory consumption and improve efficiency.

[0102] Based on the foregoing embodiments, this invention provides an enhanced database query device that supports debug output. See [link to previous embodiment]. Figure 3 The diagram shows the structure of an enhanced database query device that supports debug output. The device mainly includes the following parts:

[0103] The receiving module 302 is used to receive a query statement to be processed, wherein at least one clause in the query statement is embedded with a non-destructive observation instruction;

[0104] The parsing module 304 is used to perform lexical parsing, syntax parsing and semantic parsing on the query statement through the query parser in order to construct the semantic tree structure corresponding to the query statement. The semantic tree structure contains semantic nodes corresponding to clauses, and the semantic nodes corresponding to clauses that embed non-destructive observation instructions are associated with observation nodes.

[0105] The optimization module 306 is used to generate a query plan tree carrying observation hooks based on the semantic tree structure through the query optimizer. The observation hooks are the execution operators corresponding to the observation nodes. The observation hooks are used to trigger the execution engine to generate intermediate observation snapshot data corresponding to the non-destructive observation instructions.

[0106] The execution module 308 is used to execute the query plan tree through the execution engine to obtain the response results corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instructions.

[0107] The enhanced database query device supporting debug output provided in this invention introduces non-destructive observation instructions into the query statement and improves the processing of the query parser, query optimizer, and execution engine. The query parser generates a semantic tree structure containing semantic nodes and observation nodes, the query optimizer generates a query plan tree carrying observation hooks, and the execution engine executes the query plan tree to obtain the corresponding response results and intermediate observation snapshot data. Therefore, this invention can effectively solve the black-box debugging problem of traditional CTE while maintaining transaction consistency and execution efficiency.

[0108] In one implementation, the optimization module 306 is specifically used for:

[0109] The query optimizer generates a query plan tree based on the semantic nodes contained in the semantic tree structure, generates observation hooks based on the observation nodes contained in the semantic tree structure, and then implants the observation hooks into the query plan tree.

[0110] In one implementation, the execution engine includes a query executor and an enhanced debug output executor; the execution module 308 is specifically used for:

[0111] The query executor executes the execution operators contained in the query plan tree. During execution, if the currently executing execution operator is associated with an observation hook, the enhanced debug output executor is triggered to execute the observation hook in parallel to obtain intermediate observation snapshot data corresponding to the non-destructive observation instructions, until the query plan tree is completed and the response result corresponding to the query statement is obtained.

[0112] In one implementation, the non-destructive observation command is a single observation command; the execution module 308 is specifically used for:

[0113] Enhance the debug output executor to execute single observation instructions in parallel, and take snapshots of the intermediate results obtained by the execution operators associated with the query executor's execution observation hooks to obtain intermediate observation snapshot data.

[0114] In one implementation, the non-destructive observation command is a recursive observation command; the execution module 306 is specifically used for:

[0115] Enhance the debug output executor to execute recursive observation instructions in parallel, so as to listen to the current recursion depth of the execution operator associated with the query executor's execution observation hook. When the current recursion depth reaches the preset recursion depth carried by the recursive observation instruction, take a snapshot of the intermediate results at the current recursion depth to obtain intermediate observation snapshot data.

[0116] In one implementation, the execution module 308 is specifically used for:

[0117] Record the attribute information of intermediate observation snapshot data. The attribute information should include at least the version number and the data source.

[0118] Based on attribute information, intermediate observation snapshot data between two adjacent observations are determined, and incremental capture is performed on the two intermediate observation snapshot data to obtain the data differences between the two intermediate observation snapshot data.

[0119] In one implementation, the execution module 308 is specifically used for:

[0120] The data differences between two intermediate observation snapshots are marked using specified effects, and the data differences are displayed according to the marks through the graphical user interface of the associated terminal.

[0121] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0122] This invention provides an electronic device, specifically, the electronic device includes a processor and a memory; the memory stores a computer program, which, when run by the processor, executes the method described in any of the above embodiments.

[0123] Figure 4 The present invention provides a schematic diagram of the structure of an electronic device 100, which includes a processor 40, a memory 41, a bus 42 and a communication interface 43. The processor 40, the communication interface 43 and the memory 41 are connected through the bus 42. The processor 40 is used to execute executable modules, such as computer programs, stored in the memory 41.

[0124] The memory 41 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 43 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0125] Bus 42 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0126] The memory 41 is used to store programs. After receiving an execution instruction, the processor 40 executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0127] Processor 40 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 40 or by instructions in software form. Processor 40 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 41. The processor 40 reads the information in memory 41 and, in conjunction with its hardware, completes the steps of the above method.

[0128] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0129] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An enhanced database query method supporting debug output, characterized in that, include: Receive a query statement to be processed, wherein at least one clause in the query statement contains a non-destructive observation instruction; The query parser performs lexical, syntactic, and semantic parsing on the query statement to construct a semantic tree structure corresponding to the query statement. The semantic tree structure contains semantic nodes corresponding to the clauses, and the semantic nodes corresponding to the clauses that embed the non-destructive observation instructions are associated with observation nodes. The query optimizer generates a query plan tree carrying observation hooks based on the semantic tree structure. The observation hooks are the execution operators corresponding to the observation nodes. The observation hooks are used to trigger the execution engine to generate intermediate observation snapshot data corresponding to the non-destructive observation instructions. The execution engine executes the query plan tree to obtain the response results corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instruction.

2. The enhanced database query method supporting debug output according to claim 1, characterized in that, The query optimizer generates a query plan tree carrying observation hooks based on the semantic tree structure, including: The query optimizer generates a query plan tree based on the semantic nodes contained in the semantic tree structure, generates observation hooks based on the observation nodes contained in the semantic tree structure, and implants the observation hooks into the query plan tree.

3. The enhanced database query method supporting debug output according to claim 1, characterized in that, The execution engine includes a query executor and an enhanced debug output executor; The execution engine executes the query plan tree to obtain the response result corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instruction, including: The query executor executes the execution operators contained in the query plan tree. During execution, if the currently executed execution operator is associated with the observation hook, the enhanced debug output executor is triggered to execute the observation hook in parallel to obtain the intermediate observation snapshot data corresponding to the non-destructive observation instruction, until the query plan tree is completed and the response result corresponding to the query statement is obtained.

4. The enhanced database query method supporting debug output according to claim 3, characterized in that, The non-destructive observation command is a single observation command; The enhanced debug output executor is triggered to execute the observation hook in parallel to obtain the intermediate observation snapshot data corresponding to the non-destructive observation command, including: The enhanced debug output executor executes the single observation instruction in parallel to take a snapshot of the intermediate results obtained by the query executor executing the execution operator associated with the observation hook, and obtain the intermediate observation snapshot data.

5. The enhanced database query method supporting debug output according to claim 3, characterized in that, The non-destructive observation command is a recursive observation command; triggering the enhanced debug output executor to execute the observation hook to obtain the intermediate observation snapshot data corresponding to the non-destructive observation command also includes: The enhanced debug output executor executes the recursive observation instruction in parallel to monitor the current recursion depth of the execution operator associated with the observation hook executed by the query executor. When the current recursion depth reaches the preset recursion depth carried by the recursive observation instruction, a snapshot is taken of the intermediate results at the current recursion depth to obtain the intermediate observation snapshot data.

6. The enhanced database query method supporting debug output according to claim 3, characterized in that, After obtaining the intermediate observation snapshot data corresponding to the non-destructive observation command, the method further includes: Record the attribute information of the intermediate observation snapshot data, the attribute information including at least the version number and data source; Based on the attribute information, the intermediate observation snapshot data of two adjacent observations are determined, and incremental capture is performed on the two intermediate observation snapshot data to obtain the data difference between the two intermediate observation snapshot data.

7. The enhanced database query method supporting debug output according to claim 6, characterized in that, The method further includes: The data differences between the two intermediate observation snapshots are marked using specified effects, and the data differences are displayed according to the markings through the graphical user interface of the associated terminal.

8. An enhanced database query device supporting debug output, characterized in that, include: A receiving module is used to receive a query statement to be processed, wherein at least one clause in the query statement is embedded with a non-destructive observation instruction; The parsing module is used to perform lexical parsing, syntax parsing and semantic parsing on the query statement through the query parser, so as to construct a semantic tree structure corresponding to the query statement. The semantic tree structure contains semantic nodes corresponding to the clauses, and the semantic nodes corresponding to the clauses that embed the non-destructive observation instructions are associated with observation nodes. The optimization module is used to generate a query plan tree carrying observation hooks based on the semantic tree structure through the query optimizer. The observation hooks are the execution operators corresponding to the observation nodes. The observation hooks are used to trigger the execution engine to generate intermediate observation snapshot data corresponding to the non-destructive observation instructions. The execution module is used to execute the query plan tree through the execution engine to obtain the response result corresponding to the query statement and the intermediate observation snapshot data corresponding to the non-destructive observation instruction.

9. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images