LLVM-based analytical database high-performance operation method and system

Abstract

The application relates to the technical field of computer data processing, and provides a high-performance operation method and system of an analytical database based on LLVM, which solves the technical problems of low hardware resource utilization and limited operation performance. The method comprises the following steps: obtaining a structured query statement sent by a computing device, and collecting pipeline pause data and cache miss rate of the computing device during operation; performing syntax and semantic analysis on the structured query statement, and calculating a load pressure value representing the current hardware resource occupation degree; determining a segmentation threshold value according to the load pressure value, and obtaining a plurality of fusion operator subgraphs; in the process of generating an intermediate representation based on an LLVM compiler architecture, an LLVM intermediate representation sequence is generated; the LLVM intermediate representation sequence is compiled by using a compiler of an LLVM back end, target machine instructions are generated, and the target machine instructions are loaded to the computing device for operation processing, so that the operation efficiency of the analytical database and the hardware resource utilization can be improved.

Description

Technical Field

[0001] This application relates to the field of computer data processing technology, and in particular to a high-performance computing method and system for analytical databases based on LLVM. Background Technology

[0002] With the explosive growth in demand for big data analytics, analytical databases based on the LLVM compiler architecture have become an important direction in the field of high-performance computing. Just-in-time (JIT) compilation technology, which directly transforms structured query statements into efficient machine instructions, has broad application prospects in business intelligence and real-time decision-making systems.

[0003] Existing LLVM-based analytical database operations typically employ static rules or fixed cost models to compile and optimize query plans, directly generating intermediate representation code. These methods primarily rely on compiler-wide backend optimization strategies, usually fusing operators based on the logical structure of the query statement itself to reduce function call overhead and generate target code.

[0004] However, this traditional static compilation strategy ignores the real-time hardware state of the computing device during runtime, making it difficult to perceive the current pipeline pressure and cache usage. This results in the generated machine instruction sequence failing to dynamically adjust operator granularity or instruction issue order according to the actual hardware load, making critical computing tasks prone to blocking due to resource contention or waiting for data, thus failing to fully utilize hardware performance. Therefore, existing technologies suffer from the technical problem of compiled instruction sequences failing to adapt to dynamic hardware loads, leading to low hardware resource utilization and limited computing performance. Summary of the Invention

[0005] The purpose of this application is to provide a high-performance computing method and system for analytical databases based on LLVM, in order to solve the technical problem in the prior art where the compiled instruction sequence cannot adapt to dynamic hardware load, resulting in low hardware resource utilization and limited computing performance.

[0006] Firstly, this application provides a high-performance computing method for analytical databases based on LLVM, including:

[0007] It acquires structured query statements sent by computing devices to the analytical database, and collects pipeline pause data and cache miss rate during the operation of the computing devices;

[0008] The structured query statement is parsed in terms of syntax and semantics to obtain operator dependencies. Based on the operator dependencies, an operator topology graph is constructed. Based on pipeline pause data and cache miss rate, a load pressure value representing the current hardware resource utilization level is calculated.

[0009] The segmentation threshold is determined based on the load pressure value, and the operator topology graph is divided into multiple fusion operator subgraphs that adapt to the current hardware resource state through graph segmentation clustering based on the segmentation threshold.

[0010] In the process of generating intermediate representations based on the LLVM compiler architecture, each fusion operator subgraph is mapped to a computation function that includes at least one basic block. With the goal of satisfying the preset instruction delay constraints between instructions with data dependencies, the instruction issuance order of each basic block in the computation function is iteratively rearranged by simulated annealing to generate the LLVM intermediate representation sequence.

[0011] The LLVM backend compiler is used to compile the LLVM intermediate representation sequence to generate target machine instructions, which are then loaded into the computing device for computation.

[0012] Optionally, the structured query statement is parsed syntactically and semantically to obtain operator dependencies, including:

[0013] By performing character stream analysis on structured query statements to extract keywords and syntax identifiers, a token sequence is obtained. Then, recursive descent analysis is performed on the token sequence to generate an abstract syntax tree that includes various types of tree nodes.

[0014] Each tree node in the abstract syntax tree is converted into a logical operator according to the preset conversion rules;

[0015] Identify target tree nodes with logical judgment conditions in the abstract syntax tree, and based on the logical judgment conditions, parse the data flow between different logical operators to establish dependency edges between two logical operators with data flow, and extract operator dependencies used to limit the execution order from each dependency edge.

[0016] Optionally, an operator topology graph is constructed based on operator dependencies, and a load pressure value representing the current hardware resource utilization is calculated based on pipeline stall data and cache miss rate, including:

[0017] The logical operators corresponding to the operator dependencies are used as graph nodes, and the dependency edges corresponding to the operator dependencies are used as directed edges to construct an operator topology graph.

[0018] Based on preset pause weighting coefficients and preset cache weighting coefficients, the pipeline pause data and cache miss rate are weighted and summed to obtain a load pressure value representing the current hardware resource utilization level.

[0019] Optionally, a segmentation threshold is determined based on the load pressure value, and the operator topology graph is divided into multiple fusion operator subgraphs adapted to the current hardware resource state through graph segmentation clustering based on the segmentation threshold, including:

[0020] Based on the preset threshold comparison table, determine the upper limit of the number of nodes that matches the load pressure value, and use the upper limit of the number of nodes as the segmentation threshold.

[0021] Obtain the data communication volume transmitted from the starting graph node to the ending graph node of each directed edge in the operator topology graph, and calculate the connection weight of each directed edge in the operator topology graph based on the data communication volume.

[0022] The operator topology graph is divided into multiple fusion operator subgraphs adapted to the current hardware resource state by graph segmentation clustering. The constraint condition for the segmentation is to minimize the sum of the connection weights of the cut directed edges, and the number of graph nodes included in each fusion operator subgraph is less than or equal to the segmentation threshold.

[0023] Optionally, during the generation of intermediate representations based on the LLVM compiler architecture, each fusion operator subgraph is mapped to a computation function that includes at least one basic block, including:

[0024] In the LLVM compiler's compilation environment, create an empty computation function corresponding to each fusion operator subgraph and an empty basic block belonging to the computation function;

[0025] The computational logic corresponding to each node in each fusion operator subgraph is analyzed to determine the arithmetic operation type, memory access type, and control flow jump type of the computational logic.

[0026] Based on the arithmetic operation type, memory access type, and control flow jump type of the operation logic, the LLVM code generation interface is called to convert the operation logic into intermediate code instructions;

[0027] Based on the topological order within each fusion operator subgraph, intermediate code instructions are inserted into the corresponding empty basic blocks to obtain the computation function corresponding to each fusion operator subgraph.

[0028] Optionally, with the goal of ensuring that the intervals between data-dependent instructions meet preset instruction latency constraints, simulated annealing is used to iteratively rearrange the instruction issuance order of each basic block in the computation function to generate an LLVM intermediate representation sequence, including:

[0029] Extract the current instruction sequence from the basic block, traverse the target intermediate code instructions in the current instruction sequence, and calculate the instruction distance for each pair of target intermediate code instructions. The target intermediate code instructions are each pair of intermediate code instructions in the current instruction sequence that have data dependencies.

[0030] Calculate the difference between the preset instruction delay constraint and the distance to each instruction, and sum the differences that are greater than zero to obtain the cost function value of the current instruction sequence;

[0031] Under the premise of meeting the preset data dependency constraints, a preliminary instruction sequence is generated by randomly swapping the positions of any two intermediate code instructions in the current instruction sequence through simulated annealing, and the target cost function value of the preliminary instruction sequence is calculated.

[0032] Based on the Metropolis criterion, if the target cost function value is less than the cost function value or the probability acceptance condition of simulated annealing is met, the initial instruction sequence is updated to the current instruction sequence and the target cost function value is updated to the cost function value, until the number of update iterations reaches the preset value, and the target instruction sequence of each basic block is obtained.

[0033] Based on the control flow layout order within the computation function, the target instruction sequences of all basic blocks in the computation function are concatenated to obtain the LLVM intermediate representation sequence.

[0034] Optionally, the LLVM backend compiler is used to compile the LLVM intermediate representation sequence to generate target machine instructions, and the target machine instructions are loaded into the computing device for computation, including:

[0035] The LLVM intermediate representation sequence is converted into target assembly code using the instruction selector in the compiler of the LLVM backend;

[0036] The assembler in the compiler is used to encode the target assembly code into binary to generate the target machine instructions.

[0037] The target machine instructions are written into the executable memory area of ​​the computing device, the starting address of the executable memory area is obtained, and the value of the program counter in the computing device is updated to the starting address, so as to control the CPU to read and execute the target machine instructions.

[0038] Secondly, this application provides a high-performance computing system for analytical databases based on LLVM, including:

[0039] The acquisition module is used to acquire structured query statements sent by computing devices to the analytical database, and to collect pipeline pause data and cache miss rate during the operation of the computing devices.

[0040] The parsing module is used to perform syntactic and semantic parsing on structured query statements, obtain operator dependencies, construct an operator topology graph based on operator dependencies, and calculate the load pressure value representing the current hardware resource utilization based on pipeline pause data and cache miss rate.

[0041] The determination module is used to determine the segmentation threshold based on the load pressure value, and divide the operator topology graph into multiple fusion operator subgraphs adapted to the current hardware resource state through graph segmentation clustering based on the segmentation threshold;

[0042] The iterative module is used to map each fusion operator subgraph to a computation function that includes at least one basic block during the generation of intermediate representations based on the LLVM compiler architecture. With the goal of satisfying the preset instruction delay constraints between instructions with data dependencies, the module iteratively rearranges the instruction issuance order of each basic block in the computation function through simulated annealing to generate an LLVM intermediate representation sequence.

[0043] The compilation module is used to compile the LLVM intermediate representation sequence using the compiler in the LLVM backend, generate target machine instructions, and load the target machine instructions into the computing device for computation.

[0044] Thirdly, this application provides an electronic device, comprising:

[0045] Memory, used to store computer programs;

[0046] A processor, used to execute computer programs, implements the steps of the LLVM-based high-performance computing method for analytical databases as described in the first aspect above.

[0047] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the LLVM-based high-performance computing method for analytical databases described in the first aspect above.

[0048] The beneficial effects of this application are:

[0049] The high-performance analytical database computing method based on LLVM provided in this application overcomes the shortcomings of existing technologies that rely solely on static cost models and cannot perceive runtime hardware bottlenecks by collecting pipeline stall data and cache miss rates. This ensures that subsequent optimization strategies are based on actual resource consumption, avoiding the blind generation of resource-intensive code under high hardware load. It also solves the problem that static compilation strategies cannot flexibly adjust operator granularity according to hardware load, leading to critical task blocking or resource contention. Furthermore, it reduces pipeline stalls caused by waiting for data, thus significantly improving instruction pipeline execution efficiency without changing hardware conditions, thereby enhancing the computational performance of analytical databases.

[0050] Furthermore, by consulting a preset threshold lookup table, the upper limit of the number of nodes is determined as the segmentation threshold, and the data communication volume transmitted between each graph node in the operator topology graph is obtained to calculate the connection weight of the directed edges. Subsequently, this application divides the operator topology graph into multiple fusion operator subgraphs using a graph segmentation clustering algorithm. This segmentation process must simultaneously satisfy two conditions: first, the sum of the connection weights of the severed directed edges is minimized; second, the number of graph nodes included in each fusion operator subgraph does not exceed the segmentation threshold. This solves the contradiction between reducing function calls and preventing resource overflow in the static fusion strategy of the prior art, and achieves the best adaptation between the operator granularity and the current hardware resource state. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating the high-performance computing method for analytical databases based on LLVM provided in this application embodiment;

[0053] Figure 2 A flowchart illustrating a method for constructing an operator topology graph provided in an embodiment of this application;

[0054] Figure 3 A flowchart illustrating a method for obtaining an LLVM intermediate representation sequence, provided in an embodiment of this application;

[0055] Figure 4 A schematic diagram of the structure of a high-performance analytical database computing system based on LLVM provided in an embodiment of this application;

[0056] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0057] The existing LLVM-based analytical database operation methods use a static compilation strategy, which ignores the real-time hardware status of the computing device. This makes it difficult to perceive pipeline pressure and cache usage, resulting in the inability to dynamically adjust the operator granularity or instruction issuance order of the generated machine instruction sequence. This leads to problems such as critical task blocking, low hardware resource utilization, and limited computing performance.

[0058] This application quantifies the current hardware load by collecting pipeline pause data and cache miss rates of computing devices in real time. Secondly, it introduces a graph segmentation and clustering algorithm to dynamically adjust the granularity of operator fusion according to the load pressure, balancing function call overhead and hardware resource capacity on a macro level. Simultaneously, in the micro-level intermediate representation generation stage, it uses a simulated annealing algorithm to iteratively rearrange the instruction issuance order to meet instruction latency constraints and hide pauses caused by data dependencies, thereby ultimately generating target machine instructions that are highly adapted to the current hardware state, solving the problem that static compilation cannot handle dynamic loads.

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The core of this application is to provide a high-performance computing method for analytical databases based on LLVM. A flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0061] Step 101: Obtain the structured query statements sent by the computing device to the analytical database, and collect pipeline pause data and cache miss rate during the operation of the computing device.

[0062] In this step, an analytical database refers to database management software specifically designed for handling large-scale data analysis tasks. It typically employs columnar storage and is capable of performing complex aggregations and statistical queries on massive amounts of data. The computing device refers to the physical server or virtualized node running the analytical database, which possesses a processor, memory, and storage units, responsible for executing specific computational instructions. A structured query statement refers to text instructions conforming to the SQL standard, used to describe data operation requests to the database, typically including keywords such as SELECT, FROM, WHERE, and GROUP BY. Pipeline stall data refers to the statistical value of execution cycles paused during CPU pipeline execution due to resource conflicts, data dependencies, or branch prediction failures; this can usually be obtained through hardware performance counters. Cache miss rate refers to the proportion of times the processor accesses the cache when the required data is not in the cache and must be read from memory, out of the total number of accesses; it is a key indicator for measuring memory access efficiency.

[0063] In the specific implementation process, firstly, the analytical database receives structured query statements sent through the computing device. This process is typically implemented by the database connection management module listening to the client request port. When an SQL request is received, it is read into a memory buffer as input to be processed. Simultaneously, to monitor the current hardware operating status of the computing device, it is necessary to synchronously collect pipeline pause data and cache miss rates during operation. Specifically, this can be achieved by calling the performance monitoring interface provided by the operating system kernel to read the relevant register values ​​in the processor's built-in hardware performance counters.

[0064] For example, the obtained structured query statement could be: SELECT (c1)+1 FROM t1 WHERE c2>10. When collecting hardware data, performance counters can be periodically sampled to obtain the total number of instruction cycle pauses and cache misses within a past time window, as well as the total number of instruction cycles and total accesses. Then, the total number of pauses is divided by the total number of instruction cycles to obtain the current pipeline pause ratio as pipeline pause data, and the total number of cache misses is divided by the total number of accesses to obtain the cache miss rate. Assume that at a certain sampling time, the collected pipeline pause ratio is 0.3, meaning 30% of the cycles are in a paused state, and the collected... The cache miss rate is 0.2%, meaning that 20% of memory accesses miss the cache.

[0065] Step 102: Perform syntax and semantic parsing on the structured query statement to obtain operator dependencies, construct an operator topology graph based on the operator dependencies, and calculate the load pressure value representing the current hardware resource utilization level based on pipeline pause data and cache miss rate.

[0066] In this step, operator dependencies refer to the constraints between different logical operation steps in the query execution plan regarding data production and consumption; for example, a table must be scanned before data can be filtered. An operator topology graph is a directed acyclic graph that graphically represents operators and their dependencies, where nodes represent operators and edges represent data flow. Current hardware resource utilization is a quantitative indicator that comprehensively reflects the busy status of computing and storage units when executing the current task. Load pressure value is a numerical value calculated using a specific algorithm based on collected hardware monitoring data, used to assess the current hardware bottleneck.

[0067] like Figure 2 As shown, Figure 2This is a flowchart illustrating a method for constructing an operator topology graph provided in an embodiment of this application. The operator topology graph shows a directed acyclic graph formed after parsing an SQL statement. The nodes Scan, Filter, and Project represent the scanning, filtering, and projection operators, respectively. The directed edge Scan->Filter->Project clearly indicates the data dependencies and the predecessor-to-successor execution order that must be followed.

[0068] Step 201: Extract keywords and syntax identifiers by performing character stream analysis on the structured query statement to obtain a token sequence, and perform recursive descent analysis on the token sequence to generate an abstract syntax tree including various types of tree nodes.

[0069] In this step, character stream analysis refers to the lexical analysis process, which reads each character of the SQL statement and identifies it as a meaningful lexical. Keywords are reserved words in the SQL language with fixed meanings, such as SELECT and WHERE. Syntax tokens are user-defined table names, column names, or operators. Token sequences are ordered lists of lexical terms and their attributes, often called token streams. Recursive descent parsing is a top-down parsing method that matches grammatical rules through a set of recursive functions that call each other. An abstract syntax tree (AST) is an intermediate form that represents the grammatical structure of source code in a tree-like structure, where tree nodes represent elements in the grammatical structure, such as expressions and statements.

[0070] In this embodiment, a lexical scanner is first used to read the string of a structured query statement. The character stream is then converted into a token sequence through state machine or regular expression matching. For example, for the structured query statement `SELECT c1 FROM t1 WHERE c2>10`, it is identified as a token sequence including elements such as `KEYWORD` (SELECT, FROM) and `ID` (c1, t1). After obtaining the token sequence, a parser is invoked for recursive descent parsing. The parser includes parsing functions for different grammatical structures. The main parsing function starts from the first token in the token sequence and calls the corresponding sub-parsing functions according to predefined SQL grammar rules. When a complete grammatical unit is matched, a corresponding tree node is created, and the sub-components of that grammatical unit are used as child nodes of the tree node.

[0071] For example, for the above statement, an abstract syntax tree is finally constructed, where the root node is SelectStmt, and its child nodes include TargetList, FromClause, and WhereClause.

[0072] Step 202: Convert each tree node in the abstract syntax tree into a logical operator according to the preset conversion rules.

[0073] In this step, the preset transformation rules refer to the logical dictionary that defines the mapping relationship between AST node types and database relational algebra operators. A logical operator is the smallest logical unit in a database query execution plan, used to describe specific data processing operations such as projection, selection, and join.

[0074] In this embodiment, the visitor pattern is used to traverse the abstract syntax tree. For each tree node encountered, its corresponding logical operator type is found according to preset transformation rules. For example, when the tree node corresponding to FromClause in the above example is visited, it is converted to the Scan operator according to the rules; when the tree node corresponding to WhereClause is visited, it is converted to the Filter operator; and when the tree node corresponding to TargetList is visited, it is converted to the Project operator. During the transformation process, the attribute information included in the tree node, such as column names, table names, filtering conditions, and arithmetic expressions, is extracted and configured into the generated logical operator object, thereby completing the transformation from syntactic representation to algebraic representation.

[0075] Step 203: Identify the target tree nodes with logical judgment conditions in the abstract syntax tree, and based on the logical judgment conditions, parse the data flow between different logical operators to establish dependency edges between two logical operators with data flow, and extract the operator dependency relationship used to limit the execution order from each dependency edge.

[0076] In this step, the target tree node refers to a node in the AST that includes a WHERE clause, HAVING clause, or JOIN condition. Logical judgment conditions are Boolean expressions used to determine whether data is retained or how it is associated. Data flow refers to the path along which a data tuple is output from one operator and passed as input to another. Dependency edges are logical connections that link two operators and indicate the direction of data flow.

[0077] In this embodiment, the abstract syntax tree is first traversed to locate the target tree node with logical judgment conditions. For example, the WhereClause node is identified as the target tree node. The logical judgment condition c2>10 in this node is parsed, and it is determined that the data column c2 referenced by this condition comes from table t1, that is, from the scan operator Scan. Therefore, the scan operator Scan is determined to be the producer, the filter operator Filter is the consumer, and the data flow direction is from Scan to Filter. Similarly, the projection operator Project needs to output the calculated (c1)+1, and the data flow direction is from Filter to Project. Based on this, a dependency relationship edge is established between the two logical operators. Finally, all established dependency relationship edges are traversed to identify the predecessor and successor operators connected by each edge, and the constraint that the predecessor must be executed before the successor is extracted as the operator dependency relationship.

[0078] Step 211: Construct an operator topology graph using the logical operators corresponding to the operator dependencies as graph nodes and the dependency edges corresponding to the operator dependencies as directed edges.

[0079] In this step, a graph node refers to a vertex in the operator topology graph, which carries the specific information of the logical operator. A directed edge is an arrow line connecting vertices in the graph, indicating the flow of data and the order of execution.

[0080] In this embodiment, an empty directed acyclic graph (DAG) object is first initialized. The set of logical operators is traversed, creating a corresponding graph node for each operator and adding it to the graph object. Next, operator dependencies are traversed. For each dependency, its source and target operators are identified, and a directed edge is added between the corresponding source and target graph nodes. For example, based on the aforementioned dependencies, the constructed operator topology graph includes three nodes: Scan, Filter, and Project, and two directed edges: Scan->Filter and Filter->Project. During the edge addition process, loop detection can also be performed to ensure that the constructed graph structure is a directed acyclic graph, thus forming a complete operator topology graph.

[0081] Step 212: Based on the preset pause weighting coefficient and the preset cache weighting coefficient, the pipeline pause data and cache miss rate are weighted and summed to obtain the load pressure value representing the current hardware resource utilization level.

[0082] In this step, the pause weighting factor is a pre-defined numerical factor used to measure the impact of pipeline pauses on overall performance. The cache weighting factor is a pre-defined numerical factor used to measure the impact of cache misses on overall performance.

[0083] In this embodiment, pipeline pause data and cache miss rate are obtained. Preset pause weighting coefficients and preset cache weighting coefficients stored in the configuration system are read. These two coefficients are typically set based on historical performance test data or expert experience, and their sum is usually 1. The load stress value is calculated using a linear weighting formula. The calculation formula is: ,in, Indicates the load pressure value. This represents the pause weighting coefficient. This indicates data indicating a production line pause. Indicates the cache weighting factor. This represents the cache miss rate. The final value calculated using this formula is the load stress value, used to quantify the current strain on hardware resources.

[0084] For example, suppose we collect data on pipeline stalls. The cache miss rate is 0.3. The preset pause weighting coefficient is 0.2. The cache weighting factor is 0.5. If the value is 0.5, then the calculated load pressure value is... for .

[0085] Step 103: Determine the segmentation threshold based on the load pressure value, and divide the operator topology graph into multiple fusion operator subgraphs that adapt to the current hardware resource status through graph segmentation clustering based on the segmentation threshold.

[0086] In this step, the segmentation threshold refers to the upper limit of the maximum number of operator nodes or complexity allowed in a fusion compilation unit. Graph segmentation clustering refers to the process of using graph theory algorithms to cut a large graph into several disjoint subgraphs. Current hardware resource status refers to the hardware availability level reflected by load pressure values, such as idle, moderate, or overloaded. A fusion operator subgraph refers to the set of subgraphs formed after partitioning, which internally contain one or more closely related operators.

[0087] Step 301: Determine the upper limit of the number of nodes that matches the load pressure value according to the preset threshold comparison table, and use the upper limit of the number of nodes as the segmentation threshold.

[0088] In this step, the threshold lookup table refers to a pre-stored data structure that establishes a mapping relationship between load pressure ranges and operator node number limits, used to guide adjustments to the compilation granularity. The node number limit refers to the maximum number of logical operators allowed in a single fused operator subgraph, and this value directly determines the size of the generated machine code block.

[0089] In this embodiment, the load pressure value is first read. A preset threshold lookup table is traversed, and the load pressure value is matched with the pressure range in the table. The lookup table follows an inverse proportional relationship: the higher the load pressure value, the lower the upper limit of the corresponding number of nodes, to reduce the complexity of the compilation unit and alleviate pipeline pressure; the lower the load pressure value, the higher the upper limit of the corresponding number of nodes, to increase the degree of operator fusion and utilize register communication. As shown in Table 1:

[0090] Table 1 Preset Threshold Comparison Table

[0091]

[0092] Finally, the determined upper limit of the number of nodes is assigned to the segmentation threshold as a hard constraint for subsequent graph segmentation steps. For example, if the load pressure value calculated in the previous step is 0.25, according to Table 1, this value falls within... Within the interval, the maximum number of matched nodes is 10, therefore the segmentation threshold is set to 10. In another embodiment's high-load scenario, if the load pressure value is 0.85, falling within... If the interval is defined, the segmentation threshold is strictly limited to 2.

[0093] Step 302: Obtain the data communication volume transmitted from the starting graph node to the ending graph node of each directed edge in the operator topology graph, and calculate the connection weight of each directed edge in the operator topology graph based on the data communication volume.

[0094] In this step, data traffic refers to the total size or number of data tuples expected to flow from the upstream operator to the downstream operator during query execution, typically estimated based on database statistics. Connection weight is a value used to measure the performance overhead of cutting a directed edge; a larger weight indicates a higher memory read / write cost due to cutting the edge.

[0095] In this embodiment, for each directed edge in the operator topology graph, its starting and ending graph nodes are identified. The database optimizer's statistics module is invoked to obtain the estimated number of output rows for the operator corresponding to the starting graph node and the average width of the data rows. The data communication volume of the directed edge is calculated by multiplying the number of rows by the average width. Subsequently, the data communication volume is converted into connection weights. This can be done by directly using the communication volume value as the weight, or by mapping it using a normalization formula. The calculation formula is as follows: ,in, Indicates connection weights. This indicates the estimated number of data rows. This represents the average byte size of each row of data. Through this calculation, each edge in the topology graph is assigned a weight value representing the data transmission cost.

[0096] For example, in the operator topology graph, there exists a directed edge from Scan to Filter. Based on statistical information, Scan is expected to output 1000 rows of data, each row being 100 bytes wide. Calculate the connection weight of this edge. The other directed edge from Filter to Project is expected to output only 10 rows of data since most of the data is filtered. Therefore, calculate the connection weight of this edge. .

[0097] Step 303: Divide the operator topology graph into multiple fusion operator subgraphs adapted to the current hardware resource state through graph segmentation clustering. The constraint condition for the division is to minimize the sum of the connection weights of the cut directed edges, and the number of graph nodes included in each fusion operator subgraph is less than or equal to the segmentation threshold.

[0098] In this step, the cut directed edges refer to the edges whose start and end nodes are assigned to different subgraphs during the graph partitioning process. The data transmission represented by these edges will be carried out through memory or cache, rather than registers.

[0099] In this embodiment, the operator topology graph, the connection weight of each edge, and the segmentation threshold are used as inputs, and a graph segmentation clustering algorithm is applied. This algorithm can employ a multi-level partitioning algorithm or a greedy algorithm. Its core logic is to find a cutting scheme that minimizes the cumulative weight of all cut edges, provided that the number of nodes in each subgraph does not exceed the segmentation threshold. Specifically, the algorithm prioritizes aggregating nodes connected by high-weight edges into the same subgraph until the subgraph size reaches the threshold limit. For low-weight edges, the algorithm tends to cut them, causing the nodes they connect to belong to different subgraphs.

[0100] For example, suppose in a high-load scenario, the current segmentation threshold is 2, and the topology graph includes three nodes: Scan, Filter, and Project, with connection weights of respectively. (Scan->Filter) and (Filter->Project). The algorithm attempts to cut... The cost is 100,000; attempt to cut. The cost is 1000. Clearly, cutting... The cost is minimized. Therefore, the algorithm divides the topology graph into two fusion operator subgraphs: Subgraph A includes Scan and Filter, with 2 nodes, satisfying the threshold; Subgraph B includes Project, with 1 node, also satisfying the threshold. For the low-load scenario with a load pressure value of 0.25, since the threshold is 10, which is greater than the total number of nodes 3, the algorithm chooses not to cut any edges and divides the three operators into the same fusion operator subgraph.

[0101] Step 104: In the process of generating intermediate representations based on the LLVM compiler architecture, each fusion operator subgraph is mapped to a computation function that includes at least one basic block. With the goal of satisfying the preset instruction delay constraints between instructions with data dependencies, the instruction issuance order of each basic block in the computation function is iteratively rearranged by simulated annealing to generate the LLVM intermediate representation sequence.

[0102] In this step, the LLVM compiler architecture refers to a modular compiler infrastructure that supports multiple front-end languages ​​and back-end target architectures. Intermediate representation refers to LLVM IR, a low-level code representation independent of specific hardware. Computation functions are objects generated at the IR level, used to encapsulate specific computational logic. A basic block is a linear sequence of instructions in the code, excluding branch jumps except for entry and exit points. Data dependency means that the operands of a subsequent instruction depend on the result of a previous instruction. Preset instruction latency constraints refer to the number of clock cycles required to execute a specific instruction, such as a memory load instruction. Simulated annealing is a probabilistic global optimization algorithm used to find the optimal solution in the search space. Instruction issue order refers to the order in which instructions are arranged within a basic block. The LLVM intermediate representation sequence refers to the final optimized list of IR instructions.

[0103] Step 401: Create an empty computation function and an empty basic block belonging to each fusion operator subgraph in the LLVM compiler's compilation environment.

[0104] In this step, the compilation environment refers to a collection of runtime objects, including core components such as context, modules, and the IR builder, used to manage and build IR code. An empty computation function is a function object that has only declared a function signature but has not yet been populated with a concrete instruction body. An empty basic block belonging to a computation function is a basic block object created inside the function object, serving as a container for instructions, including an entry block and other logical blocks.

[0105] In this embodiment, multiple fusion operator subgraphs are first traversed. For each subgraph, a new function object is created in the LLVM module, and its return type and parameter list are set. Next, an initial basic block object, usually named `entry`, is created inside the function object and inserted into the function object's block list. If the operator logic is complex and involves loops or branches, empty basic blocks such as `loopBody` and `loopExit` are also created in advance to construct the basic skeleton of the computation function.

[0106] Step 402: Analyze the computational logic corresponding to each graph node in each fusion operator subgraph to determine the arithmetic operation type, memory access type, and control flow jump type of the computational logic.

[0107] In this step, arithmetic operations refer to CPU calculation operations such as addition, subtraction, multiplication, division, bitwise operations, and comparison operations. Memory access operations refer to data read and write operations. Control flow jump operations refer to operations that change the order of instruction execution, including unconditional jumps and conditional jumps.

[0108] In this embodiment, each graph node, i.e., a logic operator, in the fusion operator subgraph is traversed. The expression tree inside the operator is parsed to identify its operational nature. For example, for the conditional expression c2>10 in the filtering operator, > is identified as a comparison operation, and the reference to c2 is identified as a memory read. For the projection operator... The code identifies "+" as arithmetic addition and updates to the accumulated variable as memory writes. For connection logic between operators, such as loop structures in scanning operators, it identifies them as control flow jump types. Through this process, high-level operator logic is refined into low-level operation type descriptions, preparing for code generation.

[0109] Step 403: Based on the arithmetic operation type, memory access type, and control flow jump type of the operation logic, call the LLVM code generation interface to convert the operation logic into intermediate code instructions.

[0110] In this step, the LLVM code generation interface refers to the API methods provided by the IR builder class. Intermediate code instructions refer to the instruction objects in LLVM IR.

[0111] In this embodiment, the IR builder object provided by LLVM is used to call the corresponding APIs one by one according to the operation type. For arithmetic operations, the generation interface for addition, subtraction, etc., is called to generate calculation instructions; for memory access, the generation loading interface is called to generate data loading instructions, and the generation storage interface is called to generate data storage instructions; for control flow jumps, the generation conditional jump interface is called to generate conditional jump instructions, or the generation jump interface is called to generate unconditional jump instructions. During the generation process, type conversion and the generation of static single-assignment variables are also automatically handled to ensure that the generated intermediate code instructions conform to the LLVM IR specification.

[0112] Step 404: Based on the topological order within each fusion operator subgraph, insert intermediate code instructions into the corresponding empty basic blocks to obtain the computation function corresponding to each fusion operator subgraph.

[0113] In this step, the internal topological order refers to the order of data flow among the operator nodes within the fusion operator subgraph, ensuring that the producer logic is executed first, followed by the consumer logic.

[0114] In this embodiment, intermediate code instructions are inserted into empty basic blocks sequentially according to the operator dependencies within the subgraph. For example, the loading instruction for the scan operator is inserted first, followed by the comparison instruction for the filter operator, and finally the storage instruction for the aggregation operator. Instructions involving control flow, such as loop termination jumps, are inserted at the end of the basic block. Through this process, previously empty function objects are filled with specific IR instructions, forming a complete and logically correct computation function. However, the instruction order has not yet been optimized for hardware latency.

[0115] like Figure 3 As shown, Figure 3 This is a flowchart illustrating a method for obtaining an LLVM intermediate representation sequence, provided as an embodiment of this application.

[0116] Step 411: Extract the current instruction sequence from the basic block, traverse the target intermediate code instructions in the current instruction sequence, and calculate the instruction distance for each pair of target intermediate code instructions. The target intermediate code instructions are each pair of intermediate code instructions in the current instruction sequence that have data dependencies.

[0117] In this step, the current instruction sequence refers to the list of existing instructions in the basic block. Target intermediate code instructions refer to instruction pairs with defined and used relationships that are of primary interest during analysis. Instruction distance refers to the number of instructions in the instruction sequence that separates the defining instruction from the used instruction.

[0118] In this embodiment, for each basic block in the computation function, its instruction list is obtained. This list is traversed, and by analyzing the operand reference relationships of the instructions, all instruction pairs with data dependencies are identified. Assume that a basic block corresponding to a fusion operator subgraph contains three instructions: I1 (load variable A, delay = 3), I2 (add variable A, 1), and I3 (store variable B). The initial sequence is [I1, I2, I3]. Instruction I1 defines variable A, and instruction I2 uses variable A; therefore, I1 and I2 constitute a pair of target intermediate code instructions. The difference in position between I1 and I2 in the current sequence is calculated, which is the instruction distance. If I1 is in the first position and I2 is in the second position, the distance is 1.

[0119] Step 412: Calculate the difference between the preset instruction delay constraint and the distance to each instruction, and sum the differences that are greater than zero to obtain the cost function value of the current instruction sequence.

[0120] In this step, the preset instruction latency constraint refers to the minimum number of execution cycles or instruction intervals required to avoid pipeline stalls for a specific type of instruction. The cost function value is a numerical value used to evaluate the quality of the current instruction sequence; a higher value indicates a greater risk of pipeline stalls.

[0121] In this embodiment, the delay constraint values ​​of various instructions in the configuration table are read, for example, the delay of the load instruction is 3. For each pair of target intermediate code instructions, the delay constraint of the instruction type is subtracted from its actual instruction distance. If the difference is less than or equal to zero, it means that the distance is far enough and will not cause a pause, and the difference is recorded as 0; if the difference is greater than zero, it means that the distance is too close and will cause a pipeline pause, and the positive difference value is retained. Finally, all positive difference values ​​are summed to obtain the total cost function value, as shown in formula (1):

[0122] (1);

[0123] in, The cost function value, Preset delay for each pair of dependent instructions, This is its actual distance. For example, for the aforementioned sequence [I1,I2,I3], I2 depends on I1, and the actual distance is... Preset delay Calculate the difference If there is only this one pair of dependencies, then the current cost function value... The value is 2.

[0124] Step 413: Under the premise of meeting the preset data dependency constraints, randomly swap the positions of any two intermediate code instructions in the current instruction sequence by simulated annealing to generate a preliminary instruction sequence, and calculate the target cost function value of the preliminary instruction sequence.

[0125] In this step, pre-setting data dependency constraints means maintaining the semantic correctness of the program, that is, ensuring that all variables are defined before they are used. The initial instruction sequence refers to a temporary list of instructions generated after random swapping. The target cost function value refers to the cost score corresponding to this temporary list of instructions.

[0126] In this embodiment, a simulated annealing process is initiated. Two instructions are randomly selected from the current instruction sequence. Before attempting to swap their positions, it is checked whether the swap would violate data dependency constraints. If it violates the constraints, the swap is abandoned; if it meets the constraints, the swap is performed, generating a preliminary instruction sequence. Next, the computational logic of steps 411 and 412 is repeated to calculate the cost of this preliminary instruction sequence, i.e., the target cost function value. This step explores possible better instruction arrangements through random perturbation.

[0127] For example, simulated annealing attempts to move the independent I3 between I1 and I2, generating an initial sequence [I1, I3, I2]. At this point, I2 still depends on I1, but is much further away. It becomes 2 because of the intermediate I3. The new cost is calculated as follows: Therefore, the objective cost function value is 1.

[0128] Step 414: Based on the Metropolis criterion, if the target cost function value is less than the cost function value or the probability acceptance condition of simulated annealing is met, the initial instruction sequence is updated to the current instruction sequence and the target cost function value is updated to the cost function value, until the number of update iterations reaches the preset value, and the target instruction sequence of each basic block is obtained.

[0129] In this step, the Metropolis criterion refers to the core acceptance criterion of the simulated annealing algorithm, which allows for the acceptance of a poor solution with a certain probability in order to escape local optima. The probability acceptance condition refers to... ,in For the price difference, The current temperature. The target instruction sequence refers to the final instruction arrangement determined after multiple iterations and optimizations.

[0130] In this embodiment, the target cost function value is compared with the current cost function value. If the target value is smaller, i.e., better, the exchange is accepted directly, and the current sequence and cost value are updated. If the target value is larger, i.e. worse, the acceptance probability is calculated. ,in The target cost function value, This is the current cost function value. Given the current temperature, generate a random number between 0 and 1. If the random number is less than... If the variation is too great, then a trade-off with the worsening condition is reluctantly accepted, in the hope of finding the global optimum later. As the number of iterations increases, the temperature... As the number of iterations gradually decreases, the probability of accepting the difference decreases, and the algorithm gradually converges. When the number of iterations reaches a preset value, the iteration stops, and the current optimal instruction sequence is output as the target instruction sequence for the basic block. For example, the target cost function value of the aforementioned new sequence [I1,I3,I2] is 1, which is less than the current value of 2. Therefore, the exchange is accepted directly, and the current sequence is updated to [I1,I3,I2].

[0131] Step 415: Based on the control flow layout order inside the computation function, concatenate the target instruction sequences of all basic blocks in the computation function to obtain the LLVM intermediate representation sequence.

[0132] In this step, the control flow layout order refers to the physical storage order of basic blocks in memory, which is usually determined by the traversal order of the control flow graph, such as depth-first traversal order.

[0133] In this embodiment, the optimized target instruction sequence of each basic block is extracted sequentially according to the predetermined block layout order within the computation function, such as [entry block, loop body block, loop exit block]. These sequences are then concatenated to form a complete instruction list. This list is the final generated LLVM intermediate representation sequence, which retains the original program logic while microscopically optimizing the instruction order to adapt to hardware latency. The final generated LLVM intermediate representation sequence may take the form of [%val=load...;%res=add %val...;store %res...].

[0134] Step 105: Use the LLVM backend compiler to compile the LLVM intermediate representation sequence, generate target machine instructions, and load the target machine instructions into the computing device for computation.

[0135] In this step, the LLVM backend refers to the component in the LLVM architecture responsible for translating the intermediate representation into assembly code specific to the hardware architecture. The compiler here specifically refers to the Just-In-Time (JIT) compiler. Target machine instructions refer to binary machine code that the CPU can directly execute.

[0136] Step 501: Use the instruction selector in the LLVM backend compiler to convert the LLVM intermediate representation sequence into target assembly code.

[0137] In this step, the instruction selector refers to the component in the LLVM backend responsible for mapping platform-independent IR instructions to hardware architecture-specific assembly instructions, typically implemented based on a directed acyclic graph pattern matching algorithm. The target assembly code refers to a textual or structured low-level instruction representation specific to a particular processor architecture, which has not yet been converted to binary.

[0138] In this embodiment, the target machine object for LLVM is first initialized, specifying the target architecture and CPU characteristics. Next, the LLVM intermediate representation sequence is input to the instruction selector. The instruction selector constructs the IR sequence into a directed acyclic graph and uses a tree pattern matching algorithm to find the best-matching machine instruction in a predefined instruction pattern library. For example, for an optimized LLVM intermediate representation sequence including load, addition, and store logic, the instruction selector maps it to the following target assembly code: [mov rax,[rdi];add rax,1;mov [rsi],rax]. During this process, preparatory work for instruction scheduling and register allocation is also performed. Finally, a target assembly code tightly coupled with the hardware architecture is output.

[0139] Step 502: Use the assembler in the compiler to encode the target assembly code into binary to generate the target machine instructions.

[0140] In this step, the assembler refers to the tool that converts assembly code into a binary code stream that the machine can directly execute. The target machine instructions refer to the final sequence of binary bytes that can be decoded and executed by the CPU.

[0141] In this embodiment, the machine code layer component of the LLVM backend is invoked. The assembler receives the target assembly code and performs lexical and syntactic analysis. According to the instruction set encoding specification of the target architecture, each assembly mnemonic and operand is converted into its corresponding binary opcode and operand encoding. Simultaneously, the relative offset of jump instructions is calculated, and all symbolic references are resolved. For example, the assembler encodes the aforementioned assembly code [mov rax, [rdi]; add rax, 1; mov [rsi], rax] into a hexadecimal machine instruction sequence: [48 8B 07 48 83 C0 01 48 89 06]. Finally, a continuous binary data stream, i.e., the target machine instructions, is generated.

[0142] Step 503: Write the target machine instructions into the executable memory area of ​​the computing device, obtain the starting address of the executable memory area, and update the value of the program counter in the computing device to the starting address, so as to control the CPU to read and execute the target machine instructions.

[0143] In this step, the executable memory region refers to a block of memory with executable permissions allocated in the random access memory of a computing device, used to store dynamically generated code. The starting address refers to the initial virtual address of this memory region. The program counter is the register in the CPU that stores the address of the next instruction to be executed.

[0144] In this embodiment, the operating system's memory management interface is first invoked to request a memory page with read, write, and execute permissions. The target machine instructions are then completely copied and written to this executable memory region. After writing, the starting address of this memory region is obtained, for example, 0x7F001000. Finally, through function pointer conversion or a direct jump instruction, the CPU's program counter is updated to this starting address. The CPU then fetches, decodes, and executes instructions from this address, thereby truly initiating the database operation logic optimized by all the preceding steps, completing the loading, calculation, and writing of data.

[0145] This application overcomes the shortcomings of existing technologies that rely solely on static cost models and cannot perceive runtime hardware bottlenecks by collecting pipeline pause data and cache miss rates. This ensures that subsequent optimization strategies are based on actual resource consumption, avoiding the blind generation of resource-intensive code under high hardware load. It also solves the problem that static compilation strategies cannot flexibly adjust operator granularity according to hardware load, leading to critical task blocking or resource contention. Furthermore, it reduces pipeline pauses caused by waiting for data, thus significantly improving instruction pipeline execution efficiency without changing hardware conditions. Finally, it enhances the computational performance of analytical databases.

[0146] Figure 4 This is a schematic diagram illustrating a specific implementation of the LLVM-based high-performance computing system for analytical databases provided in this application. (Refer to...) Figure 4 The system may include:

[0147] The acquisition module 21 is used to acquire the structured query statements sent by the computing device to the analytical database, and to collect pipeline pause data and cache miss rate during the operation of the computing device.

[0148] Parsing module 22 is used to perform syntax and semantic parsing on structured query statements, obtain operator dependencies, construct operator topology graphs based on operator dependencies, and calculate load pressure values ​​representing the current hardware resource utilization based on pipeline pause data and cache miss rate.

[0149] The determination module 23 is used to determine the segmentation threshold based on the load pressure value, and divide the operator topology graph into multiple fusion operator subgraphs that adapt to the current hardware resource status through graph segmentation clustering based on the segmentation threshold.

[0150] The iteration module 24 is used to map each fusion operator subgraph to a computation function that includes at least one basic block during the process of generating intermediate representations based on the LLVM compiler architecture. With the goal of satisfying the preset instruction delay constraints between instructions with data dependencies, the module iteratively rearranges the instruction issuance order of each basic block in the computation function through simulated annealing to generate an LLVM intermediate representation sequence.

[0151] The compilation module 25 is used to compile the LLVM intermediate representation sequence using the compiler of the LLVM backend, generate target machine instructions, and load the target machine instructions into the computing device for computation.

[0152] The high-performance computing system for analytical databases based on LLVM in this application is used to implement the aforementioned high-performance computing method for analytical databases based on LLVM. Therefore, the specific implementation of the high-performance computing system for analytical databases based on LLVM can be found in the embodiment section of the high-performance computing method for analytical databases based on LLVM mentioned above. The specific implementation can be referred to the description of the corresponding embodiments, and will not be repeated here.

[0153] Figure 5 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0154] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described LLVM-based high-performance computing methods for analytical databases.

[0155] The electronic device may include a processor 510 and a memory 520 storing computer program instructions.

[0156] Specifically, the processor 510 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0157] Memory 520 may include mass storage for data or instructions. For example, and not limitingly, memory 520 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 520 may include removable or non-removable (or fixed) media. Where appropriate, memory 520 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 520 is non-volatile solid-state memory.

[0158] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of this disclosure.

[0159] The processor 510 reads and executes computer program instructions stored in the memory 520 to implement any of the LLVM-based high-performance computing methods for analytical databases in the above embodiments.

[0160] In one example, the electronic device may also include a communication interface 530 and a bus 540. Wherein, such as Figure 5 As shown, the processor 510, memory 520, and communication interface 530 are connected through bus 540 and complete communication with each other.

[0161] The communication interface 530 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0162] Bus 540 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 540 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0163] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described LLVM-based high-performance computing methods for analytical databases.

[0164] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0165] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the high-performance computing method for LLVM-based analytical databases described above.

[0166] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0167] The above provides a detailed description of the high-performance computing method and system for analytical databases based on LLVM provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A high-performance computing method for analytical databases based on LLVM, characterized in that, include: The system acquires structured query statements sent by the computing device to the analytical database and collects pipeline pause data and cache miss rate during the operation of the computing device. The structured query statement is parsed in terms of syntax and semantics to obtain operator dependencies. An operator topology graph is constructed based on the operator dependencies. The load pressure value representing the current hardware resource utilization is calculated based on the pipeline pause data and the cache miss rate. The segmentation threshold is determined based on the load pressure value, and the operator topology graph is divided into multiple fusion operator subgraphs adapted to the current hardware resource state by graph segmentation clustering based on the segmentation threshold. In the process of generating intermediate representations based on the LLVM compiler architecture, each fusion operator subgraph is mapped to a computation function that includes at least one basic block. With the goal of satisfying the preset instruction delay constraints between instructions with data dependencies, the instruction issuance order of each basic block in the computation function is iteratively rearranged by simulated annealing to generate an LLVM intermediate representation sequence. The LLVM intermediate representation sequence is compiled using the LLVM backend compiler to generate target machine instructions, which are then loaded into the computing device for computation.

2. The high-performance computing method for analytical databases based on LLVM according to claim 1, characterized in that, The structured query statement is parsed syntactically and semantically to obtain operator dependencies, including: By performing character stream analysis on the structured query statement to extract keywords and syntax identifiers, a token sequence is obtained. Then, recursive descent analysis is performed on the token sequence to generate an abstract syntax tree that includes various types of tree nodes. Each tree node in the abstract syntax tree is converted into a logical operator according to the preset conversion rules; Identify target tree nodes with logical judgment conditions in the abstract syntax tree, and based on the logical judgment conditions, parse the data flow between different logical operators to establish dependency edges between two logical operators with data flow, and extract operator dependencies for limiting the execution order from each dependency edge.

3. The high-performance computing method for analytical databases based on LLVM according to claim 1, characterized in that, Based on the operator dependencies, an operator topology graph is constructed, and based on the pipeline pause data and the cache miss rate, a load pressure value representing the current hardware resource utilization level is calculated, including: Using the logical operators corresponding to the operator dependencies as graph nodes and the dependency edges corresponding to the operator dependencies as directed edges, an operator topology graph is constructed. Based on preset pause weighting coefficients and preset cache weighting coefficients, the pipeline pause data and the cache miss rate are weighted and summed to obtain a load pressure value representing the current hardware resource utilization level.

4. The high-performance computing method for analytical databases based on LLVM according to claim 1, characterized in that, A segmentation threshold is determined based on the load pressure value, and the operator topology graph is divided into multiple fusion operator subgraphs adapted to the current hardware resource state through graph segmentation clustering based on the segmentation threshold, including: According to the preset threshold lookup table, determine the upper limit of the number of nodes that matches the load pressure value, and use the upper limit of the number of nodes as the segmentation threshold; Obtain the data communication volume transmitted from the starting graph node to the ending graph node of each directed edge in the operator topology graph, and calculate the connection weight of each directed edge in the operator topology graph based on the data communication volume; The operator topology graph is divided into multiple fusion operator subgraphs adapted to the current hardware resource state by graph segmentation and clustering. The constraint condition for the division is to minimize the sum of the connection weights of the cut directed edges, and the number of graph nodes included in each fusion operator subgraph is less than or equal to the segmentation threshold.

5. The high-performance computing method for analytical databases based on LLVM according to claim 1, characterized in that, During the generation of intermediate representations based on the LLVM compiler architecture, each fusion operator subgraph is mapped to a computation function that includes at least one basic block, including: In the LLVM compiler's compilation environment, create an empty computation function corresponding to each fusion operator subgraph and an empty basic block belonging to the computation function; The computational logic corresponding to each node in each fusion operator subgraph is parsed to determine the arithmetic operation type, memory access type, and control flow jump type of the computational logic. Based on the arithmetic operation type, memory access type, and control flow jump type of the computation logic, the LLVM code generation interface is called to convert the computation logic into intermediate code instructions; Based on the topological order within each fusion operator subgraph, the intermediate code instructions are inserted into the corresponding empty basic blocks to obtain the computation function corresponding to each fusion operator subgraph.

6. The high-performance computing method for analytical databases based on LLVM according to claim 5, characterized in that, With the goal of ensuring that the intervals between data-dependent instructions meet preset instruction latency constraints, simulated annealing is used to iteratively rearrange the instruction issuance order of each basic block in the computation function to generate an LLVM intermediate representation sequence, including: Extract the current instruction sequence from the basic block, traverse the target intermediate code instructions in the current instruction sequence, and calculate the instruction distance for each pair of target intermediate code instructions, wherein the target intermediate code instructions are each pair of intermediate code instructions in the current instruction sequence that have data dependencies. Calculate the difference between the preset instruction delay constraint and the distance to each instruction, and sum the differences that are greater than zero to obtain the cost function value of the current instruction sequence; Under the premise of meeting the preset data dependency constraints, a preliminary instruction sequence is generated by randomly swapping the positions of any two intermediate code instructions in the current instruction sequence through simulated annealing, and the target cost function value of the preliminary instruction sequence is calculated. Based on the Metropolis criterion, if the target cost function value is less than the cost function value or the probability acceptance condition of the simulated annealing is met, the preliminary instruction sequence is updated to the current instruction sequence and the target cost function value is updated to the cost function value, until the number of update iterations reaches a preset value, and the target instruction sequence of each basic block is obtained. Based on the control flow layout order within the computation function, the target instruction sequence of all basic blocks in the computation function is concatenated to obtain the LLVM intermediate representation sequence.

7. The high-performance computing method for analytical databases based on LLVM according to claim 1, characterized in that, The LLVM intermediate representation sequence is compiled using the LLVM backend compiler to generate target machine instructions, and the target machine instructions are loaded into the computing device for computation, including: The LLVM intermediate representation sequence is converted into target assembly code using the instruction selector in the compiler of the LLVM backend; The target assembly code is binary encoded using the assembler in the compiler to generate target machine instructions; The target machine instruction is written into the executable memory region of the computing device, the starting address of the executable memory region is obtained, and the value of the program counter in the computing device is updated to the starting address, so as to control the CPU to read and execute the target machine instruction.

8. A high-performance computing system for analytical databases based on LLVM, characterized in that, include: The acquisition module is used to acquire structured query statements sent by the computing device to the analytical database, and to collect pipeline pause data and cache miss rate during the operation of the computing device. The parsing module is used to perform syntax and semantic parsing on the structured query statement to obtain operator dependencies, construct an operator topology graph based on the operator dependencies, and calculate a load pressure value representing the current hardware resource occupancy level based on the pipeline pause data and the cache miss rate. The determination module is used to determine a segmentation threshold based on the load pressure value, and to divide the operator topology graph into multiple fusion operator subgraphs adapted to the current hardware resource state based on the segmentation threshold through graph segmentation clustering. The iterative module is used to map each fusion operator subgraph to a computation function that includes at least one basic block during the generation of intermediate representations based on the LLVM compiler architecture. With the goal of satisfying the preset instruction delay constraints between instructions with data dependencies, the module iteratively rearranges the instruction issuance order of each basic block in the computation function through simulated annealing to generate an LLVM intermediate representation sequence. The compilation module is used to compile the LLVM intermediate representation sequence using the compiler of the LLVM backend, generate target machine instructions, and load the target machine instructions into the computing device for computation.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the LLVM-based high-performance computing method for analytical databases as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the LLVM-based high-performance computing method for analytical databases as described in any one of claims 1 to 7.

Images