A hybrid load intelligent heterogeneous computing method, system, device and medium for a CPU-GPU heterogeneous computing platform

Abstract

The application belongs to the technical field of data processing, and discloses a hybrid load intelligent heterogeneous computing method, system, device and medium for a CPU-GPU heterogeneous computing platform, which comprises the following steps: optimizing the data distribution storage and cache of the CPU and GPU ends according to the computing characteristics of the GPU and the load, and creating a multi-dimensional computing index based on the GPU; using a pre-established multi-mode operator library for the GPU to generate a computing plan based on a cost model, intelligently selecting the operator execution site of the generated computing plan according to the spatial cost and time cost of query execution, and obtaining an optimal CPU-GPU heterogeneous execution plan; dynamically scheduling the task execution of the CPU and GPU ends according to the optimal CPU-GPU heterogeneous execution plan, and obtaining a hybrid load intelligent heterogeneous computing result. The application can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to a hybrid workload intelligent heterogeneous computing method, system, device and medium for CPU-GPU heterogeneous computing platforms. Background Technology

[0002] With the development of GPU hardware technology, accelerating database performance through GPUs has become an important technical approach to improve database performance. The differences in hardware architecture between GPUs and CPUs make them more suitable for computationally intensive workloads in databases, while complex data management and processing workloads are more suitable for CPUs. Therefore, how to achieve intelligent heterogeneous management, scheduling, and execution of different types of workloads on CPU-GPU heterogeneous computing platforms is a key issue in improving the performance of mixed workloads on CPU-GPU heterogeneous computing platforms.

[0003] Current GPU databases primarily employ on-demand caching, using GPU memory as a cache for CPU data. This approach lacks in-depth storage optimization tailored to the computational characteristics of the workload, sacrificing GPU storage access efficiency. While they inherit and optimize traditional database query processing models in terms of operators and GPU computation models, the different hardware architectures of GPUs and CPUs impact algorithm efficiency. In multi-model mixed workload scenarios (such as relational models, document models, and graph models), when different models use different computation engines, mixed workload execution requires scheduling on the CPU side, resulting in significant CPU-GPU transmission latency and reduced performance. Summary of the Invention

[0004] To address the problems of existing technologies, such as difficulty in optimizing data distribution and caching for load computing characteristics, low computing efficiency, and low performance under mixed loads, the purpose of this invention is to provide a method, system, device, and medium for intelligent heterogeneous computing under mixed loads for CPU-GPU heterogeneous computing platforms, thereby improving the performance of GPU databases under mixed loads through intelligent heterogeneous computing technology.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a hybrid load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms, comprising: optimizing the data distribution storage and caching of the CPU and GPU sides according to the computing characteristics of the GPU and the load, and creating a GPU-based multi-dimensional computing index; using a pre-established multi-mode operator library for GPUs to generate a computing plan based on a cost model, and intelligently selecting the operator execution site of the generated computing plan according to the query execution space cost and time cost to obtain the optimal CPU-GPU heterogeneous execution plan; dynamically scheduling the task execution of the CPU and GPU sides according to the optimal CPU-GPU heterogeneous execution plan to obtain the hybrid load execution result.

[0006] Secondly, this invention provides a hybrid load intelligent heterogeneous computing system for CPU-GPU heterogeneous computing platforms, comprising: an adaptive data distribution and caching module configured to optimize data distribution, storage, and caching on the CPU and GPU sides according to the computing characteristics of the GPU and the load, and to create a GPU-based multi-dimensional computing index; a heterogeneous computing plan optimization module configured to generate a cost-based computing plan using a pre-established multi-mode operator library for GPUs, and to intelligently select different computing plans based on query execution space cost and time cost to obtain the optimal CPU-GPU heterogeneous execution plan; and a computing resource management and scheduling module configured to dynamically schedule task execution on the CPU and GPU sides according to the optimal CPU-GPU heterogeneous execution plan to obtain hybrid load execution results.

[0007] Thirdly, the present invention provides a computer-readable storage medium for storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any method.

[0008] Fourthly, the present invention provides a computing device comprising: one or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, the one or more programs including instructions for performing any method.

[0009] The present invention has the following advantages due to the adoption of the above technical solutions: 1. This invention creates a GPU-based multidimensional computing index for CPU-GPU heterogeneous computing platforms. It adopts a metadata and value storage model and a vector index caching technology oriented towards dimension tables and dimension levels to achieve distributed storage and caching across CPU and GPU platforms. 2. This invention creates a multi-mode operator library for GPUs, supporting multi-dimensional computation operators for relational models, document models, and graph models suitable for GPU acceleration. During the execution of computational tasks, it generates optimized fusion kernel execution plans based on storage space costs and execution costs, and selects optimized heterogeneous computation plans through a heterogeneous execution plan intelligent selector. 3. This invention generates optimized execution plans for mixed workloads on the CPU and GPU sides and different query processing engines based on different loads, GPU computing resources, and heterogeneous computing plans, effectively improving the performance of intelligent heterogeneous computing for mixed workloads.

[0010] Therefore, this invention can be widely applied in the field of data processing technology. Attached Figure Description

[0011] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings: Figure 1 This is a flowchart of a hybrid load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the storage model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of distributed storage and caching provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a computing plan for a GPU platform provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the execution of the extended graph operator provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the execution plan for the extended graph operator provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the query fusion technology provided in an embodiment of the present invention; Figure 8 A schematic diagram of the hybrid load optimization technology provided in an embodiment of the present invention; Figure 9 This is a schematic diagram illustrating the aggregation calculation of a column in a fact table for a query provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of a query update provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0013] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0014] In some embodiments of this invention, a hybrid load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms is provided. First, the data distribution, storage, and caching on the CPU and GPU sides are optimized based on the computational characteristics of the GPU and the load. Metadata representing semantics (such as compressed dictionary tables, dimension tables, and dimension hierarchies) is stored on the CPU side for query parsing processing, while numerical data for computation is stored in GPU memory to improve computational performance, thus achieving data distribution and caching oriented towards the computational characteristics of the load. Second, a GPU operator library for different models is provided to support computation on both the CPU and GPU sides. The operator execution site is determined based on the space and time costs of query execution, providing an optimized CPU-GPU heterogeneous execution plan. Finally, during distributed storage, data is horizontally or vertically sharded according to GPU storage resources to fully utilize GPU storage resources. Storage model optimization and transformation are performed for different models to improve GPU computational efficiency. Task execution on the CPU and GPU sides is dynamically scheduled according to the generated heterogeneous computing execution plan. On the GPU side, query fusion technology is used to optimize the hybrid load execution performance. This invention can optimize the management and execution of hybrid loads on CPU-GPU heterogeneous computing platforms, improving the storage efficiency and query performance of the GPU platform.

[0015] Correspondingly, in other embodiments of the present invention, a hybrid-load intelligent heterogeneous computing system, device, and medium for CPU-GPU heterogeneous computing platforms are provided.

[0016] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0017] Example 1 The following is combined Figures 1-10 The present invention describes a hybrid load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms using the E-Commerce mode of the multi-modal query benchmark M2Bench as an example. This example includes a hybrid load of relational model, document model and graph model.

[0018] like Figure 1 As shown in this embodiment, a hybrid workload intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms is proposed. This method mainly consists of adaptive data distribution and caching, and includes the following steps: S1. Optimize the data distribution, storage, and caching on the CPU and GPU sides based on the computing characteristics of the GPU and the workload, and create a multi-dimensional computing index based on the GPU to realize data distribution and caching oriented towards the computing characteristics of the workload; S2. Using a pre-established multi-mode operator library for GPUs, a computation plan based on a cost model is generated. The operator execution site of the generated computation plan is intelligently selected according to the query execution space cost and time cost to obtain the optimal CPU-GPU heterogeneous execution plan. S3. Dynamically schedule the execution of tasks on the CPU and GPU sides according to the optimal CPU-GPU heterogeneous execution plan to obtain the results of intelligent heterogeneous computing with mixed load.

[0019] Furthermore, in step S1 above, the purpose is to establish a data storage model optimized for CPU-GPU heterogeneous platforms, to divide storage between the CPU and GPU sides, to implement distributed storage for GPU computing characteristics, and to implement a cache for computing on the GPU side.

[0020] Specifically, it includes the following steps: S11. Monitor the GPU computing resources in the CPU-GPU heterogeneous computing platform in real time and determine the currently available GPU computing resources.

[0021] GPU storage and computing resources are the core resources of heterogeneous computing platforms. Monitoring GPU storage and computing resources determines the allocatable storage and computing resources, serving the design of the CPU-GPU heterogeneous platform distributed storage model described below. GPU computing resources include statically allocated resources and dynamically allocated resources. Static resource allocation mainly involves GPU storage resource management, used to allocate the storage scale of GPU multidimensional computing indexes that can be created on the GPU, the size of computational data subsets, and GPU indexes or storage structures created by graphs or other models, providing a decision-making basis for the intelligent selection of heterogeneous execution plans and the optimization of mixed load execution plans in step S203. Dynamic resource allocation includes the management of GPU multidimensional computing indexes and the management of GPU intermediate result cache space, used to assess whether the available cache resources on the GPU can meet the result caching requirements of query tasks, thereby assessing whether the query task is suitable for execution on the GPU, providing space-cost-based execution site selection for the following steps (i.e., steps S22 and S23).

[0022] S12. Determine the adaptive data distribution and caching strategy for the CPU-GPU heterogeneous platform based on whether the available GPU computing resources are sufficient.

[0023] Addressing the challenge of GPUs' strong computing power but relatively weaker capabilities in complex data management, storage, and logical processing, this approach stores computationally intensive subsets of data from the database suitable for GPU acceleration on the GPU. An operator library for these computationally intensive data subsets is built, and multidimensional indexing is implemented on these subsets using GPU multidimensional computation indexing. Creating the GPU multidimensional computation index requires obtaining the available GPU storage resources on the current GPU computing platform through computing resource allocation and monitoring functions. The available GPU storage resources are then used to determine the CPU-GPU heterogeneous platform distributed storage model. When GPU storage resources are sufficient, all computationally intensive subsets of data can be stored in GPU memory to support high-performance GPU computing. When GPU storage resources are insufficient, horizontal or vertical sharding strategies are used to accelerate hotspots or user-defined accelerated subsets of data using limited GPU storage resources, thereby improving the performance of local computing.

[0024] S13. Based on the determined adaptive data distribution and caching strategy for CPU-GPU heterogeneous platforms, optimize the data distribution, storage and caching on the CPU and GPU sides according to the computing characteristics of GPU and workload, and create a multi-dimensional computing index based on GPU to realize data distribution and caching oriented to the computing characteristics of workload.

[0025] Specifically, it includes the following steps: S131. Convert the original database schema according to the computing plan optimized for CPU-GPU heterogeneous computing platforms, especially GPU platform data storage and computing characteristics, to obtain a unified storage schema.

[0026] like Figure 2 As shown, the original database schema includes relational model, document model, and property graph model. When performing schema transformation on the original database schema, the document storage model and graph model data need to be converted into a column-stored relational model to adapt to GPU storage and computing.

[0027] Specifically, for the document storage model, if the document data table `Reviews` does not contain any subordinate nodes, it is directly converted into a `Reviews` relational table. If the document data table `Orders` contains subordinate nodes, the parent node is converted into an `Orders` relational table, and the subordinate node is converted into an `Orderline` relational table. Simultaneously, a surrogate primary key `order_SPK` (a consecutive integer sequence of 1, 2, 3…) is added to the converted `Orders` relational table as an access index for the subordinate node. A corresponding foreign key `order_SFK` is added to the `Orderline` relational table to store the primary key value `order_SPK` of its corresponding parent node record in the `Orders` relational table. This foreign key value can serve as an access index for the subordinate node to access the parent node in the `Orders` relational table. Similarly, a foreign key `order_SFK` is added to the `Reviews` relational table as an index for accessing records in the `Orders` relational table. Through relational data storage and the setting of surrogate primary and foreign keys, index access between the original document data tables and from subordinate nodes to parent nodes is achieved.

[0028] For the relational model, surrogate primary keys are added to the relational tables Customer, Product, and Brands, and the original primary keys are reduced to ordinary fields. At the same time, foreign keys in the reference tables are replaced with surrogate foreign keys, supporting index access from the foreign key table to the primary key table based on the surrogate key.

[0029] For graph model data, the node table of the graph model is stored as a vertex table with a surrogate primary key added; the edge table of the graph model is stored as an edge table with a surrogate foreign key pointing to the vertex table, supporting index access from the edge table to the vertex table records.

[0030] The transformed schema serves as the main storage on the CPU side. Document storage model and graph model data are uniformly stored as relational data. Graph model data can also be stored in a dedicated graph engine to support general graph model query processing.

[0031] S132. Adopting a distributed storage strategy based on CPU and GPU, the metadata representing semantics and the numerical data used for computation in the unified storage mode are stored on the CPU and GPU respectively, resulting in a metadata and value storage model for CPU-GPU heterogeneous computing platforms.

[0032] like Figure 2 As shown, the GPU stores a subset of data used for computation, storing only numeric foreign keys and measure columns. Non-numeric data containing semantic information, such as Brands.industry, is stored as dictionary table metadata on the CPU, while the corresponding dictionary table encoding is stored in the Brands table on the GPU. This distributed storage strategy on both CPU and GPU stores semantic metadata uniformly on the CPU, while the GPU only stores value data without semantic information. This simplifies the complexity of data management, storage, and processing on the GPU, maximizing its computing power. Queries on the semantic columns of the original table are transformed into preprocessing of the metadata dictionary table on the CPU and mapping computation on the GPU. Through distributed storage and collaborative computation between the CPU and GPU, equivalent query processing is achieved, essentially converting computation on the original semantic data columns into two processing stages: semantic metadata computation on the CPU and semantic mapping computation on the GPU.

[0033] The metadata and value storage model for CPU-GPU heterogeneous computing platforms defines both the subsets of data stored on the CPU and GPU sides, as well as the computation on the CPU and GPU sides. The CPU side stores multi-dimensional metadata, such as dimensional attributes and dimensional hierarchies (e.g., ...). Figure 2 The Productid, title, Brandid, and industry tables store the original dimensional attributes or dimensional hierarchical attribute values ​​as metadata dictionary tables. They are mainly used for multidimensional filtering or grouping operations on complex data types on the CPU side. During query execution, the query subtask is executed on the original table or on a separate metadata table, depending on the metadata storage structure, query type, and CPU-side metadata computation cost model. It is then executed in conjunction with subsequent GPU-side computation.

[0034] S133. Based on the computational characteristics of the load, a multi-dimensional computation index is established on the GPU side as a GPU cache for the intermediate result set structure to improve query processing performance.

[0035] like Figure 3As shown, the metadata and value storage model decomposes the unified storage model into CPU-side metadata storage and GPU-side computational value set storage. This achieves distributed storage that separates management datasets and computational datasets on both the CPU and GPU sides, simplifying data type storage on the GPU side to computational value storage. Based on the operator implementation technology and cost model-based computation plan generator implementation technology provided by the operator library below, a vector index is created for each primary key table on the GPU side as a GPU cache for the primary key table's computation results, accelerating GPU-side computation performance. Each vector index is a vector structure of the same length as the primary key table and can be used as a bitmap index or a grouped vector index, representing the pre-computed results on the primary key table participating in the next stage of query processing.

[0036] For example, operations such as `order_date = current_date - 1 year` and `order_date = '2018-07-07'` on the `Orders` table generate an `Orders` bitmap index. This bitmap indicates the status of records in the `Orders` table that meet the filtering conditions and also serves as a filter when the `Orderline` and `Reviews` tables are joined using the `order_SFK` foreign key. When executing `GROUP BY Brand.industry`, `BrandVec` stores the `industry_SFK` from the `Brands` table as a compressed grouping index, integrates it into the CPU-side `industry` metadata dictionary table (of equal length), and uses it as the grouping vector for the query. The aggregation calculation results mapped from the `Products` table are stored in the `industry` vector for further aggregation calculations.

[0037] The vector index cached on the GPU provides a unified intermediate result set structure for queries, replacing the original data to perform filtering or grouping and aggregation calculation tasks. The smaller vector index structure can efficiently utilize the GPU cache and significantly improve query processing performance.

[0038] GPU-side computation is divided into two stages: vector index update and multidimensional computation. The primary key table is updated in the cached vector index in the form of bitmap or grouped vectors through preprocessing. The cached vector index provides a multidimensional vector filter for multidimensional computation on the GPU fact table to complete the GPU multidimensional computation task.

[0039] Furthermore, step S2 above includes the following steps: S21. Establish a multi-mode operator library for GPUs.

[0040] This invention uses multi-modal queries, which include relational models, document models, and graph models, as an example to illustrate the implementation method of the GPU multi-modal operator library.

[0041] The relational model-based multi-mode operator library contains two main categories of operators: dimensional vector mapping and multi-dimensional computation. The multi-dimensional computation operators are macro operators designed based on kernel fusion technology, including traditional selection, connection, grouping, and aggregation operators, to realize the multi-dimensional computation function of OLAP.

[0042] In query tasks, predicates or grouping operations on the primary key table are first converted into vector indexes, that is, dimension table vector indexes are generated through dimension vector mapping operators, which contain bitmap indexes or grouping information mapped from the underlying hierarchical tables, providing dimension filters for multidimensional calculation operators.

[0043] In multidimensional computation operators, access based on primary and foreign key join tables is designed as a multi-level access based on surrogate key mapping on the GPU. The schema defines the access path corresponding to the query, and vector access operations based on multi-level nested address mapping are executed on the GPU. When multi-operator computation is performed on the GPU (such as multi-table join operations), the PyNuclear technology is used to implement multiple operators in a unified kernel function. Multi-level mapping access is achieved through surrogate keys between multiple tables, eliminating the intermediate result materialization cost brought by the traditional one-at-a-time (operator-at-a-time) computation model of GPU databases, and improving the multi-operator GPU computing performance and storage space utilization.

[0044] The UNNEST operation in the document model is transformed into a primary-foreign-key join operation based on the relational model, and the match operation in the graph model is transformed into a multidimensional mapping calculation based on the bitmap index of the point set.

[0045] Figure 4 shows a query example on a hybrid relational and document model.

[0046] SELECT DISTINCT Customer.person_id as person_id, SUM(Order.Order_line.price) as total_spent FROM Product, Order, Brand UNNEST Order.order_line WHERE Product.product_id=Order.order_line.product_id AND Brand.brand_name=Product.brand_name AND Brand.industry='x' AND Order.customer_id = Customer.customer_id GROUP BY person_id HAVING total_spent>'x'; The query involves the processing of queries after the document model is transformed into a relational model. The original document table Orders is transformed into Orderline and Orders tables. The Orderline table accesses the records corresponding to the order_SPK (OSPK) column of the Orders table through the order_SFK (OSFK) mapping, realizing the equivalent operation of Order UNNEST Order.order_line.

[0047] The operator library includes CPU-side and GPU-side operators. CPU-side operators execute the `Brand.industry='x'` operation on the CPU-side Industry metadata dictionary table, generating a bitmap index that meets the filtering conditions. The smaller bitmap index is then transferred via PCIe to the GPU-side cached IndustryVec vector index. The GPU-side stores the `industry_SFK` (ISFK) column of the `Brands` table, mapping values ​​to the address of the bitmap index `IndustryVec`, and updates the `BrandVec` vector index (used as a bitmap index, storing only 0s or 1s) based on whether the bitmap index cell is 0 or 1. The GPU-side stores the `brand_SFK` (BSFK) column of the `Products` table, mapping values ​​to the address of the vector index `BrandVec`, and updates the `ProductVec` vector index based on the vector index cell value. This completes the execution of the dimension vector mapping operator, serving as a filter at the `Products` dimension level for multidimensional filtering calculations with the fact table data.

[0048] In the multidimensional computation operator, the multifact table constructs a cascading access path based on address mapping through surrogate primary and foreign keys. In this embodiment, PersonVec is used as a grouping vector. Orderline records access the PersonVec grouping vector through OSFK-(OSPK)–CSFK-(CSPK)–PSFK. When executing the GPU aggregation computation operator, the Orderline table is mapped to the ProductVec vector index through PSFK. Records with a mapping value of 1 are used for subsequent grouping aggregation computation tasks, while records with a mapping value of 0 are filtered out. For example, the first record in the Orderline table with a PSFK value of 3 is mapped to the 3rd unit of ProductVec. A value of 1 is mapped to the 4th record in the Orders table through OSFK value 4, to the 5th record in the Customer table through CSFK value 5, and to the 6th unit of the PersonVec grouping vector through PSFK value 6, where the Oprice value is aggregated and computed. After all records in the Orderline table have been processed, PersonVec becomes the final grouped vector result. The PersonVec vector is scanned and its value is checked to see if it is greater than 'x' (corresponding to the HAVING total_spent>'x' operation). If the condition is not met, its cell value is set to 0; otherwise, it is set to 1. The PersonVec grouped vector is then simplified into a bitmap index to identify the set of personids that meet the query conditions, which is used for subsequent query processing.

[0049] The operator library contains extended relational operators, such as the match operation on the graph model in multimodal queries. (n:Person)-[FOLLOWS]->(influencer:Person) AND SNS.influencer.person_id = A.person_id can be seen as an extended graph predicate operation, which can be computed by a dedicated graph engine or implemented by the extended match operator on the relational storage model.

[0050] like Figure 5 As shown, the vertex table and edge table of the graph model can be uniformly stored as a relation table. The vertex table uses a surrogate primary key, and the edge table uses a surrogate foreign key, realizing the address mapping from the edge table surrogate foreign key to the vertex table record.

[0051] The following is an example of a forward mapping of the Match operator: (p1:Person) - [FOLLOWS] ->(p2:Person) AND SNS.p1.person_id in (1,3,6) The execution steps of this operator are as follows: First, mark the input personid in the PersonVec1 vector index to generate an input mapping bitmap (positions 1, 3, and 6 are marked as 1, and other positions are marked as 0); Second, scan the P1.id column of the Follows edge table, map the surrogate foreign key value to the corresponding cell of the PersonVec1 vector index, and if it is 1, map the P2.id surrogate foreign key value to the corresponding position of the output PersonVec2 vector index and mark it as 1, indicating that the personid currently followed by the person corresponding to P1.id; Finally, output the vector index PersonVec2 to indicate that the users with personids 1, 2, 3, 4, and 5 are followed by the users with personids 1, 3, and 6.

[0052] The following is an example of the reverse mapping of the Match operator: (p1:Person) - [FOLLOWS] ->(p2:Person) AND SNS.p2.person_id=4 The execution steps of this operator are as follows: First, mark the input personid in the PersonVec2 vector index to generate an input mapping bitmap (position 4 is marked as 1, and other positions are marked as 0); Second, scan the P2.id column of the Follows edge table, map the surrogate foreign key value to the corresponding cell of the PersonVec2 vector index, and if it is 1, map the P1.id surrogate foreign key value to the corresponding position of the output PersonVec1 vector index and mark it as 1, indicating that the person corresponding to the current P2.id is followed by this personid; Finally, the output vector index PersonVec1 indicates that the users with personid 3 and 5 have followed the user with personid 4.

[0053] like Figure 6 The diagram shows the execution of the multi-hop match operator. An example query is: (p1:Person) - [FOLLOWS*2] ->(p2:Person) AND SNS.p2.person_id =4 To query which personids follow the personid with personid 4 after a two-hop mapping, the operator is a multi-hop reverse mapping calculation.

[0054] The execution steps of this multi-hop match operator are as follows: First, mark the input personid in the PersonVec2 vector index to generate an input mapping bitmap (position 4 is marked as 1, and other positions are marked as 0); Second, scan the P2.id column of the Follows edge table, map the surrogate foreign key value to the corresponding cell of the PersonVec2 intermediate vector index. If it is 1, map the P1.id surrogate foreign key value to the corresponding position of the output PersonVec0.1 vector index and mark it as 1, indicating that the person corresponding to the current P2.id is followed by this personid; Subsequently, scan the P2.id column of the Follows edge table again, map the surrogate foreign key value to the corresponding cell of the PersonVec0.1 intermediate vector index. If it is 1, map the P1.id surrogate foreign key value to the corresponding position of the output PersonVec1 vector index and mark it as 1, indicating that the person corresponding to the current P2.id is followed by this personid; Finally, the output vector index PersonVec1 indicates that the user followed by the user with personid 1, 3, and 6 has followed the user with personid 4.

[0055] The multi-modal operator library provides operator implementations suitable for GPU-based computational features, including operators for document and graph models suitable for unified computation. The CPU-based library stores the complete dataset and supports full operator implementations, such as relational engines, document engines, and graph engines, enabling general multi-modal query processing. GPU-based operators require further evaluation based on operator type, execution time, and space cost before execution on the CPU or GPU.

[0056] S22. Based on the multi-mode operator library for GPUs, generate a computation plan based on the cost model.

[0057] The GPU multidimensional computing index designed in this invention is based on the distributed storage and vector index cache of data computing features on both the CPU and GPU sides implemented above. As a GPU multidimensional computing index of the complete dataset on the CPU side, it stores computationally intensive data subsets and provides multidimensional computing functions on GPU data subsets, thereby achieving GPU acceleration for querying subtrees for computationally intensive loads.

[0058] The GPU multidimensional computing index proposed in this invention adopts a data-centric computing plan generation strategy. It merges multiple independently executed but highly data-related query tasks into a unified GPU computing plan for execution on the GPU through query fusion technology. It also uses GPU vector index caching to pass intermediate result data between tasks, eliminating the caching and transmission costs of query results between the GPU and CPU when executing individual queries.

[0059] Taking the following query task as an example, the entire query task is divided into 4 independent query clauses, which need to be executed independently and the query results need to be materialized. The intermediate query results may be redundantly transmitted between the GPU and the CPU or occupy a large amount of storage space on the GPU. The space and time costs of query execution are high.

[0060] A = SELECT oid, order_line.pid AS pid, order_line.price AS order_price FROM Order UNNEST order_line WHERE order_date = current_date - 1 year B=SELECTA.oid AS oid, Product.pid AS pid, Brand.brand_name AS brand_name, A.order_price AS order_price FROM A, Product, Brand WHERE Product.pid=A.pid AND Product. brand_name = Brand.brand_name C=SELECTB.brand_name AS topseller_brand, SUM(B.order_price) ASrevenue FROM B GROUP BY B.brand_name ORDER BY revenue DESC LIMIT 1 D=SELECTB.pid,(SUM(B.order_price) / C.revenue*100) AS percent_of_revenue FROM B, C WHERE C.topseller_brand=B.brand_name GROUP BY B.pid This invention optimizes GPU computing plans by employing query fusion and multi-pass computation strategies on the GPU side. Based on storage space and computational cost assessments, executing the original query order would result in significant intermediate result materialization costs and data transfer costs between the CPU and GPU. Estimated transmission costs based on GPU memory bandwidth and PCIe bandwidth are far higher than the GPU's local computation costs.

[0061] like Figure 7 As shown, the computational plan based on cost model optimization is rewritten as follows: First, convert query A into creating a bitmap index for the Orders table.

[0062] Secondly, queries A, B, and C are integrated into a single multidimensional computation task, which is achieved through two rounds of computation.

[0063] In the first pass of computation, a Bname grouping vector is first created using the dimension vector mapping operator, with Bname_SFK as the index address. The ProductVec vector index is generated by accessing Brands.Bname_SFK through Products.brand_SFK. Then, the Orderline table is scanned using the multidimensional computation operator. The bitmap index generated by mapping order_SFK to the Orders table is used for filtering. Records that meet the filtering conditions are accessed through the ProductVec vector index via Orderline.product_SFK. The Bname_SFK value stored therein is mapped to the grouping vector Bname for SUM(B.order_price) aggregation calculation. After the calculation is completed, the Bname grouping vector is scanned to calculate the maximum value and the grouping vector ID is recorded. The Bname grouping vector is scanned again, and non-maximum value cells are marked as 0. The Bname grouping vector is then output. In the second pass of computation, a BrandVec vector index is created based on the Bname grouping vector, mapping the maximum grouping value in BnameVec. Then, a ProductVec vector index and a ProductVec1 grouping vector are created based on the BrandVec vector index as outputs of the dimensional vector mapping operator. The Orderline table is rescanned using a multidimensional computation operator, and the Orders table bitmap index is filtered by mapping order_SFK. Records that meet the filtering conditions are mapped to the ProductVec vector index by the PSFK foreign key value. Non-null values ​​are calculated with the Oprice value and mapped to the ProductVec1 grouping vector unit for aggregation computation.

[0064] The query computation plan employs a data-centric optimization strategy, fusing multiple query tasks based on data computation characteristics to eliminate the materialization cost of intermediate results. When computation tasks are divided into multiple stages, a multi-pass computation plan is used, replacing the space-intensive materialized execution plan with high-performance redundant computation on the GPU, thus executing a computation plan that optimizes both storage space and computation time.

[0065] S23. Based on the currently available GPU computing resources determined in step S11, intelligently select the operator execution site for different computing plans according to the query execution space cost and time cost to obtain the optimal CPU-GPU heterogeneous execution plan.

[0066] Based on the storage and computing cost model, the computing engine generates the optimal computing plan for the GPU and returns the GPU computing plan result to the CPU for subsequent operations based on the different computing plans generated and the dynamic management module of computing resources.

[0067] The GPU acceleration engine provided by this invention adopts a GPU multidimensional computation index, which is a storage-computation index compared to traditional storage-retrieval indexes. It provides storage and multidimensional computation functions based on GPU hardware, and is a GPU-integrated storage-computation index structure. Traditional indexes are created based on a single table or a subset of columns. The GPU multidimensional computation index of this invention is an index structure oriented towards multidimensional data models, including multiple tables, computation-related columns, and inter-table surrogate key index structures, forming a multidimensional data index. It provides multidimensional computation functions on multidimensional data, realizing integrated storage-computation multidimensional index computation on the GPU side.

[0068] A GPU multidimensional computation index is a computational subset of data in a database. Users can create GPU multidimensional computation indexes for specific data subsets based on data access frequency and customized needs. GPU storage resources are allocated through the computational resource allocation and monitoring module to accelerate multidimensional computation performance on specific data subsets. GPU multidimensional computation indexes can be set to "on" or "off." Enabling the index function supports multidimensional computation index acceleration for CPU-GPU heterogeneous platforms, while disabling the index function redirects query tasks to the CPU platform, pausing the GPU multidimensional computation index acceleration function. When system GPU storage resources are insufficient or a higher-priority GPU computation resource allocation requirement occurs, the GPU multidimensional computation index can be removed to release GPU computation resources. The functionality of the GPU multidimensional computation index is implemented by the CPU-side query processing engine.

[0069] Similarly, GPU computing platforms can be extended to other models, such as graph models, vector models, document models, array models, and similar GPU indexes, to accelerate data retrieval or customized computing functions on the corresponding models. Dynamic management of computing resources for computing tasks can provide dynamic management of computing resources at the index granularity. By allocating and releasing corresponding GPU storage and computing resources through GPU index management, the execution performance of different workloads can be accelerated.

[0070] The following query task serves as an example to illustrate the heterogeneous execution plan selection strategy.

[0071] A = SELECT p.person_id, t.tag_id FROM (MATCH (p: Person)-[:Interested_in]->(t: Hashtag) RETURN p, t B = SELECT Customer.person_id, Product.brand_id, COUNT(Review.review_id) as cnt FROM Product, Review, Order, Customer WHERE Customer.customer_id = Order.customer_id AND Order.order_id = Review.order_id AND Review.product_id = Product.product_id AND Review.rating = 5 GROUP BY Customer.person_id, Product.brand_id C = SELECT B.person_id, MIN(B.brand_id) FROM B, (SELECT person_id, MAX(cnt) as max_cnt FROM B GROUP BYperson_id) AS T1 WHERE B.person_id = T1.person_id AND B.cnt = T1.max_cnt GROUP BY B.person_id D = A.toArray(dim1: person_id, dim2: tag_id, val: 1) E = C.toArray(dim1: person_id, val: (brand_id == given_brand_id) ? 1: 0) F = LogisticRegression(D, E) Query tasks A and B can be executed on either the CPU or the GPU. Besides the execution costs of different platforms, the space cost of the query tasks also needs to be considered. Query task A generates a two-dimensional array. When the data size is large, the two-dimensional data storage space is large, which may exceed the available GPU storage resources. With optimization strategies, it can be allocated to CPU execution. Query task B outputs a relatively small result set (e.g., 256MB when SF=100), satisfying the GPU's available storage resource space constraints. Query task C further reduces the result set based on the result set of query task B. Query tasks B and C, after evaluation of storage space and computational costs, are suitable for execution on the GPU. Query fusion technology is used to optimize execution on the GPU, generating a smaller person_id vector, which is then transmitted to the CPU for execution of query tasks D, E, and F.

[0072] For example, the following query task, after evaluating the cost of storage space and computation time, generates a large result set, which does not conform to the optimization principle of small input-large computation-small output of the GPU computing engine. Therefore, a CPU-side execution plan is adopted.

[0073] A = SELECT customer_id, product_id, rating FROM Review, "Order" WHERE Review.order_id="Order".order_id B, C = A.toArray( dim1: customer_id, dim2: product_id, val: avg(rating)).Factorization D = MatMul(B,C) When a query task includes a multi-model computation task, such as query task A being a relational query task and task B being a graph model computation task, if task B is not included in the relational engine's extended operator library, the execution result of query task A is returned to the CPU or GPU-side graph engine to complete task B.

[0074] A = SELECT Customer.person_id AS person_id FROM Order, Review, Customer WHERE Review.product_id='X' AND Review.order_id=Order.order_id AND Order.order_date = current_date – 1 year AND Order.customer_id=Customer.customer_id AND Customer.gender='female' / / Relational B = SELECT p, r, node AS subGraph FROM A, SNS WHERE (p:Person) - [r] ->(node) AND SNS.p.person_id=A.person_id / / Graph Furthermore, in step S3 above, a unified GPU multidimensional computing indexing implementation technology for relational, partial graph, and document models is provided to enable heterogeneous computing across CPU and GPU for computationally intensive datasets.

[0075] In step S1, distributed storage and caching across CPU and GPU were implemented, and multidimensional computation indexing acceleration on computationally intensive datasets was provided in the form of GPU multidimensional computation index. The implementation of the multi-mode operator library in step S201 supported efficient multi-mode computation acceleration on the GPU side.

[0076] like Figure 8 As shown, the database system generates a complete query plan at the CPU-side database query engine layer. Based on the GPU multidimensional computing index metadata, it determines the multidimensional computing query subtasks that can be accelerated by the GPU. The execution location of the query subtask is determined based on whether the query subtask data is within the range of the GPU multidimensional index, whether the computing task is supported by the GPU multidimensional computing index, and whether the space and time costs of the query subtask are lower on the GPU side.

[0077] The query subtasks executed on the GPU are decomposed into a mixed workload of CPU-side metadata computation and GPU multidimensional index computation, which are executed by the CPU and GPU functional modules respectively. If the multidimensional computation subtasks included in the query are accelerated by GPU multidimensional index computation, the process first involves filtering or grouping compression operations based on a compressed data dictionary or dimension hierarchy on the CPU-side metadata storage layer. Then, the generated bitmap or compressed grouped vector is transmitted to the GPU via PCIe. The GPU-side dimension vector mapping is then called to generate a dimension table cache vector index. Next, the multidimensional computation module is called to perform multidimensional computation, generating grouped vectors and executing other multidimensional computation tasks on the GPU. Finally, the generated grouped vectors, bitmaps, or dimension vectors are transmitted to the CPU via PCIe. The metadata management module parses the query results and outputs them to the CPU query plan tree as the result set of the accelerated query subtasks, which is then used to execute subsequent query processing tasks.

[0078] When the query load includes query subtasks on different models, the GPU multidimensional computing index provided by this invention can support some graph operators and document operators based on the unified relational storage model. Other operators need to be executed in collaboration with dedicated acceleration engines on the CPU or GPU. The database query processing engine optimizer needs to schedule different acceleration engines to execute mixed loads on different platforms and different computing engines through a unified query execution plan.

[0079] When the query task involves HTAP load, the present invention adopts the following implementation strategy: S31. Dimension tables are centrally stored in the CPU-side database engine. The CPU-side database engine completes the dimension mapping query task, generates dimension vectors, and transmits them to the GPU to update the multidimensional vector index. S32. The fact table is stored in the CPU-side database engine and the GPU multidimensional vector index. The GPU-side stores read-only data after the last data synchronization. For distributed aggregate computing tasks, such as sum and count queries, HTAP query processing is supported. Multidimensional computing tasks on read-only data are executed on the GPU side. S33. Add a delta table to the fact table on the CPU side. After the fact table record is updated, its change record (Insert, Delete, Update) is stored in the delta table, marking the record modification type (I, D, U is decomposed into D and I operations) and modification timestamp, for querying the visibility determination of the updated data version; When executing S34 and HTAP queries, multidimensional calculation tasks are performed simultaneously on the Delta table on the CPU side and the read-only fact table on the GPU side. On the CPU side Delta table, the metric values ​​of type I records are aggregated and calculated, and the negative values ​​of type D records are aggregated and calculated to obtain the aggregated calculation results of incremental records on the CPU side. On the GPU side, the aggregated calculation results on the read-only data are output. Then, the aggregated calculation results from the CPU side and the GPU side are merged, and the impact of the update of records on the GPU side is offset by the aggregated calculation results from the Delta table. S35. When querying the total_price attribute in the Orders table, such as sum(total_price), if the same order_id record in the order_line table at the next level is updated, the corresponding total_price calculation result is generated by using the order_line record in the delta table. The impact of the order_line change in the query on sum(total_price) is subtracted, and the incremental value corresponding to sum(total_price) is calculated by adding the updated order_line record. The update delta data and the calculation result of the GPU data end are merged through merging. The S36 and HTAP hybrid workload is decomposed into multidimensional computational workloads on the CPU-side Delta table dataset and the GPU dataset, and consistent query processing results on the updatable fact table are obtained by merging the multidimensional computation results on the CPU-side.

[0080] like Figure 9 As shown, the query performs a clustering calculation sum(A3) on column A3 of the fact table. The CPU-side database engine stores the fact table, which supports real-time updates, generating Insert (records with IDs 13, 14, 15, and 16), Update (record with ID 5), and Delete (record with ID 11). When executing the clustering calculation sum(A3), the updated records in the fact table are accessed in real time for clustering calculation, resulting in a query result of 62.

[0081] like Figure 10As shown, assuming a GPU multidimensional computation index is created at timestamp 100, the GPU fact table stores a subset of records with IDs up to 12. Records with timestamps exceeding 100 are updated in real-time by the CPU database storage engine, and their updates are inserted into the Delta table. This includes Insert records with timestamps greater than 100, marked with Type I, and the timestamp of the insertion operation. Update records are decomposed into two records with the same timestamp: a Delete record and an Insert record (the record with ID 5), marked with Types D and I respectively, and their timestamps updated. Delete records are added to the Delta table, marked with Type I, and their timestamps updated. When a record is updated multiple times (e.g., the record with ID 5 is updated again after the first update), Delta records of types D and I are added for each update.

[0082] The HTAP query is executed simultaneously in the CPU Delta table and the GPU multidimensional vector index. The sum(A3) result on the GPU read-only dataset is 49. When sum(A3) is executed on the Delta table, the accumulated value of A3 for records of type D is negative, eliminating the impact of record deletion on the sum operation. Finally, the sum(A3) result on the CPU Delta table is 13. Finally, the two result sets are merged, and the HTAP query sum(A3) = 13 + 49 = 62.

[0083] During an HTAP query, if an update operation occurs after the query operation, such as initiating a CPU and GPU query task at timestamp 128 and inserting a record with ID 16 at timestamp 130 during query execution, the HTAP query will determine the visibility of each Delta record to the current query based on the query timestamp 128 when scanning the Delta table on the CPU side. If the Delta record timestamp is greater than the query timestamp (e.g., the record with ID 16 was generated after the query execution), then that record is not visible in the current query and will not participate in the aggregation calculation task.

[0084] Hybrid load for HTAP decomposes the query into a CPU-side Delta table query load and a GPU-side multidimensional vector index computation load, which are executed in parallel on both the CPU and GPU sides, and finally merge the query result sets on the CPU side.

[0085] Example 2 The above-described embodiment 1 provides a hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms. Correspondingly, this embodiment provides a hybrid-load intelligent heterogeneous computing system for CPU-GPU heterogeneous computing platforms. The system provided in this embodiment can implement the hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms described in embodiment 1. This system can be implemented through software, hardware, or a combination of both. For example, the system may include integrated or separate functional modules or units to execute the corresponding steps in the methods of embodiment 1. Since the system in this embodiment is basically similar to the method embodiment, the description process in this embodiment is relatively simple. Relevant details can be found in the description of embodiment 1. The system embodiment provided in this embodiment is merely illustrative.

[0086] The hybrid-load intelligent heterogeneous computing system for CPU-GPU heterogeneous computing platforms provided in this embodiment includes: The adaptive data distribution and caching module is configured to optimize the data distribution, storage, and caching on both the CPU and GPU sides based on the computational characteristics of the GPU and workload, and to create a GPU-based multidimensional computation index. The heterogeneous computing plan optimization module is configured to use a pre-built multi-mode operator library for GPUs to generate a cost model-based computing plan, and intelligently select different computing plans based on the query execution space cost and time cost to obtain the optimal CPU-GPU heterogeneous execution plan. The computing resource management and scheduling module is configured to dynamically schedule task execution on the CPU and GPU sides according to the optimal CPU-GPU heterogeneous execution plan, and optimize the performance of mixed load execution on the GPU side through query fusion technology.

[0087] In this embodiment, the hybrid-load intelligent heterogeneous computing system for CPU-GPU heterogeneous computing platforms is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0088] Example 3 This embodiment provides a processing device corresponding to the hybrid load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms provided in Embodiment 1. The processing device can be a client-side processing device, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Embodiment 1.

[0089] The processing device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms provided in Embodiment 1.

[0090] Preferably, the memory may be high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0091] Preferably, the processor can be any type of general-purpose processor such as a central processing unit (CPU) or a digital signal processor (DSP), and there is no limitation herein.

[0092] Example 4 The hybrid load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms described in Embodiment 1 can be specifically implemented as a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions are loaded for executing the hybrid load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms described in Embodiment 1.

[0093] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms, characterized in that, include: Optimize data distribution storage and caching on both CPU and GPU sides based on the computational characteristics of GPUs and workloads, and create GPU-based multidimensional computation indexes; Using a pre-built multi-mode operator library for GPUs, a cost-based computation plan is generated. The operator execution site of the generated computation plan is intelligently selected based on the query execution space cost and time cost to obtain the optimal CPU-GPU heterogeneous execution plan. The CPU and GPU tasks are dynamically scheduled based on the optimal CPU-GPU heterogeneous execution plan to obtain intelligent heterogeneous computing results with mixed workloads.

2. The hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms as described in claim 1, characterized in that, Before optimizing the data distribution storage and caching on the CPU and GPU sides based on the computational characteristics of the GPU and workload, and creating a GPU-based multidimensional computation index, the following is also included: Real-time monitoring of GPU computing resources in CPU-GPU heterogeneous computing platforms to determine currently available GPU computing resources; Compare the currently available GPU computing resources with the actual required GPU computing resources, and determine the data distribution and storage model for the CPU-GPU heterogeneous platform based on the comparison results: If the available GPU storage resources are greater than the actual required GPU computing resources, then all computationally intensive data subsets in the database will be stored in the GPU. Otherwise, a horizontal or vertical sharding strategy will be used to store acceleration hotspots or user-defined acceleration data subsets in the database in the GPU.

3. The hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms as described in claim 1, characterized in that, The optimization of data distribution storage and caching on the CPU and GPU sides based on the computational characteristics of the GPU and workload, and the creation of a GPU-based multidimensional computation index, include: The original database schema is transformed to be compatible with CPU-GPU heterogeneous computing platforms, resulting in a unified storage schema. A distributed storage strategy based on CPU and GPU is adopted, in which the metadata representing semantics and the numerical data used for calculation are stored on the CPU and GPU respectively in the unified storage mode. Based on the computational characteristics of the workload, a multi-dimensional computation index is built on the GPU side as a GPU cache for intermediate result set structures to improve query processing performance.

4. The hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms as described in claim 3, characterized in that, The process of converting the original database schema to a unified storage schema for CPU-GPU heterogeneous computing platforms includes: For the document storage model, if the document data table is Reviews and does not contain any subordinate nodes, it is directly converted into a Reviews relation table. If the document data table is Orders and contains subordinate nodes, the parent node is converted into an Orders relation table, and the next-level node is converted into an Orderline relation table. At the same time, a surrogate primary key order_SPK is added to the converted Orders relation table as an access index for the next-level node, and a corresponding foreign key order_SFK is added to the Orderline relation table to store the primary key value order_SPK of its corresponding parent node recorded in the Orders relation table. The foreign key order_SFK serves as an access index for the next-level node to access the parent node in the Orders relation table. For the relational model, surrogate primary keys are added to the relational tables Customer, Product, and Brands, and the original primary keys are reduced to ordinary fields. At the same time, foreign keys in the reference tables are replaced with surrogate foreign keys, supporting index access from the reference tables to the relational tables based on the surrogate keys. For graph model data, the node table of the graph model is stored as a vertex relationship table, with a surrogate primary key added; the edge table of the graph model is stored as an edge relationship table, with a surrogate foreign key pointing to the vertex relationship table, supporting index access from the edge relationship table to the vertex relationship table records.

5. The hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms as described in claim 2, characterized in that, The process involves using a pre-established multi-mode operator library for GPUs to generate a cost-based computation plan. The generated computation plan's operator execution venue is intelligently selected based on the query's execution space and time costs to obtain the optimal CPU-GPU heterogeneous execution plan. This includes: A multi-mode operator library for GPUs is established; the multi-mode operator library includes dimension vector mapping operators and multi-dimensional computation operators. The dimension vector mapping operators are used to convert predicates or grouping operations in query tasks into dimension table vector indexes; the multi-dimensional computation operators are used to perform multi-level access based on surrogate key mapping based on dimension table vector indexes to realize multi-dimensional computation. Based on a multi-mode operator library for GPUs, a cost model-based computation plan is generated. Based on the currently available GPU computing resources, the operator execution site of the generated computing plan is intelligently selected according to the query execution space cost and time cost to obtain the optimal CPU-GPU heterogeneous execution plan.

6. The hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms as described in claim 5, characterized in that, The generation of a cost-based computation plan, based on a multi-mode operator library for GPUs, includes: The entire query task is divided into multiple independently executed query subtasks, and a dimension table vector index is created for one of the query subtasks using the dimension vector mapping operator. By using query fusion technology, multiple query subtasks are merged according to data calculation features to generate a multidimensional calculation task; Based on the established dimension table vector index, the multidimensional computation task is executed multiple times on the GPU using multidimensional computation operators at different computation stages.

7. The hybrid-load intelligent heterogeneous computing method for CPU-GPU heterogeneous computing platforms as described in claim 1, characterized in that, The process of dynamically scheduling CPU and GPU tasks according to the optimal CPU-GPU heterogeneous execution plan to obtain hybrid load execution results includes: Obtain the complete CPU-GPU heterogeneous execution plan at the CPU-side database query engine layer: If the query task includes a multidimensional computation query subtask accelerated by the GPU, the query subtask executed on the GPU side will be decomposed into a mixed load of CPU-side metadata computation and GPU-side multidimensional computation index computation, which will be executed by the CPU and GPU-side functional modules respectively. If the query task contains query subtasks on different models, then for some graph operators and document operators based on the unified relational storage model, the GPU multidimensional computing index is used to perform operator computation; for other operators, the GPU multidimensional computing index is executed in conjunction with the dedicated acceleration engine on the CPU or GPU, and the database query processing engine optimizer is used to schedule different acceleration engines to perform mixed load execution on different platforms and different computing engines through a unified query execution plan.

8. A hybrid-load intelligent heterogeneous computing system for CPU-GPU heterogeneous computing platforms, characterized in that, include: The adaptive data distribution and caching module is configured to optimize the data distribution, storage, and caching on both the CPU and GPU sides based on the computational characteristics of the GPU and workload, and to create a GPU-based multidimensional computation index. The heterogeneous computing plan optimization module is configured to use a pre-built multi-mode operator library for GPUs to generate a cost model-based computing plan, and intelligently select different computing plans based on the query execution space cost and time cost to obtain the optimal CPU-GPU heterogeneous execution plan. The computing resource management and scheduling module is configured to dynamically schedule the execution of tasks on the CPU and GPU sides according to the optimal CPU-GPU heterogeneous execution plan, so as to obtain the mixed load execution result.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 7.

10. A computing device, characterized in that, include: One or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 7.

Images