CSP architecture for database access acceleration

Abstract

This application discloses a CSP architecture for database access acceleration, involving the field of FPGA technology, including a database acceleration engine, a high-speed converged network subsystem, a storage access acceleration subsystem, and a database management subsystem. The high-speed converged network subsystem integrates UDP and application layer full offload support for business data transmission, and integrates a TOE engine. The storage access acceleration subsystem includes a memory management access engine and a storage management access engine. The database management subsystem includes initialization configuration for managing the database acceleration engine, enabling control and configuration of various database resources based on client requests. The database acceleration engine includes several database controllers and database task pipelines to facilitate the database management subsystem's access to database services. This CSP architecture is independent of the server and provides data consolidation capabilities, significantly improving processing performance and database compatibility.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of FPGA technology, and in particular to a CSP architecture for database access acceleration. Background Art

[0002] The Computational Storage Architecture and Programming Model v1.0 standard, released by the Storage Network Industry Association (SNIA) on August 30, 2022, defines three types of computational storage architectures: CSP, CSD, and CSA. This standard aims to guide and encourage relevant companies and institutions in the industry to design near-data computing for FPGA or chip products. In FPGA or chip-level designs, all are CSP types, characterized by the simultaneous inclusion of a storage engine, a compute engine, a resource management engine, and high-speed communications (PCIe or network). This architecture is designed for FPGA or SoC chip-level architecture in the database acceleration field, and is a CSP architecture type.

[0003] Traditional accelerator card architectures for database acceleration typically consist of a PCIe subsystem module, a database acceleration engine, a UDP engine, and a memory engine. This model only offloads certain business scenarios, and initialization and link establishment rely on the server. Furthermore, the accelerator card cannot provide disk storage, and the database lacks data persistence, resulting in incomplete functionality and requiring server-based storage. All of these factors complicate deployment and significantly increase total cost of ownership (TCO) and maintenance costs. Furthermore, due to limitations in the server's CPU's ability to process storage IOPS and PCIe transmission performance, deploying multiple cards doesn't deliver linear performance gains, hindering cluster deployment. Summary of the Invention

[0004] In response to the technical problems existing in the above-mentioned related technologies, the embodiments of the present application provide a CSP architecture for database access acceleration, including a database acceleration engine, a high-speed converged network subsystem, a storage access acceleration subsystem, and a database management subsystem;

[0005] The high-speed converged network subsystem integrates a full offload engine at the UDP / IP layer and a TOE engine for directly responding to user requests and transmitting service data;

[0006] The storage access acceleration subsystem includes a memory management access engine and a storage memory management access engine, which are used to provide an integrated virtual address mapping service, respectively perform access management on volatile storage and non-volatile storage devices, and solidify client data;

[0007] The database management subsystem includes initialization configuration information for managing the database acceleration engine, and is used to start control and configure various database resources according to client requests, and simultaneously initialize configuration and operate the high-speed converged network subsystem, the database acceleration engine, and the storage access acceleration subsystem;

[0008] The database acceleration engine includes several independent database controllers and a large number of database task pipelines, which are used to execute and allocate request tasks of each user and call computing resources through a queue interface, so as to facilitate the database management subsystem to access different types of database services.

[0009] Specifically, the TOE engine integrated in the high-speed integrated network subsystem supports high-speed TCP / IP protocol, and controls the communication between the user terminal and the database management subsystem directly; the database management subsystem directly determines the applied database resources, usage type, computing and storage resource allocation based on the initialization or access permission request.

[0010] Specifically, the database acceleration engine includes a database controller resource pool Database controllerPool, a database task thread pool Database task pipline pool, a database acceleration engine storage access interface of the database task thread pool, and a task routing controller; the database controller resource pool includes N independent database controllers, and the database task thread pool contains M database task pipelines; the task routing controller is located between the database controller resource pool and the database task thread pool, allocates a target number of target database task pipelines to the selected target database controller, and provides task instructions and return results.

[0011] Specifically, after receiving the user request, the database management subsystem determines the priority of the target business and the required computing resource size;

[0012] Query the idle database controllers and idle database task pipelines in the database controller resource pool and the database task thread pool respectively;

[0013] The target database controller and its corresponding number of target database task pipelines are determined according to the priority and computing resource size of the target task, and a virtual mapping relationship is established through the task routing controller; the multiple target tasks are processed in parallel, and the priority level is positively correlated with the number of task pipelines.

[0014] Specifically, the database acceleration engine storage access interface is used to connect to the storage access acceleration subsystem; the storage access acceleration subsystem also includes a virtual address mapping space and a storage medium routing;

[0015] The memory engine at least includes an HBM controller and its HBM access interface, a DDR controller and its DDR access interface, and an SRAM controller and its SRAM access interface;

[0016] The Storage engine includes at least a SATA Controller and its SATA access interface, an NVMe Controller and its NVMe access interface, and a Flash Controller and its Flash access interface;

[0017] The virtual address mapping space is used to analyze the target task and the occupied storage space and configuration, and map it to a unified virtual address space with continuous addresses based on the storage medium mounted by the Memory engine and / or the Storage engine and the address space used to configure the mapping relationship table; the storage medium routing is used to determine the target storage controller and the corresponding access interface, the storage space of the target storage medium according to the mapping relationship table, and forward the returned data to the corresponding virtual address space.

[0018] Specifically, the memory engine and the storage engine are connected to storage devices of their respective types through corresponding storage access interfaces, and the non-volatile storage device is used to store at least the memory data and the solidified data of the target database.

[0019] Specifically, the CSP architecture further includes a computing resource pool, which is separated from the database task pipeline and is used to call computing resources when responding to target tasks;

[0020] When responding to the target task, the target database controller selects a task from the task queue according to the task priority and performs task analysis, and estimates the command execution time based on the database access volume, calculation volume, data volume, etc.;

[0021] The database task pipeline performs a kv search on the target task, calls the computing resource pool through the queue interface, and returns the calculation result to locate the position of the relevant element information in the database;

[0022] Call the computing resource pool, and perform data labeling and packaging to construct a response package based on the obtained relevant element information and the returned computing results;

[0023] Send the constructed response package to the task response queue.

[0024] Specifically, when the target task includes write and / or modify operations; after performing kv retrieval, write / modify management operations are performed based on relevant element information, and write request tasks are generated and stored in a write request queue; wherein, the database controller processes write / modify requests serially.

[0025] Specifically, when the initialized database is a relational database, the database task pipeline performs relational operations on the calculation results after the KV retrieval, thereby reducing repeated database query actions;

[0026] When the target task includes data preprocessing calculation, the database task pipeline calls the computing resource pool through the queue interface to perform preprocessing calculation.

[0027] Specifically, when the target task is set with a screening condition, the database task pipeline screens the calculation results / preprocessing results according to the preset conditions to filter out redundant data and data irrelevant to the target task, thereby reducing network bandwidth.

[0028] The technical solutions provided by the embodiments of the present application bring at least the following beneficial effects: introducing a high-speed converged network subsystem to replace the PCIe subsystem in the traditional architecture, supporting UDP and full application layer offload to support business data transmission, and integrating the TOE engine to directly respond to client requests, thus avoiding the drawbacks of the existing architecture that can only carry out business operations while management must be performed through the server;

[0029] Adding a storage engine to the existing memory engine can mask differences between different storage media and achieve unified access to hot and cold storage on the system architecture side. Databases can be quickly deployed on various storage devices, and user-side data can be solidified without occupying server storage space. This also addresses insufficient server CPU IOPS capabilities.

[0030] The independent database controller and database task pipeline in the database acceleration engine can select the target database controller and allocate database task pipelines based on the priority of the target business; it allows the CSP architecture to provide different types of database services to multiple clients, solving the current architecture of one card corresponding to one service and avoiding resource waste; it provides higher storage density and higher access performance in the same space.

[0031] The database task pipeline is designed to be separate from the computing resource pool. When the database task pipeline needs to execute computing tasks, it calls on public computing resources. This achieves higher parallelism and freedom, conserving hardware resources, accommodating more pipelines, and achieving higher performance. With equivalent performance, the construction and maintenance costs are lower.

[0032] The architecture has a built-in virtual database abstraction layer that enables cross-type and cross-vendor database conversion and execution, making it compatible with more application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a block diagram of the traditional CSP architecture for database access acceleration;

[0034] Figure 2 This is a block diagram of the CSP architecture for database access acceleration provided by an embodiment of the present application;

[0035] Figure 3 This is a structural block diagram of the storage access acceleration subsystem provided by an embodiment of the present application;

[0036] Figure 4 This is a structural block diagram of the database acceleration engine provided by an embodiment of the present application;

[0037] Figure 5 It is a processing flow chart of the database acceleration engine when responding to database access tasks;

[0038] Figure 6 This is a structural block diagram of the virtual database translation dictionary abstraction layer provided by an embodiment of the present application;

[0039] Figure 7 This is a structural block diagram of the computing resource pool provided in an embodiment of the present application;

[0040] Figure 8 This is a flowchart of a database access method provided by an embodiment of the present application;

[0041] Figure 9 This is a flowchart of database initialization provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0043] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0044] Traditional CSP architecture for database access acceleration is as follows: Figure 1As shown in the figure, the Accel Card, based on the accelerator card architecture, is a card-based service that plugs into an X86 server (or other server). It generally includes a PCIe subsystem, a database acceleration engine, a UDP engine, and a memory engine. Client service initialization instructions are sent directly to the server, which initializes the accelerator card through the PCIe subsystem. The client then communicates with the accelerator card's database acceleration engine through the UDP engine to accelerate database services, typically only accelerating key-value (KV) access. Furthermore, this type of accelerator card often has a one-to-one relationship with the database, meaning one accelerator card corresponds to one data service. Therefore, large-scale data service scenarios often require multiple accelerator cards or even multiple servers.

[0045] Furthermore, traditional CSP architectures only include a memory engine, which accesses hot storage devices (volatile storage, such as DDR and SRAM) or on-chip hot storage (volatile storage, such as HBM) through corresponding controllers and storage access interfaces. However, these storage media have limited storage space, and for most business and processing data, they rely solely on the server's memory devices (such as mechanical hard drives and SSDs), meaning they cannot persist data themselves.

[0046] Figure 2 This is a diagram of the CSP architecture for database access acceleration provided by an embodiment of the present application. The CSP Architecture includes a database acceleration engine, a high-speed converged network subsystem, a storage access acceleration subsystem, a (customized) computing resource pool, and a database management subsystem.

[0047] Based on the high-speed Ethernet PHY layer + MAC layer, the high-speed converged network subsystem integrates a UDP / IP layer full offload engine (UWOE) to support business data transmission with the database controller. At the same time, it integrates a TOE engine to support management data interaction with the database management subsystem through the high-speed TCP / IP protocol. It has the advantages of low data latency and high-bandwidth transmission. This allows direct communication with the database management subsystem of the CSP architecture through the client. In other words, the client's initialization or access permission request is not sent to the server, and then the server notifies the accelerator card. Instead, the CSP Chip directly receives the client's initialization or access permission request, applies for database resources, and defines the usage type, computing, and storage resource allocation. This avoids the disadvantage of the existing architecture that can only conduct business, while management must be carried out through the server.

[0048] The database management subsystem responds to client requests for initializing database resources, access permission, and other requests, and configures multiple database services. It also internally configures and manages the high-speed converged network subsystem, database acceleration engine, storage access acceleration subsystem, and custom computing resource pool.

[0049] This CSP architecture can provide complete database access services with high concurrency, low latency, and high QPS (Query Per Second) without relying on servers. It has a shorter access path to solidified data and higher IOPS (Input / Output Operations Per Second). Compared with existing acceleration architectures, it improves processing performance and database compatibility while significantly reducing TCO (Total Cost of Ownership).

[0050] like Figure 3 As shown, the storage access acceleration subsystem includes a memory management and access engine and a storage management and access engine. The memory engine manages access to volatile storage devices, also known as the system's hot storage; the storage engine manages access to non-volatile storage, also known as the system's cold storage. The CSP architecture provides access interfaces for the memory engine and storage engine, enabling access to the corresponding volatile and non-volatile storage devices.

[0051] The Memory Engine is the same as the existing architecture, consisting of at least the HBM controller and its HBM access interface, the DDR controller and its DDR access interface, and the SRAM controller and its SRAM access interface. Different types of storage access interfaces connect to their respective storage devices. The Storage Engine includes at least the SATA controller and its SATA access interface, the NVMe controller and its NVMe access interface, and the Flash controller and its Flash access interface.

[0052] This CSP architecture is different from conventional computer devices equipped with hot and cold storage devices. Computer devices temporarily import data from storage memory into memory through the CPU accessing the file system, and the CPU application or CPU external accelerator card then accesses this data by accessing the temporary memory. The storage access acceleration subsystem in this embodiment can unify the address space, which is specifically controlled by the virtual address mapping space General interface of the storage access acceleration subsystem, the storage medium routing Task storage analyzer, and the database management subsystem. The virtual address mapping space is used to analyze the target task and occupied storage space and configuration. According to the storage medium mounted by the Memory engine and / or Storage engine and its used address space configuration mapping relationship table, all devices are mapped to a unified address continuous virtual address space to achieve virtual to physical space address conversion. The storage medium routing is used to determine the target storage controller and the corresponding access interface and storage space of the target storage medium based on the mapping relationship table, that is, data location, and forward the returned data to the corresponding virtual address space.

[0053] This design architecture implements a universal storage manager in the firmware to shield the differences between different storage media and achieve unified access to hot and cold storage on the system architecture side. The memory manager manages different storage media, and the database can be quickly deployed to different memories. In particular, for large-capacity non-volatile storage devices, data such as cache and logs generated by business execution can be stored, that is, client data can be solidified on the accelerator card without occupying server storage. Compared with the traditional accelerator card architecture, this step does not require the server to provide storage services, which reduces server overhead and latency.

[0054] The database acceleration engine includes several independent database controllers and a large number of database task pipelines, which are used to execute and allocate access request tasks for each user and call computing resources through the queue interface, making it easy to access different types of database services, support simultaneous access to multiple different databases, and support high concurrency and low latency access to a single database. Figure 4 As shown, it includes the database controller resource pool Database controller Pool, the database task pipline pool Database task pipline pool, the database acceleration engine storage access interface Memory & storage entries and the task routing controller Configurable dataswitch.

[0055] The database controller resource pool contains N independent database controllers, such as Database1controller, Database2controller, and DatabaseNcontroller. The database task thread pool contains M database task pipelines (M and N are positive integers, the number of which depends on the hardware configuration). The task routing controller, located between the database controller resource pool and the database task thread pool, allocates a target number of target database task pipelines to the selected target database controller, essentially establishing a virtual mapping relationship between the target database controller and the target database task pipeline.

[0056] The selection of pipelines can also be based on two specific requirements: access performance requirements and computing performance requirements. The number of pipelines is determined by the access performance or computing performance requirements of the corresponding database.

[0057] It should be noted that for multi-database processing scenarios with parallel processing, this CSP architecture is equipped with multiple database controllers, and each database controller independently manages a database. After receiving the user's request to initialize the database, the database management subsystem first needs to determine the priority of the target business and the required computing resources and storage space size. Because database controllers are independent of each other and can provide services for different databases respectively, different numbers of Piplines may be required for tasks for different databases. Therefore, this application needs to perform the following steps when determining the target database controller and the target database task pipeline:

[0058] a. Query the idle database controllers and idle database task pipelines in the database controller resource pool and database task thread pool respectively.

[0059] b. Determine the target database controller and its corresponding number of target database task pipelines according to the priority of the target task and the size of computing resources, and establish a virtual mapping relationship through the task routing controller.

[0060] In one possible implementation, a positive correlation function can be established between priority and the number of pipelines. This means that high-priority tasks are assigned first and receive a larger number of pipelines. Furthermore, the computing resources required for each task must be determined to ensure sufficient pipelines are available. Multi-objective tasks are processed in parallel.

[0061] Specifically, after receiving the client's access permission application request, the database management subsystem also confirms whether the client has access permission based on the permission management settings. If it is determined to have access permission, the UWOE filtering rules in the high-speed converged network subsystem are configured to allow the access traffic from this client IP to pass through and reach the requested database controller, thereby preventing traffic attacks.

[0062] After receiving a client's request for database resource usage, the database management subsystem checks whether there are idle database controllers in the database controller resource pool, whether the available storage space can meet the requested capacity, whether the number of idle database task execution flows in the database task thread pool meets the QPS performance requirements, and whether the custom computing resource pool can provide sufficient computing resources. If all of these conditions are met, the relevant resources are allocated, the database is initialized and configured, and these resources are combined to instantiate a separate database that meets the requested requirements.

[0063] The database acceleration engine storage access interface (Memory & storage entries) is the storage access interface used by the Database taskpipline to call. The selected pipeline establishes a communication connection with the storage access acceleration subsystem by calling this interface. Figure 3 As shown in .

[0064] The computing resource pool in the CSP architecture is a public computing resource that exists independently of the database task pipeline. The two are separated from each other. It contains resource data for processing various computing tasks and is called when each acceleration module performs computing business. Figure 7 As shown, it specifically includes a task forwarding module (including several task forwarding interfaces), a storage access transfer module (including several storage access transfer interfaces), several task queues for calling calculations, and SAND BOX sandboxes, etc. Because computing resources are usually exclusive in pipelines, but the mismatch of computing performance in different levels of pipeline calculations may lead to waste of computing resources. This architecture provides a computing resource pool for faster computing parts and computing parts that only some tasks contain. The pipeline threads call these computing resources through the queue interface. Ultimately, hardware resources are saved, so more pipelines can be placed to achieve the goal of higher performance. The computing resource pool allows users to define multiple common algorithms in different SAND BOXes according to business needs for task pipeline scheduling. The database management subsystem is specifically responsible for controlling resource scheduling.

[0065] Figure 5 This is a flowchart of the CSP architecture's response to a business request. The database acceleration engine's response to a database access task includes the following steps:

[0066] When responding to a target task, the target database controller selects a task from the task queue based on its priority and performs task analysis. It then estimates the command execution time based on database access volume, computational effort, and data volume. Based on the principles of traffic and time balance, the database controller dispatches the task to a specific task pipeline's task queue for execution, according to the task pipeline configuration and the corresponding task pipeline's working status.

[0067] b. The database task pipeline sequentially obtains tasks waiting to be executed from the queue. First, it performs a key-value search based on the target task instruction, calls the hash algorithm defined in the computing resource pool SAND BOX through the queue interface, and returns the calculation result to locate the location of the accessed related element information in the database.

[0068] KV retrieval is a database retrieval service that is mainly used to quickly retrieve or locate the location of related elements in the database. The process first performs index calculation, calls the specific hash calculation defined in the sandbox SAN BOX corresponding to the computing resource pool through the queue interface, and returns the index calculation result. Then, the element storage address is calculated based on the configuration information to locate the location of the relevant element information in the database. Optionally, in a multi-task parallel processing scenario, the calling computing resources are also set with a computing call queue and a computing result queue. The computing resource pool calls the computing resources according to the aforementioned priority and queue order and returns the corresponding results.

[0069] In some embodiments, for task pipelines that require disk write / modification, further write / modification management is required. Specifically, after performing a key-value (KV) search, write management operations are performed based on relevant element information, generating and storing write request tasks in a write request queue. This step involves invoking the database acceleration engine storage access interface to manage disk writes for the StorageEngine and / or Memory Engine. Fixed database controller operations correspond to the database serializing write requests to prevent write conflicts.

[0070] In other embodiments, if the database controller is initialized as a relational database, the task flow will perform relational operations (inter-set calculations) on the above-mentioned kv retrieval results, which can reduce repeated database query actions and alleviate the computing pressure brought by the client's relational operations; if the relational operations in the task flow are configured as bypass during the initialization process, the data skips this process and proceeds to the next level of processing.

[0071] In addition, when the target task includes data preprocessing calculations, the database task pipeline calls the calculation defined in the initialization phase in the corresponding sandbox SAN BOX in the computing resource pool through the queue interface to perform preprocessing calculations.

[0072] Optionally, when a target task is set with a filtering condition, the database task pipeline filters the calculation results / preprocessing results according to the preset conditions to generate redundant data and data irrelevant to the target task, thereby reducing network bandwidth.

[0073] c. Based on the obtained relevant element information and the returned calculation results, data is marked and packaged to build a response package. If necessary, the calculation defined by the sandbox SAN BOX corresponding to the computing resource pool will be called.

[0074] This step inputs raw data, which lacks any data tags or pointer information. Therefore, this step requires tag packaging to clearly indicate where the data is being sent and to notify the backend module or client of query success or failure, facilitating task status determination. Optionally, during the packaging process, to prevent data transmission errors, a packet verification function can be configured during initialization. This function uses the verification algorithm defined by the corresponding SAN BOX in the computing resource pool to calculate a checksum on the raw data and packages the result in the response packet.

[0075] d. Send the constructed response package to the task response queue.

[0076] It should be noted that Figure 5 The "Write Management", "Inter-Collection Computation", "Configurable Preprocessing (Business)", and "Configurable Filter" steps can be skipped during initialization based on actual task requirements.

[0077] In the above process, different database task pipelines will execute the tasks issued by the corresponding database controller in parallel, thereby improving task responsiveness, reducing the response delay of a single task, and preventing task execution blocking caused by a task executing for too long.

[0078] As mentioned above, the CSP Chip can provide one-to-many database services, but for cross-type or cross-manufacturer databases, as well as the instruction information format used by the client, the CSP Chip needs to have certain data language conversion capabilities. Because database operations mainly use the SQL language, SQL is a language designed for database programming, mainly used for accessing data, updating query data, and managing relational databases. Since its development, SQL has had multiple different versions of standards. The two main standards are SQL92 and SQL99. Although the standard versions have developed more, the syntax of the basic functions in the database is generally similar in each standard. In addition, the SQL statements supported by databases from different manufacturers are also different. The basic functions are the same, but the statements are written differently by each manufacturer.

[0079] like Figure 8As shown, to address these two issues, the CSP architecture was designed to support databases from various vendors. The firmware implements a virtual database translation dictionary abstraction layer. Its interface is primarily based on the SQL92 standard, providing users with common basic database operations. Based on the vendor's database being used, the database controller automatically adjusts the SQL statements supported by the database to the vendor's specific SQL expression format. All DataBase Tasks contain at least Access database statements and their corresponding Access-Compatible SQL translations, MySQL database statements and their corresponding MySQL-Compatible SQL translations, and Redis database statements and their corresponding Redis-Compatible SQL translations. During database initialization and configuration, the database management subsystem generates a command translation dictionary based on the requested database type, translates client commands into standard commands for this architecture, and sends them to the database controller. Upon receiving the task, the database controller translates the client command and sends the task packet containing the standard command to the task execution pipeline. This statement translation process is completed before the database controller sends the task to the task pipeline's task queue. The purpose is to convert between different database languages to facilitate subsequent data processing.

[0080] In summary, this application introduces a high-speed converged network subsystem into the CSP architecture to replace the PCIe subsystem and UDP engine in the traditional architecture. This supports low-latency, high-throughput transmission of service data with the database acceleration engine via UDP. At the same time, the integration of the TOE engine allows the client to communicate directly with the database management subsystem to respond to management requests, thus avoiding the drawback of the existing architecture that can only carry out business operations, while management must be carried out through the server's own CPU and network communication.

[0081] Adding a storage engine to the existing memory engine can mask differences between different storage media and achieve unified access to hot and cold storage on the system architecture side. Databases can be quickly deployed on various storage devices, and user-side data can be solidified without occupying server storage space. This also addresses insufficient server CPU IOPS capabilities.

[0082] The independent database controller and database task pipeline in the database acceleration engine can select the target database controller and allocate database task pipelines based on the priority of the target business; it allows the CSP architecture to provide different types of database services to multiple clients, solving the current architecture of one card corresponding to one service and avoiding resource waste; it provides higher storage density and higher access performance in the same space.

[0083] The database task pipeline is designed to be separate from the computing resource pool. When the database task pipeline needs to execute computing tasks, it calls on public computing resources. This achieves higher parallelism and freedom, conserving hardware resources, accommodating more pipelines, and achieving higher performance. With equivalent performance, the construction and maintenance costs are lower.

[0084] The abstraction layer built into this architecture enables cross-type and cross-vendor database conversion and execution, making it compatible with more application scenarios.

[0085] Figure 8 This is a flowchart of a database access process provided by an embodiment of the present application. The modules and systems described in the above aspects specifically include the following steps:

[0086] Step 1: The UWOE engine of the converged network subsystem receives the traffic packet;

[0087] Step 2: Determine whether it is a whitelist client and the access port is correct. If so, proceed to step 3. Otherwise, discard the traffic packet and do not process it, and continue to step 1.

[0088] Step 3: The database controller queries the task flow status of each database and writes it into a task flow task queue according to the task balance principle.

[0089] Step 4: The task execution pipeline polls the task queue.

[0090] Step 5: Determine whether there is a task. If yes, execute step 6; otherwise, continue polling.

[0091] Step 6: The task executes the pipeline reading task and calls the hash calculation for KV retrieval.

[0092] Step 7: Determine whether it is a write / modify task. If yes, execute step 8; otherwise, execute step 9.

[0093] Step 8: Generate a task and write it into the database write request queue.

[0094] Step 9: Determine whether the set operation is bypassed. If not, proceed to step 10. If yes, proceed to step 11.

[0095] Step 10: Perform set operations.

[0096] Step 11, determine whether the preprocessing operation is bypassed, if yes, execute step 12, otherwise execute step 13.

[0097] Step 12: Call the corresponding calculation in the computing resource pool.

[0098] Step 13: Determine whether data filtering is bypassed. If not, go to step 14; if yes, go to step 15.

[0099] Step 14: Filter the current result data.

[0100] Step 15: The final result forms a return packet, generates a return packet sending task and sends a request to the fusion network subsystem.

[0101] Step 16: Confirm that the result is sent correctly and the task execution pipeline returns to the IDLE state (step 4).

[0102] Figure 9 The following is a flowchart of database initialization provided by an embodiment of the present application, including the following steps:

[0103] Step 1: The database management subsystem receives a request from the client to create a database.

[0104] Step 2: Query and determine whether the idle resources meet the application. If so, execute step 3. Otherwise, send an application failure message and return to step 1 to wait for a new application.

[0105] Step 3: The database management subsystem initializes an idle database controller and sends the database command translation dictionary to this controller.

[0106] Step 4: The database management subsystem assigns several database task execution pipelines to the database controller and configures task routing to establish task transmission relationships.

[0107] Step 5: The database management subsystem allocates cold and hot storage resources to the database and maps them to the virtual address of the storage access acceleration subsystem, and configures the database controller and the corresponding task execution pipeline storage access address range.

[0108] In step 6, the database management subsystem configures the computing algorithm into the sandbox of the computing resource pool according to business requirements. If the same algorithm is shared with other controllers and the performance meets the requirements, there is no need to reconfigure it.

[0109] Step 7: The database management subsystem configures the task execution flow corresponding to this application and bypasses unnecessary processing.

[0110] Step 8: The database management subsystem configures the converged network subsystem UWOE to allow several clients to access the database.

[0111] Step 9: The database management subsystem returns a message indicating that the database has been successfully created to the client.

[0112] The above describes the preferred embodiments of the present invention; it should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can make many possible changes and modifications without departing from the technical solution of the present invention, or modify them into equivalent embodiments with equivalent changes, which does not affect the essential content of the present invention; therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention that do not depart from the content of the technical solution of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A CSP architecture for database access acceleration, characterized by: It includes database acceleration engine, high-speed converged network subsystem, storage access acceleration subsystem, computing resource pool, translation dictionary and database management subsystem; The high-speed converged network subsystem integrates a full offload engine at the UDP / IP layer and a TOE engine for directly responding to user requests and transmitting service data; The storage access acceleration subsystem includes a memory management access engine and a storage memory management access engine, which are used to provide an integrated virtual address mapping service, respectively perform access management on volatile storage and non-volatile storage devices, and solidify client data; The Memory memory management access engine and the Storage memory management access engine are respectively connected to storage devices of their respective types through corresponding storage access interfaces, and the non-volatile storage devices store the memory data and solidified data of the target database; The database management subsystem includes initialization configuration information for managing the database acceleration engine, and is used to start control and configure various database resources according to client requests, and simultaneously initialize configuration and operate the high-speed converged network subsystem, the database acceleration engine, and the storage access acceleration subsystem; The database acceleration engine includes several independent database controllers and a large number of database task pipelines, which are used to execute and distribute the requested tasks of each user and call computing resources through the queue interface, so that the database management subsystem can access different types of database services. The computing resource pool is separated from the database task pipeline and includes several task queues and SAND BOX sandboxes for calling calculations, which are used to define multiple public algorithms and call computing resources in different SAND BOXes when responding to target tasks; The translation dictionary at least includes Access database statements and corresponding Access Compatible SQL translation statements, MySQL database statements and corresponding MySQL Compatible SQL translation statements, redis database statements and corresponding redis Compatible SQL translation statements; When the database management subsystem performs database initialization configuration, it generates a command translation dictionary based on the requested database type and sends it to the database controller; After receiving the task, the database controller translates the client task command and sends the task package containing the standard command to the database task execution pipeline queue; When responding to the target task, the target database controller selects a task from the task queue according to the task priority and performs task analysis, and estimates the command execution time according to the database access volume, calculation volume, and data volume; The database task pipeline performs a kv search on the target task, calls the computing resource pool through the queue interface, and returns the calculation result to locate the position of the relevant element information in the database; Data tags and packaging are performed to construct response packages based on the acquired relevant element information and the returned calculation results; Send the constructed response package to the task response queue.

2. The CSP architecture for database access acceleration according to claim 1, characterized in that: The TOE engine integrated in the high-speed integrated network subsystem supports high-speed TCP / IP protocol, and controls the communication between the user terminal and the database management subsystem directly; the database management subsystem directly determines the applied database resources, usage type, computing and storage resource allocation based on the initialization or access permission request.

3. The CSP architecture for database access acceleration according to claim 2, characterized in that: The database acceleration engine includes a database controller resource pool, a database task thread pool, a database acceleration engine storage access interface of the database task thread pool, and a task routing controller; the database controller resource pool includes N independent database controllers, and the database task thread pool contains M database task pipelines; the task routing controller is located between the database controller resource pool and the database task thread pool, allocates a target number of target database task pipelines to the selected target database controller, and provides task instructions and return results.

4. The CSP architecture for database access acceleration according to claim 3, characterized in that: After receiving the user request, the database management subsystem determines the priority of the target business and the size of the required computing resources; Query the idle database controllers and idle database task pipelines in the database controller resource pool and the database task thread pool respectively; The target database controller and its corresponding number of target database task pipelines are determined according to the priority and computing resource size of the target task, and a virtual mapping relationship is established through the task routing controller; the multiple target tasks are processed in parallel, and the priority level is positively correlated with the number of task pipelines.

5. The CSP architecture for database access acceleration according to claim 3, characterized in that: The database acceleration engine storage access interface is used to connect to the storage access acceleration subsystem; the storage access acceleration subsystem also includes a virtual address mapping space and a storage medium router; The memory engine at least includes an HBM controller and its HBM access interface, a DDR controller and its DDR access interface, and an SRAM controller and its SRAM access interface; The Storage engine includes at least a SATA Controller and its SATA access interface, an NVMe Controller and its NVMe access interface, and a Flash Controller and its Flash access interface; The virtual address mapping space is used to analyze the target task and the occupied storage space and configuration, and map it to a unified virtual address space with continuous addresses according to the storage medium mounted by the Memory engine and / or the Storage engine and the address space used by the storage medium. The storage medium routing is used to determine the target storage controller and the corresponding access interface, the storage space of the target storage medium according to the mapping relationship table, and forward the returned data to the corresponding virtual address space.

6. The CSP architecture for database access acceleration according to claim 1, characterized in that: When the target task includes write and / or modify operations; after performing kv retrieval, perform write / modify management operations based on relevant element information, generate and store write request tasks in a write request queue; wherein, the database controller processes write / modify requests serially.

7. The CSP architecture for database access acceleration according to claim 6, characterized in that: When the initialized database is a relational database, the database task pipeline performs relational operations on the calculation results after kv retrieval to reduce repeated database query actions; when the target task includes data preprocessing calculations, the database task pipeline calls the computing resource pool through the queue interface to perform preprocessing calculations.

8. The CSP architecture for database access acceleration according to claim 7, characterized in that: When the target task is set with a screening condition, the database task pipeline screens the calculation results / preprocessing results according to the preset conditions to filter out redundant data and data irrelevant to the target task, thereby reducing network bandwidth.

Images