FPGA-based multi-database acceleration query method and device, equipment and medium

Abstract

The application discloses a FPGA-based multi-database acceleration query method and device, equipment and medium, and relates to the field of database application design, and comprises the following steps: determining to-be-processed data from to-be-processed data packets; determining an estimated execution time and a corresponding data processing difficulty of an instruction based on an instruction table, the to-be-processed data and a data processing instruction, allocating an acceleration unit to the to-be-processed data and marking an address to obtain a processing unit address; performing a format conversion operation on the processing unit address, the to-be-processed data and the data processing instruction to obtain converted data, performing sorting and screening processing on the converted data to obtain a processing result, and sending and saving the processing result to a first-in first-out queue; and sending the processing result in the first-in first-out queue to a memory so that the memory sends the processing result to a server for display, which can simultaneously perform data query on multiple databases, improve resource utilization, accelerate query efficiency and reduce power consumption of the query operation system.

Description

Technical Field

[0001] The present invention relates to the field of database application design, and in particular to a method, device, equipment and medium for accelerating querying multiple databases based on FPGA. Background Art

[0002] In a broad sense, a database is a "warehouse that organizes, stores, and manages data according to a data structure." It's a collection of large amounts of data stored on a server in an organized, shareable, and uniformly managed manner. As an integrated system for recording and storing data, a database employs a variety of models to organize data, storing data in rows, columns, tables, and other formats. Databases support comprehensive data storage, search, and analysis, and are now widely used in commercial, industrial, smart home, and healthcare applications.

[0003] With the advent of the artificial intelligence era and the continuous advancement of big data and informatization, installed database capacity has increased exponentially, and the demand for scenarios that require the joint invocation of multiple databases to perform data query processing is also increasing. Currently, database users mostly implement joint multi-database query processing using the server's CPU. However, the massive amount of data and complex calculations involved in simultaneous multi-database queries significantly impact the performance of the server's CPU, which is typically skilled in scheduling and management. This also significantly reduces the efficiency of user-side queries on multiple databases. Therefore, a new FPGA-based multi-database query acceleration engine urgently needs to be explored by researchers in the relevant field.

[0004] As can be seen from the above, how to implement simultaneous data query on multiple databases, improve resource utilization, speed up query efficiency, and reduce power consumption of query operations is a problem to be solved in this field. Summary of the Invention

[0005] In view of this, the present invention aims to provide a method, apparatus, device, and medium for accelerating multi-database queries based on FPGAs. This method enables simultaneous data queries across multiple databases, improves resource utilization, accelerates query efficiency, and reduces power consumption of the query operation system. The specific solution is as follows:

[0006] In a first aspect, the present application discloses an FPGA-based multi-database accelerated query method, which is applied to FPGA and includes:

[0007] Obtaining data processing instructions and data packets to be processed including data in each database, determining data to be processed from the data packets to be processed, and saving the data to be processed and the data processing instructions locally;

[0008] Determining an estimated instruction execution time and a corresponding data processing difficulty based on a preset instruction table, the data to be processed, and the data processing instruction, allocating an acceleration unit to the data to be processed according to the estimated instruction execution time and the corresponding data processing difficulty, and marking an address for the data to be processed to obtain a processing unit address;

[0009] Performing a format conversion operation on the processing unit address, the data to be processed, and the data processing instruction to obtain converted data, sorting and filtering the converted data according to the data processing instruction to obtain a processing result, and sending and saving the processing result to a local first-in-first-out queue;

[0010] The processing results in the first-in-first-out queue are sent to a preset memory, so that the memory sends the processing results to a server for display.

[0011] Optionally, determining the data to be processed from the data packets to be processed, and saving the data to be processed and the data processing instructions locally, includes:

[0012] Using a preset MIG IP core and FPGA sequential logic, determine the data to be processed from the data packets to be processed, unload the data packets to be processed, and then save the data to be processed to the local running memory;

[0013] The data processing instructions are saved in the local control execution memory.

[0014] Optionally, determining an estimated instruction execution time and a corresponding data processing difficulty based on a preset instruction table, the data to be processed, and the data processing instruction, allocating an acceleration unit to the data to be processed according to the estimated instruction execution time and the corresponding data processing difficulty, and marking an address for the data to be processed includes:

[0015] Based on the preset instruction table, the data to be processed and the data processing instructions, the estimated execution time of the instruction and the corresponding data processing difficulty are determined using the local arbitration module. According to the estimated execution time of the instruction and the corresponding data processing difficulty, an acceleration unit is allocated to the data to be processed, and the AXIslave address is marked for the data to be processed.

[0016] Optionally, performing a format conversion operation on the processing unit address, the data to be processed, and the data processing instruction to obtain converted data includes:

[0017] The processing unit address, the data to be processed, and the data processing instruction are converted into a standard AXI bus format using a local protocol conversion module to obtain converted data, and the converted data is sent to a local acceleration unit.

[0018] Optionally, the sorting and screening the converted data according to the data processing instruction to obtain a processing result, and sending and saving the processing result to a local first-in-first-out queue includes:

[0019] determining a data query relevance based on the data processing instruction;

[0020] The converted data is sorted and filtered according to the data query relevance and by using the acceleration unit to obtain a processing result, which is sent and saved to a local first-in-first-out queue via an AXI bus.

[0021] Optionally, obtaining the data processing instructions and the data packets to be processed including the data in each database includes:

[0022] Establishing a first connection relationship between a local device and a preset memory, and establishing a second connection relationship between the preset memory and a server through an XDMA interface, so that the server generates a data processing instruction based on an information processing request sent by a user terminal, then obtains a data packet to be processed from a third-party database, and uses the second connection relationship to send the data processing instruction and the data packet to be processed including data from each database to the memory;

[0023] The data processing instruction in the memory and the data packet to be processed including the data in each database are acquired according to the first connection relationship.

[0024] Optionally, sending the processing result in the first-in-first-out queue to a preset memory, so that the memory sends the processing result to a server for display, includes:

[0025] The processing results in the first-in-first-out queue are sent to a preset memory through the AXI bus, so that when the memory obtains the transmission signal sent by the server, the processing results are sent to the server through the second connection relationship, so that the server performs a secondary verification on the processing results. If the secondary verification passes, the processing results are sent to a preset display device for display.

[0026] In a second aspect, the present application discloses an FPGA-based multi-database accelerated query device, comprising:

[0027] a module for determining data to be processed, configured to obtain data processing instructions and data packets to be processed including data in each database, determine data to be processed from the data packets to be processed, and save the data to be processed and the data processing instructions locally;

[0028] an acceleration unit allocation module, configured to determine an estimated instruction execution time and a corresponding data processing difficulty based on a preset instruction table, the data to be processed, and the data processing instruction; allocate an acceleration unit to the data to be processed based on the estimated instruction execution time and the corresponding data processing difficulty; and mark an address for the data to be processed to obtain a processing unit address;

[0029] a format conversion module, configured to perform a format conversion operation on the processing unit address, the data to be processed, and the data processing instruction to obtain converted data, sort and filter the converted data according to the data processing instruction to obtain a processing result, and send and save the processing result to a local first-in-first-out queue;

[0030] The processing result display module is used to send the processing results in the first-in-first-out queue to a preset memory, so that the memory sends the processing results to the server for display.

[0031] In a third aspect, the present application discloses an electronic device, comprising:

[0032] Memory, used to store computer programs;

[0033] A processor is used to execute the computer program to implement the aforementioned FPGA-based multi-database accelerated query method.

[0034] In a fourth aspect, the present application discloses a computer storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned FPGA-based multi-database accelerated query method are implemented.

[0035] It can be seen that the present application provides an FPGA-based multi-database accelerated query method, including obtaining data processing instructions and a data packet to be processed including data in each database, determining the data to be processed from the data packet to be processed, and saving the data to be processed and the data processing instructions locally; determining the estimated execution time of the instruction and the corresponding data processing difficulty based on a preset instruction table, the data to be processed, and the data processing instructions, allocating an acceleration unit to the data to be processed according to the estimated execution time of the instruction and the corresponding data processing difficulty, and marking the address of the data to be processed to obtain the processing unit address; performing a format conversion operation on the processing unit address, the data to be processed, and the data processing instruction to obtain the converted data, sorting and filtering the converted data according to the data processing instruction to obtain the processing result, sending and saving the processing result to a local first-in-first-out queue; sending the processing result in the first-in-first-out queue to a preset memory so that the memory can send the processing result to the server for display. The present application is based on an FPGA-based multi-database query acceleration engine. The FPGA-based multi-database query acceleration engine not only supports high-speed data query requirements across multiple database types, but also improves resource utilization through FPGA module reuse. This reduces query operation time on the master control server CPU and also reduces the power consumption of the query operating system. Furthermore, the FPGA-based multi-database query acceleration engine can reconfigure acceleration modules based on actual user query needs, flexibly responding to query operation updates. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0037] Figure 1 This is a flow chart of a method for accelerating querying multiple databases based on FPGA disclosed in this application;

[0038] Figure 2 A schematic diagram of an FPGA structure and workflow disclosed in this application;

[0039] Figure 3 This is a diagram of a multi-database structure based on FPGA disclosed in this application;

[0040] Figure 4 This is a flow chart of a method for accelerating querying multiple databases based on FPGA disclosed in this application;

[0041] Figure 5 A specific structure and implementation mode of a multi-database system disclosed in the present application are shown in the figure;

[0042] Figure 6 A structure schematic diagram of a multi-database acceleration query device based on FPGA disclosed in the present application is shown in the figure;

[0043] Figure 7 A structure diagram of an electronic device provided in the present application is shown in the figure. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0045] A database in a broad sense is a "warehouse for organizing, storing and managing data according to a data structure", which is a long-term storage in a server, an organized, shareable and unified management of a large amount of data collection. As an integrated system for data record storage, the database uses a variety of different models to organize data, which stores data in different forms such as rows, columns and tables. The database supports all-round storage, search and analysis of data, and has been widely used in business, industry, smart home and medical care fields. With the advent of the era of artificial intelligence and the continuous advancement of the process of big data and informatization, the installed capacity of the database presents exponential growth, and the demand for multi-database joint call to realize data query processing is increasing day by day. At present, most database users realize joint query processing of multiple databases through the CPU of the server. However, the massive data and complex calculations involved in multi-database simultaneous query have a huge impact on the performance of the server-mounted CPU, which is good at scheduling and management, and the response efficiency of the user end to the multi-database query operation is also greatly reduced. Therefore, a new multi-database query acceleration engine based on FPGA is urgently needed for in-depth exploration by researchers in the relevant field. As can be seen from the above, how to realize simultaneous data query of multiple databases, improve resource utilization, speed up query efficiency and reduce query operation power consumption is a problem to be solved in the field.

[0046] The present invention relates to the field of database design, and more specifically, to a multi-database query acceleration engine based on an FPGA (Field Programmable Gate Array). This design, based on FPGA programmable array logic, implements a novel multi-database query acceleration engine to meet users' needs for high-speed queries on multiple databases simultaneously. The system implementation for multi-database query acceleration includes a master server containing multiple databases (DBs) and an FPGA-based multi-database query acceleration engine. The DB master server analyzes user needs and controls the implementation of the accelerator, focusing on leveraging the advantages of the CPU (Central Processing Unit) to achieve high-speed scheduling of instructions and data, while the FPGA-based multi-database query acceleration engine focuses on meeting the high-speed query processing requirements of the target databases. The master server and the FPGA-based multi-database query acceleration engine are connected via the XDMA protocol (XDMA connects the main memory of the DB master server and the DDR4 memory contained in the FPGA-based multi-database query acceleration engine, respectively, to achieve end-to-end direct high-speed data exchange and access). The physical layer of this protocol is implemented using PCIe 4.0 (PCI-Express, a high-speed serial bus). The XDMA protocol enables high-speed data transmission and exchange between the two. The multi-database query acceleration engine implemented on an FPGA consists of a Xilinx MIG DDR4 memory read controller, a data query controller, data RAM, control RAM, result RAM, a transfer arbiter, FIFOs, and a data transfer path. The multi-database master control server terminal controls the FPGA-based multi-database query acceleration engine via the XDMA interface.

[0047] See also Figure 1 As shown, the embodiment of the present invention discloses an FPGA-based multi-database accelerated query method, which is applied to FPGA and may specifically include:

[0048] Step S11: obtaining data processing instructions and data packets to be processed including data in each database, determining data to be processed from the data packets to be processed, and saving the data to be processed and the data processing instructions locally.

[0049] In this embodiment, data processing instructions and a data packet to be processed containing data from each database are obtained. Then, using a preset MIG IP core and FPGA timing logic, the data to be processed is determined from the data packet to be processed, the data packet to be processed is unloaded, and the data packet to be processed, including data from each database, is saved to local run memory, while the data processing instructions are saved to local control run memory. In other words, after obtaining the data processing instructions and the data packet to be processed containing data from each database, the data packet to be processed, transmitted by the host via XDMA, is unloaded from the memory, and the data processing instructions and the data to be processed, including data from each database, are stored in the data RAM module and the control RAM module, respectively. This stage is completed by the MIG IP core officially provided by Xilinx and implemented in conjunction with appropriate FPGA timing logic.

[0050] Step S12: Based on the preset instruction table, the data to be processed and the data processing instructions, the estimated execution time of the instruction and the corresponding data processing difficulty are determined, and an acceleration unit is allocated to the data to be processed according to the estimated execution time of the instruction and the corresponding data processing difficulty, and the address of the data to be processed is marked to obtain the processing unit address.

[0051] In this embodiment, based on the preset instruction table, the data to be processed and the data processing instructions, the estimated execution time of the instruction and the corresponding data processing difficulty are determined by using a local arbitration module. According to the estimated execution time of the instruction and the corresponding data processing difficulty, an acceleration unit is allocated to the data to be processed, and the AXI slave address is marked for the data to be processed to obtain the processing unit address. The specific structure and workflow of FPGA are as follows. Figure 2 As shown, the control instruction table identifies the multi-database query operation instructions (i.e., data processing instructions) stored in the control RAM, allocates acceleration units according to the estimated execution time of the instructions and the corresponding data processing difficulty, and marks the AXI slave address for the data stream. This operation is implemented by the local arbitration module.

[0052] Step S13: Perform format conversion on the processing unit address, the data to be processed, and the data processing instruction to obtain converted data, sort and filter the converted data according to the data processing instruction to obtain a processing result, and send and save the processing result to a local first-in-first-out queue.

[0053] In this embodiment, a local protocol conversion module is used to convert the processing unit address, the data to be processed, and the data processing instruction into a standard AXI bus format to obtain converted data, and the converted data is sent to a local acceleration unit, and then the converted data is sorted and filtered according to the data processing instruction to obtain a processing result, and the processing result is sent and saved to a first-in-first-out queue in the data processing module. The specific sorting and filtering processing process is as follows: based on the data processing instruction, the data query relevance is determined, and then the converted data is sorted and filtered according to the data query relevance and using the acceleration unit to obtain a processing result, and the processing result is sent and saved to a local first-in-first-out queue via the AXI bus. Figure 2 As shown, the data RAM and control RAM, along with the data processing instructions and processing unit addresses, stored in the data RAM and control RAM, are converted into a standard AXI bus format. This operation is implemented by a protocol conversion module (AXI BUS module) between the two. The converted data packets are sent via the AXI bus to the data packet acceleration units (the acceleration units are considered AXI slaves, AXI Lite 0 to AXI Lite 15) determined by the arbitration module for sorting and screening. Different query functions are then implemented as required. The acceleration units, in accordance with the data processing instructions stored in the control RAM, first aggregate data from multiple databases (Data Mix module), then sort them according to their query relevance (Sorting module), and filter out data that meets the query requirements according to the relevance (Filter module). Finally, the results of the relevance sorting (i.e., the processing results) are stored in the FIFOs (Result FIFO_0 to Result FIFO_7).

[0054] The structure of the FPGA-based multi-database system is as follows: Figure 3 As shown, when a user needs to query a database, the database application initiates a read instruction for multiple databases (i.e., Database 1-Database N) at the DB master control server end of the data application layer; then, the DB master control reads all the contents in the corresponding storage partitions of the multiple databases. Due to the large amount of data (usually tens of thousands of items), it is temporarily stored in the server cache waiting for data processing instructions; after the data is completely read, the CPU of the DB master control server performs scheduling control, packages the data processing instructions and the pending data in the server cache and sends them to the multi-database query acceleration engine (the multi-database query acceleration engine includes DDR4 and FPGA); the FPGA-based multi-database query acceleration engine performs parallel accelerated aggregation, sorting, and screening processing on the data according to the data processing instructions, and promptly feeds back the processing results to the DB master control server to complete the query acceleration operation and display it on the server.

[0055] Step S14: sending the processing results in the first-in-first-out queue to a preset memory, so that the memory sends the processing results to the server for display.

[0056] In this embodiment, Figure 2 As shown, the processing results are sent to the processing result memory (Result RAM module) through the AXI bus in sequence, and then transmitted to the DDR4 storage by the AXI bus, and finally sent to the server through the XDMA port for display.

[0057] In this embodiment, data processing instructions and data packets to be processed including data from each database are obtained, data to be processed is determined from the data packets to be processed, and the data to be processed and the data processing instructions are saved locally; based on a preset instruction table, the data to be processed and the data processing instructions, an estimated execution time of the instructions and the corresponding data processing difficulty are determined, an acceleration unit is allocated to the data to be processed according to the estimated execution time of the instructions and the corresponding data processing difficulty, and an address is marked for the data to be processed to obtain a processing unit address; a format conversion operation is performed on the processing unit address, the data to be processed and the data processing instructions to obtain converted data, the converted data is sorted and filtered according to the data processing instructions to obtain a processing result, and the processing result is sent and saved to a local first-in-first-out queue; the processing result in the first-in-first-out queue is sent to a preset memory so that the memory can send the processing result to a server for display. This application is based on an FPGA-based multi-database query acceleration engine. The FPGA-based multi-database query acceleration engine not only supports high-speed data query requirements across multiple database types, but also improves resource utilization through FPGA module reuse. This reduces query operation time on the master control server CPU and also reduces the power consumption of the query operating system. Furthermore, the FPGA-based multi-database query acceleration engine can reconfigure acceleration modules based on actual user query needs, flexibly responding to query operation updates.

[0058] See also Figure 4 As shown, the embodiment of the present invention discloses a method for accelerating querying multiple databases based on FPGA, which may specifically include:

[0059] Step S21: Establish a first connection relationship between the local and a preset memory, and establish a second connection relationship between the preset memory and a server through an XDMA interface, so that the server generates a data processing instruction based on an information processing request sent by the user end, obtains a to-be-processed data packet from a third-party database, and then sends the data processing instruction and the to-be-processed data packet including data in each database to the memory by using the second connection relationship, and acquires the data processing instruction and the to-be-processed data packet including data in each database in the memory according to the first connection relationship.

[0060] Step S22: Determine the to-be-processed data from the to-be-processed data packet, and save the to-be-processed data and the data processing instruction to the local.

[0061] Step S23: Determine the estimated execution time of the instruction and the corresponding data processing difficulty based on a preset instruction table, the to-be-processed data, and the data processing instruction, allocate an acceleration unit to the to-be-processed data according to the estimated execution time of the instruction and the corresponding data processing difficulty, and label the address of the to-be-processed data to obtain a processing unit address.

[0062] Step S24: Perform a format conversion operation on the processing unit address, the to-be-processed data, and the data processing instruction to obtain converted data, perform sorting and screening processing on the converted data according to the data processing instruction to obtain a processing result, and send and save the processing result to a first-in-first-out queue in the local.

[0063] Step S25: Send the processing result in the first-in-first-out queue to a preset memory through an AXI bus, so that when the memory acquires a transmission signal sent by the server, the processing result is sent to the server through the second connection relationship, so that the server performs secondary verification on the processing result, and if the secondary verification passes, the processing result is sent to a preset display device for display.

[0064] The implementation method of the present application is as follows: 1. The user issues a query processing demand based on certain information to multiple databases at the server terminal. The multi-database application of the server terminal receives the accurate information query instruction and sends it to the Database controller located at the server terminal; 2. The Database controller controls the server terminal to read the relevant stored information from multiple databases respectively, and sends the unprocessed read information received from multiple databases to the server main memory, and the main memory temporarily stores it. The amount of unprocessed information is huge and cannot be directly displayed to the user without processing; 3. After all the relevant information collected from multiple databases is stored in the server main memory, the cached data and query processing instructions are packaged by the server CPU and directly The data is sent to the DDR4 temporary storage of the FPGA multi-database query acceleration engine through the XDMA interface; 4. After the DDR4 data is received, the FPGA chip of the multi-database query acceleration engine reads the instruction information and corresponding unprocessed data stored in the DDR4 memory, and performs high-speed parallel aggregation sorting and filtering processing on the read data as required; 5. The FPGA stores the processing results again in the specified DDR4 storage partition; 6. The server CPU receives the accelerated processing results stored in the specific DDR4 partition through the XDMA interface and directly maps the results to the server main memory; 7. The database controller performs a secondary check on the processing results of the server main memory, outputs them to the user end, and displays the query results through the query result display device. The FPGA module processing process of the multi-database query acceleration engine can be divided into four stages: (1) DDR4 storage data unloading; (2) accelerated processing data arbitration and processing unit allocation; (3) bus format conversion; (4) data query acceleration processing. Specifically, in the first stage, DDR4 storage data unloading. The data packets to be processed transmitted by the host through XDMA are unloaded from the memory and stored in the data RAM (Data RAM module) and control RAM (Control RAM module) respectively. This phase is completed using the MIGIP core officially provided by Xilinx, implemented in conjunction with appropriate FPGA timing logic. The second phase involves accelerated data arbitration and processing unit allocation. The instruction table identifies the multi-database query operation instructions stored in the control RAM. Accelerator units are assigned to the data to be processed based on the estimated execution time and the corresponding data processing difficulty. The data stream is then marked with AXI slave addresses. This operation is implemented by the arbitration module. The third phase involves bus format conversion.The data and operation instruction data streams stored in the data RAM and control RAM are converted into a standard AXI bus format. This operation is implemented by the protocol conversion module (AXI BUS module) between the two. The converted data packets are sent via the AXI bus to the packet acceleration unit (the acceleration unit is considered an AXI slave, AXI Lite 0 to AXI Lite 15), which is determined by the arbitration module for aggregation, sorting, and filtering. In the fourth stage, different query functions are implemented according to the requirements. Based on the operation content stored in the control RAM, the acceleration unit first aggregates data from multiple databases (Data Mix module), then sorts the data by relevance (Sorting module), and filters the data that meets the query requirements based on relevance (Filter module). Finally, the relevance-sorted results are stored in FIFOs (Result FIFO_0 to Result FIFO_7). The processing results are sequentially sent via the AXI bus to the processing result memory (Result RAM module), and then transferred to DDR4 storage via the AXI bus. Finally, the XDMA port sends them to the host.

[0065] The specific structure and implementation of the multi-database system are as follows Figure 5As shown, the structure: multi-database query acceleration system implementation mode includes a multi-database DB master server and a multi-database query acceleration engine based on FPGA, the DB master server analyzes user requirements and controls the implementation of the accelerator, the focus of its work is to give full play to the advantages of CPU, realize the high-speed scheduling of instructions and data, and the focus of the multi-database query acceleration engine based on FPGA is to realize the high-speed query processing requirements of the target database. The master server and the multi-database query acceleration engine based on FPGA are connected through the XDMA protocol (XDMA connects the main memory of the DB master server and the DDR4 memory contained in the multi-database query acceleration engine based on FPGA, realizes the end-to-end direct high-speed data exchange and access of the memory), the physical layer of the protocol is realized by PCIe4.0, and the XDMA protocol realizes the high-speed transmission and exchange operation of the data of the two. The multi-database query acceleration engine implemented on the FPGA is composed of Xilinx MIG DDR4 memory reading controller, data query overall controller, data RAM, control RAM, result RAM, transmission arbitrator, FIFO and data transmission path and other main components. Implementation mode: 1, the user issues a query processing requirement based on certain information of multiple databases at the server terminal, the multi-database application program of the server terminal receives the accurate information query instruction and sends it to the Database controller located at the server terminal; 2, the Database controller controls the server terminal to read related storage information from multiple databases respectively, and sends the unprocessed read information received from multiple databases to the server main memory for temporary storage, the unprocessed information is large in quantity and cannot be directly displayed to the user without processing; 3, after the related information collected from multiple databases is all stored in the server main memory, the cached data and query processing instructions are packaged by the server CPU and sent directly to the DDR4 temporary storage carried by the FPGA multi-database query acceleration engine through the XDMA interface; 4, after the DDR4 data is received, the FPGA chip of the multi-database query acceleration engine reads the instruction information and the corresponding unprocessed data stored in the DDR4 memory, and performs high-speed parallel aggregation sorting and screening processing on the read data according to the requirements; 5, the FPGA stores the processing result in the specified storage partition of the DDR4 again; 6, the server CPU receives the accelerated processing result stored in the specified partition of the DDR4 through the XDMA interface, and maps the result directly to the server main memory; 7 and 8, the Database controller performs secondary verification on the processing result of the server main memory, outputs to the user end, and displays the query result through the query result display device.

[0066] In the embodiment, a data processing instruction and a to-be-processed data packet including data in each database are acquired, to-be-processed data is determined from the to-be-processed data packet, and the to-be-processed data and the data processing instruction are saved to the local; an estimated execution time of an instruction and a corresponding data processing difficulty are determined based on a preset instruction table, the to-be-processed data and the data processing instruction, an acceleration unit is allocated for the to-be-processed data according to the estimated execution time of the instruction and the corresponding data processing difficulty, and an address is labeled for the to-be-processed data to obtain a processing unit address; a format conversion operation is performed on the processing unit address, the to-be-processed data and the data processing instruction to obtain converted data, the converted data is sorted and filtered according to the data processing instruction to obtain a processing result, and the processing result is sent and saved to a first-in first-out queue in the local; and the processing result in the first-in first-out queue is sent to a preset memory, so that the memory sends the processing result to a server for display. The application is a multi-database query acceleration engine based on FPGA. The multi-database query acceleration engine based on FPGA not only supports high-speed query requirements of data in multiple different types of databases, but also improves the utilization rate of resources through FPGA module reuse; reduces the time consumption of CPU query operation of the total control server and also reduces the power consumption of the query operation system. In addition, the multi-database query acceleration engine based on FPGA can also be reconfigured according to the actual needs of user query to flexibly cope with query operation updates.

[0067] Referring to Figure 6 The embodiment of the application discloses a multi-database acceleration query device based on FPGA, which can specifically include:

[0068] The to-be-processed data determination module 11 is configured to acquire a data processing instruction and a to-be-processed data packet including data in each database, determine to-be-processed data from the to-be-processed data packet, and save the to-be-processed data and the data processing instruction to the local;

[0069] The acceleration unit allocation module 12 is configured to determine an estimated execution time of an instruction and a corresponding data processing difficulty based on a preset instruction table, the to-be-processed data and the data processing instruction, allocate an acceleration unit for the to-be-processed data according to the estimated execution time of the instruction and the corresponding data processing difficulty, and label an address for the to-be-processed data to obtain a processing unit address;

[0070] The format conversion module 13 is configured to perform a format conversion operation on the processing unit address, the to-be-processed data and the data processing instruction to obtain converted data, sort and filter the converted data according to the data processing instruction to obtain a processing result, send and save the processing result to a first-in first-out queue in the local; and send the processing result in the first-in first-out queue to a preset memory, so that the memory sends the processing result to a server for display.

[0071] The processing result display module 14 is used to send the processing results in the first-in-first-out queue to a preset memory, so that the memory sends the processing results to the server for display.

[0072] In this embodiment, data processing instructions and data packets to be processed including data from each database are obtained, data to be processed is determined from the data packets to be processed, and the data to be processed and the data processing instructions are saved locally; based on a preset instruction table, the data to be processed and the data processing instructions, an estimated execution time of the instructions and the corresponding data processing difficulty are determined, an acceleration unit is allocated to the data to be processed according to the estimated execution time of the instructions and the corresponding data processing difficulty, and an address is marked for the data to be processed to obtain a processing unit address; a format conversion operation is performed on the processing unit address, the data to be processed and the data processing instructions to obtain converted data, the converted data is sorted and filtered according to the data processing instructions to obtain a processing result, and the processing result is sent and saved to a local first-in-first-out queue; the processing result in the first-in-first-out queue is sent to a preset memory so that the memory can send the processing result to a server for display. This application is based on an FPGA-based multi-database query acceleration engine. The FPGA-based multi-database query acceleration engine not only supports high-speed data query requirements across multiple database types, but also improves resource utilization through FPGA module reuse. This reduces query operation time on the master control server CPU and also reduces the power consumption of the query operating system. Furthermore, the FPGA-based multi-database query acceleration engine can reconfigure acceleration modules based on actual user query needs, flexibly responding to query operation updates.

[0073] In some specific embodiments, the to-be-processed data determination module 11 may specifically include:

[0074] A module for storing data to be processed, which is used to use a preset MIG IP core and FPGA sequential logic to determine the data to be processed from the data packets to be processed, unload the data packets to be processed, and then save the data to be processed to a local running memory;

[0075] The data processing instruction storage module is used to save the data processing instruction to the local control execution memory.

[0076] In some specific embodiments, the acceleration unit allocation module 12 may specifically include:

[0077] An acceleration unit allocation module is used to determine the estimated instruction execution time and the corresponding data processing difficulty based on a preset instruction table, the data to be processed, and the data processing instructions, and using a local arbitration module. According to the estimated instruction execution time and the corresponding data processing difficulty, an acceleration unit is allocated to the data to be processed, and an AXI slave address is marked for the data to be processed.

[0078] In some specific embodiments, the format conversion module 13 may specifically include:

[0079] The format conversion module is used to convert the processing unit address, the data to be processed and the data processing instruction into a standard AXI bus format using a local protocol conversion module to obtain converted data, and send the converted data to a local acceleration unit.

[0080] In some specific embodiments, the processing result display module 14 may specifically include:

[0081] A data query relevance determination module, configured to determine the data query relevance based on the data processing instruction;

[0082] The processing result sending module is used to query the relevance of the data and use the acceleration unit to sort and filter the converted data to obtain a processing result, and send the processing result through the AXI bus and save it to a local first-in-first-out queue.

[0083] In some specific embodiments, the to-be-processed data determination module 11 may specifically include:

[0084] a connection relationship establishing module, configured to establish a first connection relationship between a local device and a preset memory, and to establish a second connection relationship between the preset memory and a server via an XDMA interface, so that the server generates a data processing instruction based on an information processing request sent by a user terminal, then obtains a data packet to be processed from a third-party database, and transmits the data processing instruction and the data packet to be processed including data from each database to the memory via the second connection relationship;

[0085] The module for obtaining data packets to be processed is configured to obtain the data processing instructions in the memory and the data packets to be processed including data in each database according to the first connection relationship.

[0086] In some specific embodiments, the processing result display module 14 may specifically include:

[0087] The processing result display module is configured to send the processing result in the first-in first-out queue to a preset memory through an AXI bus, so that when the memory acquires a transmission signal sent by the server, the processing result is sent to the server through the second connection relationship, so that the server performs secondary verification on the processing result, and if the secondary verification passes, the processing result is sent to a preset display device for display.

[0088] Figure 7 A structural schematic diagram of an electronic device provided in the embodiment of the present application is provided. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the related steps in the FPGA-based multi-database accelerated query method performed by the electronic device disclosed in any of the foregoing embodiments.

[0089] In the embodiment, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not specifically limited herein; the input / output interface 25 is configured to acquire external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not specifically limited herein.

[0090] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon include an operating system 221, a computer program 222 and data 223, etc., and the storage mode can be temporary storage or permanent storage.

[0091] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to realize the operation and processing of the processor 21 on the data 223 in the memory 22, and the operating system 221 can be Windows, Unix, Linux, etc. The computer program 222 can further include computer programs for completing other specific work in addition to the computer program for completing the FPGA-based multi-database accelerated query method performed by the electronic device 20 disclosed in any of the foregoing embodiments. The data 223 can include data transmitted by external devices received by the FPGA-based multi-database accelerated query device, and can also include data collected by the input / output interface 25 itself, etc.

[0092] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0093] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the FPGA-based multi-database accelerated query method disclosed in any of the aforementioned embodiments are implemented.

[0094] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0095] The above is a detailed introduction to the FPGA-based multi-database accelerated query method, device, equipment and storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A multi-database accelerated query method based on FPGA, characterized in that: Applications in FPGAs include: Obtaining data processing instructions and data packets to be processed including data in each database, determining data to be processed from the data packets to be processed, and saving the data to be processed and the data processing instructions locally; Determining an estimated instruction execution time and a corresponding data processing difficulty based on a preset instruction table, the data to be processed, and the data processing instruction, allocating an acceleration unit to the data to be processed according to the estimated instruction execution time and the corresponding data processing difficulty, and marking an address for the data to be processed to obtain a processing unit address; Performing a format conversion operation on the processing unit address, the data to be processed, and the data processing instruction to obtain converted data, sorting and filtering the converted data according to the data processing instruction to obtain a processing result, and sending and saving the processing result to a local first-in-first-out queue; The processing results in the first-in-first-out queue are sent to a preset memory, so that the memory sends the processing results to a server for display.

2. The FPGA-based multi-database accelerated query method according to claim 1, characterized in that: The determining the data to be processed from the data packets to be processed, and saving the data to be processed and the data processing instructions locally, includes: Using a preset MIG IP core and FPGA sequential logic, determine the data to be processed from the data packets to be processed, unload the data packets to be processed, and then save the data to be processed to the local running memory; The data processing instructions are saved in the local control execution memory.

3. The FPGA-based multi-database accelerated query method according to claim 1, characterized in that: The method includes determining an estimated instruction execution time and a corresponding data processing difficulty based on a preset instruction table, the data to be processed, and the data processing instruction, allocating an acceleration unit to the data to be processed according to the estimated instruction execution time and the corresponding data processing difficulty, and marking an address for the data to be processed, including: Based on the preset instruction table, the data to be processed and the data processing instructions, the estimated execution time of the instruction and the corresponding data processing difficulty are determined using a local arbitration module. According to the estimated execution time of the instruction and the corresponding data processing difficulty, an acceleration unit is allocated to the data to be processed, and an AXI slave address is marked for the data to be processed.

4. The FPGA-based multi-database accelerated query method according to claim 1, characterized in that: The performing a format conversion operation on the processing unit address, the data to be processed, and the data processing instruction to obtain converted data includes: The processing unit address, the data to be processed, and the data processing instruction are converted into a standard AXI bus format using a local protocol conversion module to obtain converted data, and the converted data is sent to a local acceleration unit.

5. The FPGA-based multi-database accelerated query method according to claim 4, characterized in that: The sorting and screening of the converted data according to the data processing instruction to obtain a processing result, and sending and saving the processing result to a local first-in-first-out queue, includes: determining a data query relevance based on the data processing instruction; The converted data is sorted and filtered according to the data query relevance and by using the acceleration unit to obtain a processing result, which is sent and saved to a local first-in-first-out queue via an AXI bus.

6. The FPGA-based multi-database accelerated query method according to any one of claims 1 to 5, characterized in that: The step of obtaining the data processing instructions and the data packets to be processed including the data in each database includes: Establishing a first connection relationship between a local device and a preset memory, and establishing a second connection relationship between the preset memory and a server through an XDMA interface, so that the server generates a data processing instruction based on an information processing request sent by a user terminal, then obtains a data packet to be processed from a third-party database, and uses the second connection relationship to send the data processing instruction and the data packet to be processed including data from each database to the memory; The data processing instruction in the memory and the data packet to be processed including the data in each database are acquired according to the first connection relationship.

7. The FPGA-based multi-database accelerated query method according to claim 6, characterized in that: The sending the processing results in the first-in-first-out queue to a preset memory, so that the memory sends the processing results to a server for display, includes: The processing results in the first-in-first-out queue are sent to a preset memory through the AXI bus, so that when the memory obtains the transmission signal sent by the server, the processing results are sent to the server through the second connection relationship, so that the server performs a secondary verification on the processing results. If the secondary verification passes, the processing results are sent to a preset display device for display.

8. An FPGA-based multi-database accelerated query device, characterized in that: include: a module for determining data to be processed, configured to obtain data processing instructions and data packets to be processed including data in each database, determine data to be processed from the data packets to be processed, and save the data to be processed and the data processing instructions locally; an acceleration unit allocation module, configured to determine an estimated instruction execution time and a corresponding data processing difficulty based on a preset instruction table, the data to be processed, and the data processing instruction; allocate an acceleration unit to the data to be processed based on the estimated instruction execution time and the corresponding data processing difficulty; and mark an address for the data to be processed to obtain a processing unit address; a format conversion module, configured to perform a format conversion operation on the processing unit address, the data to be processed, and the data processing instruction to obtain converted data, sort and filter the converted data according to the data processing instruction to obtain a processing result, and send and save the processing result to a local first-in-first-out queue; The processing result display module is used to send the processing results in the first-in-first-out queue to a preset memory, so that the memory sends the processing results to the server for display.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the FPGA-based multi-database accelerated query method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store computer programs; wherein, when the computer program is executed by a processor, it implements the FPGA-based multi-database accelerated query method according to any one of claims 1 to 7.

Images