Database data format conversion system
By introducing an FPGA computing kernel into the database to perform data row and column format conversion, the problem of high computing power and time consumption caused by format conversion in data query tasks is solved, thereby improving database performance and response speed.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG INSPUR SCI RES INST CO LTD
- Filing Date
- 2024-04-18
- Publication Date
- 2026-07-31
AI Technical Summary
Databases consume excessive computing power and time due to data row and column format conversions during data query tasks, increasing performance barriers.
An FPGA computing core is used to perform data row and column format conversion tasks, reducing the processing burden on the database server CPU. It connects to the processor through a PCIe interface and uses modules such as input recognition, data filtering, row splitting, and column concatenation to perform data format conversion.
It reduces the computing power overhead of the database server, improves the efficiency and response speed of data queries, saves CPU resources, and reduces the format conversion of invalid data.
Smart Images
Figure CN118503315B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of database technology, and in particular to a database data format conversion system. Background Technology
[0002] With the continuous development of artificial intelligence and the information society, the types and installed capacity of databases in various industries are increasing daily, and the data storage methods of databases are becoming more diversified. Currently, there are two main data storage modes involved in databases: row-based storage and column-based storage. Row-based storage stores each row of data in a data block, with each block stored contiguously; column-based storage stores each column of data together, but different columns can be stored separately. When a database server CPU executes SQL data query tasks, it typically processes data by row or column. For example, aggregation is usually processed by column indexes, while filtering and sorting queries are usually processed by row indexes. Converting the row / column stored data read from memory into the predetermined row / column format required by the query task generates enormous database CPU computing power and time consumption, exacerbating the database performance bottleneck problem. Summary of the Invention
[0003] This invention provides a database data format conversion system to solve the problem that in the prior art, the data row and column format conversion involved in database data query tasks leads to a significant loss of computing power and time, exacerbating the performance barrier problem of the database.
[0004] This invention provides a database data format conversion system, including a database control server and a data storage engine, wherein:
[0005] The database control server includes a processor, and the data storage engine includes a format conversion module and a data storage array;
[0006] The processor is used to receive and parse data query instructions sent by the user terminal, and determine the required format and storage partition address of the data to be queried by the user.
[0007] The format conversion module is used to read the data to be processed from the data storage array according to the storage partition address, and when the target format of the data to be processed is different from the required format, convert the data format of the data to be processed into the required format, and send the target data to the processor.
[0008] The processor sends the target data to the user terminal in response to the data query command.
[0009] According to the database data format conversion system provided by the present invention, the format conversion module includes an FPGA computing core, which is connected to the processor via a PCIe interface; the FPGA computing core includes:
[0010] An input recognition module is used to compare the target format of the data to be processed with the required format to determine whether the target format of the data to be processed is the same as the required format.
[0011] A data filtering module is used to filter the data to be processed;
[0012] A data storage module is used to store the target data to be sent.
[0013] According to the data format conversion system for the database provided by the present invention, the FPGA computing core further includes a row splitting module, a column sending module, and several first buffer queues, and the data filtering module includes a row filtering unit; when the required format is column format and the target format of the data to be processed is row format:
[0014] The row filtering unit is used to perform row filtering on the data to be processed.
[0015] The row splitting module is used to split the filtered data into multiple target columns.
[0016] The plurality of first cache queues are used to cache the target column data;
[0017] The column sending module is used to send the target column data to the data storage module for caching.
[0018] According to the data format conversion system for the database provided by the present invention, the FPGA computing core further includes a column concatenation module, a row sending module, and several second buffer queues, and the data filtering module includes a column filtering unit; when the required format is a row format and the target format of the data to be processed is a column format:
[0019] The column filtering unit is used to perform column filtering on the data to be processed;
[0020] The plurality of second cache queues are used to cache the data to be processed after column filtering;
[0021] The column concatenation module is used to perform column concatenation processing on the data to be processed cached in the plurality of second cache queues to obtain the target row data;
[0022] The row sending module is used to send the target row data to the data storage module for caching.
[0023] According to the data format conversion system of the database provided by the present invention, the FPGA computing core further includes an interface sending module for sending the data stored in the data storage module to the processor through the PCIe interface.
[0024] According to the data format conversion system for a database provided by the present invention, the data filtering module includes a row and column filtering unit, and the FPGA computing core further includes a row and column sending module, wherein when the required format is the same as the target format of the data to be processed:
[0025] The row and column filtering unit is used to filter the data to be processed to obtain the target data;
[0026] The row and column sending module is used to send the target data to the interface sending module.
[0027] In the database data format conversion system provided by the present invention, the input recognition module is further configured to determine the data conversion frequency according to the data query instruction.
[0028] According to the data format conversion system of the database provided by the present invention, the FPGA computing core further includes a clock generation module for generating a clock signal based on the data conversion frequency.
[0029] According to the data format conversion system of the database provided by the present invention, the FPGA computing kernel further includes a format conversion control module for generating scheduling control signals for the FPGA computing kernel.
[0030] According to the data format conversion system of the database provided by the present invention, the data storage engine further includes a power supply module and a crystal oscillator module of the FPGA computing core; the FPGA computing core further includes a set / reset / debugging interface.
[0031] The database data format conversion system provided by this invention performs data row and column format conversion tasks involved in the data query process through the FPGA computing kernel. The format conversion tasks that originally needed to be performed by the processor of the database control server are transferred to the FPGA computing kernel, which reduces the computing power overhead of the database control server, saves the computing power resources of the database server, and helps to improve the performance of the database. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0033] Figure 1 This is a schematic diagram of the structure of the database data format conversion system provided in this embodiment of the invention;
[0034] Figure 2 This is a schematic diagram of the FPGA computing kernel provided in an embodiment of the present invention. Detailed Implementation
[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0036] It should be noted that in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral connection; a mechanical connection or an electrical connection; a direct connection or an indirect connection through an intermediate medium; or a connection within two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0037] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0038] To address the issue of increased computational and time consumption in database query tasks due to row and column format conversion of queried data, this invention provides a database data format conversion system. This system utilizes an FPGA (Field Programmable Gate Array) to perform row and column format conversion on queried data during database query tasks, thereby reducing the additional performance overhead and computational power consumption caused by the database server CPU performing row and column format conversion, and improving database performance.
[0039] Specifically, refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a database data format conversion system provided in an embodiment of the present invention, based on... Figure 1 The data format conversion system for databases provided in this embodiment of the invention is applied to databases and includes a database control server and a data storage engine. The database control server includes a CPU, and the data storage engine includes a format conversion module and a data storage array.
[0040] Furthermore, the processor receives and parses data query commands sent by the user terminal to the database. These commands contain SQL query statements. The processor also parses the data query commands to determine the required format and storage partition address of the data the user wants to query. The format conversion module reads the data to be processed from the data storage array based on the storage partition address of the data the user wants to query. This data to be processed is the data to be queried corresponding to the data query command sent by the user terminal.
[0041] Optionally, after reading the data to be processed, the format conversion module determines whether the target format of the data to be processed is the same as the required format of the data to be queried by the user. If they are not the same, the format conversion module performs format conversion on the read data to be processed, thereby converting the target format of the data to be processed into the required format corresponding to the user's data query instruction, obtaining target data with the same data format as the user's requirement, and sending the target data to the processor. The processor sends the received target data to the user terminal, thereby responding to the user terminal's query instruction.
[0042] In one embodiment, when the required format of the data to be queried by the user is different from the target format of the data to be processed, for example, the required format of the data to be queried by the user is column format, while the target format of the data to be processed is row format; or, the required format of the data to be queried by the user is row format, while the target format of the data to be processed is column format.
[0043] Optionally, such as Figure 1 As shown, the format conversion module includes an FPGA computing core, which is connected to the processor of the database control server via a PCIe (Peripheral Component Interconnect Express) interface. The PCIe interface is used to facilitate data exchange between the processor and the FPGA computing core. The FPGA computing core includes:
[0044] The input recognition module is used to compare the target format of the data to be processed with the required format of the data to be queried by the user, so as to determine whether the target format of the data to be processed is the same as the required format of the user.
[0045] The data filtering module is used to filter the data to be processed, thereby filtering out invalid and empty data, reducing the amount of data that needs to be converted into formats, that is, reducing invalid format conversions and saving computing resources.
[0046] The data storage module is used to store the target data to be sent after format conversion.
[0047] Optionally, the FPGA computing core also includes a row splitting module, a column sending module, and several first buffer queues. The data filtering module includes a row filtering unit that performs row filtering on the data to be processed when the user's required format is column format and the target format of the data to be processed is row format. The row splitting module splits the filtered data to be processed into multiple target column data. Several first buffer queues are used to cache multiple target column data. The column sending module is used to send the target column data to the data storage module for caching.
[0048] In other words, when the user's required format differs from the target format of the data to be processed, and the user's required format is column-based while the target format of the data to be processed is row-based, the row filtering unit first performs row filtering on the data to be processed. The row splitting module then splits the filtered data into multiple columns according to the column width required by the user, and sends these multiple columns to the first cache queue for caching. The column sending module then sends the cached column data from each of the first cache queues to the data storage module for caching. Optionally, the first cache queues and column data can have a one-to-one correspondence, with each column data being sent to a corresponding first cache queue for caching, and multiple first cache queues used to cache different columns of data separately.
[0049] Optionally, the FPGA computing core also includes a column concatenation module, a row sending module, and several second buffer queues. The data filtering module includes a column filtering unit. When the user's required format is row format and the target format of the data to be processed is column format, the column filtering unit is used to perform column filtering on the data to be processed. Several second buffer queues are used to cache the data to be processed after column filtering. The column concatenation module is used to perform column concatenation processing on the data to be processed cached in several second buffer queues to obtain the target row data. The row sending module is used to send the target row data to the data storage module for caching.
[0050] In other words, when the user's required format differs from the target format of the data to be processed, and the user's required format is column-based while the target format of the data to be processed is row-based, firstly, the column filtering unit filters the data to be processed by columns; several second cache queues cache the filtered data to be processed; the column concatenation module concatenates the column data in the data to be processed to obtain the target row data; and the row sending module sends the concatenated target row data to the data storage module for caching. Optionally, the second cache queues and the column data in the data to be processed can be in one-to-one correspondence. Each column data, after being filtered by the column filtering unit, is sent to a corresponding second cache queue for caching, and multiple second cache queues are used to cache different column data in the data to be processed separately.
[0051] Optionally, the FPGA computing core also includes an interface sending module for sending data stored in the data storage module to the processor of the database control server via a PCIe interface. The data stored in the data storage module includes target column data and / or target row data.
[0052] Optionally, the data filtering module also includes a row and column filtering unit, and the FPGA computing core also includes a row and column sending module. When the user's required format is the same as the target format of the data to be processed, the row and column filtering unit is used to filter the data to be processed to obtain the target data. The row and column sending module is used to send the filtered target data to the interface sending module, which then sends the target data to the processor of the database central control server through the PCIe interface.
[0053] In some implementations, the data to be processed may include data in different formats at the same time. That is, the data to be processed may include data in row format and data in column format at the same time, and the target format of the data to be processed may include data in row format and data in column format at the same time.
[0054] Optionally, the data filtering module may include only row and column filtering units for filtering the data to be processed. Specifically, if the target format of the data to be processed is row format, the row and column filtering units are used to filter the data by rows; if the target format of the data to be processed is column format, the row and column filtering units are used to filter the data by columns; if the target format of the data to be processed includes both row and column formats, the row and column filtering units perform column filtering on the column data and row filtering on the row data of the data to be processed, respectively.
[0055] Optionally, the data filtering module may include a row filtering unit and a column filtering unit for filtering the data to be processed. Specifically, if the target format of the data to be processed is row format, the row filtering unit is used to filter the data by row; if the target format of the data to be processed is column format, the column filtering unit is used to filter the data by column; if the target format of the data to be processed includes both row format and column format, the row filtering unit is used to filter the row data in the data to be processed, and the column filtering unit is used to filter the column data in the data to be processed.
[0056] Optionally, the data filtering module may include a row filtering unit and a row-column filtering unit for filtering the data to be processed. Specifically, if the target format of the data to be processed is row format, the row filtering unit is used to filter the data by rows; if the target format of the data to be processed is column format, the row-column filtering unit is used to filter the data by columns; if the target format of the data to be processed includes both row and column formats, the row filtering unit is used to filter the row data in the data to be processed, and the row-column filtering unit is used to filter the column data in the data to be processed.
[0057] Optionally, the data filtering module may include column filtering units and row-column filtering units for filtering the data to be processed. Specifically, if the target format of the data to be processed is row format, the row-column filtering unit is used to filter the data by rows; if the target format of the data to be processed is column format, the column filtering unit is used to filter the data by columns; if the target format of the data to be processed includes both row and column formats, the column filtering unit is used to filter the column data in the data to be processed, and the row-column filtering unit is used to filter the row data in the data to be processed.
[0058] In one embodiment, the input recognition module is further configured to determine the data conversion frequency based on the data query instruction. The FPGA computing core also includes a clock generation module, configured to determine the clock frequency based on the data conversion frequency recognized by the input recognition module, thereby generating a clock signal. The FPGA computing core further includes a format conversion control module, configured to generate a scheduling control signal for the FPGA computing core. This scheduling control signal is used to schedule and control the functional modules participating in the format conversion process.
[0059] Optionally, such as Figure 1 As shown, the data storage engine also includes a power supply module and a crystal oscillator module for the FPGA computing core. The FPGA computing core further includes a set / reset / debugging interface. The power supply module supplies power to the FPGA computing core, the crystal oscillator module provides a reference frequency to the FPGA computing core to synchronize the speed and timing of data transmission and reception, and the set / reset / debugging interface is used to reset the FPGA computing core to its initial state.
[0060] In one embodiment, the database control server is the core of the database, capable of independently completing all data query tasks corresponding to complex SQL statements of the user, such as data sorting, aggregation, filtering, and joining. The data storage engine has a format conversion module with an FPGA computing kernel deployed, used to perform data filtering and row and column format conversion. It also has a data storage array, PCIe interface, crystal oscillator module, power supply module, and set / reset debugging interface, etc., responsible for processing the data required by the user read from the data storage array and performing data filtering tasks, and performing row and column format conversion tasks on the read data according to the user's required format.
[0061] For the data format conversion process in a single data query task, firstly, after receiving the SQL query command from the user terminal, the host CPU of the database control server performs command parsing and optimization to determine the data to be queried by the SQL query statement, the required data format, and the corresponding storage partition address in the data storage array. Then, the required data format and the storage partition address in the data storage array are sent to the FPGA computing core of the format conversion module in the data storage engine via the PCIe interface. The FPGA computing core reads the data to be processed from the data storage array based on the received storage partition address. According to the format conversion instructions in the data query command, the FPGA computing core adjusts the data format conversion processing frequency based on the principle of maximizing efficiency, and pre-filters the read data to be processed, converting the filtered data to the data format required by the data query command. The FPGA computing core packages the format-converted data and sends it to the processor of the database control server via the PCIe interface, and then sends a transmission completion signal to the processor. After receiving the transmission completion signal, the processor of the database central control server performs corresponding calculations and processing on the received data that has completed row and column format conversion according to the user's data query command, and then sends the results to the user terminal to respond to the user's data query command.
[0062] In other words, when a user initiates a data query request on the user terminal of the database control server, the CPU of the database control server parses the user's query statement based on the SQL query statement entered on the user terminal, determines the required data format, and sends the corresponding required format instruction and the storage partition address of the data to be queried in the data storage array to the FPGA computing core through the PCIe interface. Then, the FPGA computing core reads the data block stored at the corresponding storage partition address in the data storage array to obtain the data to be processed. Combined with the CPU's processing instructions, it begins to perform high-speed data row and column format conversion processing. The entire process involves operations such as instruction reading, frequency setting, data filtering, row and column format conversion, and result sending, all of which are implemented by the FPGA computing core, and the various modules of the FPGA computing core adopt a pipelined design.
[0063] Specifically, refer to Figure 2The FPGA computing core structure of the format conversion module shown includes an input recognition module, a clock generation module PLL (Phase Locked Loop), row filtering units, column filtering units, row-column filtering units, a row splitting module, a column concatenation module, a column sending module, a row sending module, a row-column sending module, several buffer queues composed of FIFO (First In First Out) memories, a data storage module, an interface sending module, and a format conversion control module. The following section combines... Figure 2 The data format conversion system for the database provided in the embodiments of the present invention will be described in detail.
[0064] like Figure 2 As shown, the input recognition module receives the request format and the storage partition address of the data to be queried from the processor of the database central control server. This request format and storage partition address are obtained by the processor parsing the data query command sent by the user terminal. The input recognition module also compares the target format of the data to be processed read from the data storage array with the user's request format to determine whether the target format of the data to be processed is the same as the user's request format.
[0065] If they are the same, for example, if the user's required format is row format and the target format of the data to be processed is row format, or if the user's required format is column format and the target format of the data to be processed is column format, then the row and column filtering unit is used to filter the data to be processed to obtain the target data. The row and column sending module is used to send the target data obtained after filtering by the row and column filtering unit to the interface sending module. The interface sending module is used to send the target data to the processor of the database central control server through the PCIe interface. The processor sends the target data to the user terminal to respond to the user's data query command.
[0066] If they are different, the read data to be processed needs to be sent to the corresponding row filtering unit for row filtering, or to the column filtering unit for column filtering. The input recognition module is also used to determine the data conversion frequency required by the user and send the data conversion frequency requirement to the PLL clock generation module. The PLL clock generation module is used to determine the clock frequency according to the user's required data conversion frequency, and generate the clock signal required for format conversion based on the clock frequency and send it to the corresponding functional module. The format conversion control module is used to generate a scheduling control signal based on the clock signal generated by the PLL clock generation module, and is used to schedule and control the functional modules required for format conversion.
[0067] Furthermore, if the target format of the data to be processed differs from the user's required format, and the user's required format is column format while the target format of the data to be processed is row format, the column filtering unit sequentially receives m columns (i.e., columns to be converted to row format) to perform column-row format conversion. Figure 2 The system takes input column data (columns 1 to m) of different bit widths, filters out empty and invalid data in each column according to user requirements, and then temporarily stores the filtered column data into m FIFO buffer queues (i.e., ...). Figure 2 The data is temporarily stored in FIFO1 to FIFOm. The column concatenation module reads column data one by one from each FIFO buffer queue according to the column concatenation instruction. It then truncates the effective width of the column data according to the input column width requirement and concatenates the columns sequentially based on the truncated effective width to obtain the target row data. The concatenated target row data is then sent to the row sending module, which packages the target row data and sends it to the data storage module for caching. When the column concatenation module no longer receives a valid column filtering signal, it indicates that all the read data to be processed has been concatenated and packaged. The data storage module then sends the temporarily stored concatenated target row data to the interface sending module, which sends the target row data to the database server's processor via the PCIe interface. The column concatenation instruction is generated by the input recognition module when the user's required format is row format and the target format of the data to be processed is column format, and is then sent to the column concatenation module.
[0068] Furthermore, if the target format of the data to be processed differs from the user's required format, and the user's required format is row format while the target format of the data to be processed is column format, the row filtering unit sequentially receives the data to be processed for column format conversion, filters out empty and invalid data in each row of the data to be processed according to the user's requirements, and then sends the filtered rows of data to the row splitting module. The row splitting module determines the number of columns to split and the width requirements of each column according to the row splitting instruction, and splits the entire row of data to obtain multiple target column data, i.e. Figure 2 The data from column 1 to column n is split and then sent to multiple FIFO buffer queues sequentially, column by column. Figure 2 The data is cached in FIFO1 to FIFOn. The column sending module packages the data of each column and sends it to the data storage module for caching. When the column sending module does not receive a valid signal from the row splitting module, it proves that all the data to be processed has been split and packaged. The data storage module transmits the cached target column data obtained from the splitting to the interface sending module in sequence. The interface sending module sends the target column data to the processor of the database server through the PCIe interface. Among them, the row splitting instruction is generated by the input recognition module when the user's required format is column format and the target format of the data to be processed is row format, and then sent to the row splitting module.
[0069] The functional modules involved in the row splitting and column concatenation processes are all uniformly scheduled and controlled by the format conversion control module, employing a pipelined design for high efficiency. Furthermore, the control clock required by the FPGA computing core of the format conversion module is uniformly generated by the PLL clock generation module, allowing the calculation frequency to be changed in real time according to user needs when processing row splitting and column concatenation tasks. Optionally, to ensure format conversion efficiency, database query efficiency, and response speed to user terminal data query commands, execution can be performed at a preset maximum frequency.
[0070] In this embodiment, the format conversion module based on the FPGA computing core completes the format conversion tasks that originally needed to be executed by the processor of the database control server. This significantly reduces the computing power overhead of the database control server and saves its computing resources. Furthermore, the FPGA computing core operates in parallel processing mode, is not limited by the bit width of the input data, and performs data format conversion processing more efficiently and faster. Moreover, the pre-filtering method reduces the format conversion of invalid and empty data, significantly reducing the time cost of data queries performed by the database server and improving the response speed to data query requests.
[0071] Furthermore, the FPGA computing core can be directly embedded into the database's read / write engine as a chip. After reading and writing, it directly performs format conversion processing without adding additional devices, making it simple, convenient, and inexpensive to use. Finally, by employing frequency conversion processing, the row and column format conversion can seamlessly interface with the frequency of the database's central control server, avoiding data loss issues caused by cross-clock domain data transmission and ensuring high reliability.
[0072] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A database data format conversion system, characterized in that, This includes a database control server and a data storage engine, among which: The database control server includes a processor, and the data storage engine includes a format conversion module and a data storage array; The processor is used to receive and parse data query instructions sent by the user terminal, and determine the required format and storage partition address of the data to be queried by the user. The format conversion module is used to read the data to be processed from the data storage array according to the storage partition address, and when the target format of the data to be processed is different from the required format, convert the target format of the data to be processed into the required format, and send the target data to the processor. The processor sends the target data to the user terminal in response to the data query command; The format conversion module includes an FPGA computing core, which is connected to the processor via a PCIe interface; the FPGA computing core includes: An input recognition module is used to compare the target format of the data to be processed with the required format to determine whether the target format of the data to be processed is the same as the required format. A data filtering module is used to filter the data to be processed; A data storage module is used to store the target data to be sent; The FPGA computing core further includes a row splitting module, a column sending module, and several first buffer queues; the data filtering module includes a row filtering unit; when the required format is column format and the target format of the data to be processed is row format: The row filtering unit is used to perform row filtering on the data to be processed. The row splitting module is used to split the filtered data into multiple target columns. The plurality of first cache queues are used to cache the target column data; The column sending module is used to send the target column data to the data storage module for caching; The FPGA computing core further includes a column concatenation module, a row sending module, and several second buffer queues; the data filtering module includes a column filtering unit; when the required format is row format and the target format of the data to be processed is column format: The column filtering unit is used to perform column filtering on the data to be processed; The plurality of second cache queues are used to cache the data to be processed after column filtering; The column concatenation module is used to perform column concatenation processing on the data to be processed cached in the plurality of second cache queues to obtain the target row data; The row sending module is used to send the target row data to the data storage module for caching.
2. The database data format conversion system according to claim 1, characterized in that, The FPGA computing core also includes an interface sending module, which is used to send the data stored in the data storage module to the processor through the PCIe interface.
3. The database data format conversion system according to claim 2, characterized in that, The data filtering module includes a row and column filtering unit, and the FPGA computing core further includes a row and column sending module, when the required format is the same as the target format of the data to be processed: The row and column filtering unit is used to filter the data to be processed to obtain the target data; The row and column sending module is used to send the target data to the interface sending module.
4. The database data format conversion system according to claim 1, characterized in that, The input recognition module is also used to determine the data conversion frequency based on the data query instruction.
5. The database data format conversion system according to claim 4, characterized in that, The FPGA computing core also includes a clock generation module, which is used to generate a clock signal based on the data conversion frequency.
6. The database data format conversion system according to claim 1, characterized in that, The FPGA computing kernel also includes a format conversion control module for generating scheduling control signals for the FPGA computing kernel.
7. The database data format conversion system according to claim 1, characterized in that, The data storage engine also includes a power module and a crystal oscillator module for the FPGA computing core; the FPGA computing core also includes a set / reset / debugging interface.