Data decompression method, acceleration card, storage medium and program product
By using decompression acceleration cards and direct memory access data transmission in the computing system, end-to-end transmission is realized, and the problems of many transmission paths and large delays in the prior art are solved, and the efficiency of data decompression is improved.
Patent Information
- Application Number
- CN202510389096.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-28
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2045-03-28
AI Technical Summary
In the prior art, the database to be decompressed requires two data transmissions initiated by the controller to reach the decompression card, which increases the transmission path and delay, and reduces the efficiency of data decompression.
By introducing a decompression acceleration card into the computing system, using direct memory access data transmission operations, an end-to-end transmission method is realized, and the database to be decompressed is directly transmitted from the disk to the decompression acceleration card, reducing the transmission path.
The transmission delay is reduced, the acquisition efficiency of the database to be decompressed is improved, and the data capacity obtained per unit time is increased at the same communication speed and distance, further improving the transmission efficiency.
Smart Images

Figure CN119917474A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data decompression method, an acceleration card, a storage medium and a program product. Background Art
[0002] At present, with the development of industries such as artificial intelligence, high-performance data analysis, communication transmission, and financial analysis, the computing demand for computing-intensive fields has increased, making heterogeneous computing receive widespread attention from all walks of life.
[0003] In the related art, the user initiates a decompression instruction based on the host, and the controller in the disk initiates data transmission to transfer the database to be decompressed to the host memory. Further, the controller in the host initiates data transmission again to transfer the database to be decompressed to the decompression card in the computing system. The decompression card decompresses the database to be decompressed based on the decompression instruction to obtain the decompression result. In the above heterogeneous computing, the database to be decompressed needs to be transmitted by the controller twice before it can reach the decompression card. The two transmissions increase the transmission path, thereby increasing the transmission delay, which reduces the efficiency of obtaining the database to be decompressed. Summary of the invention
[0004] The present application provides a data decompression method, an acceleration card, a storage medium and a program product, so as to at least solve the problem in the related art that data transmission increases the transmission path, thereby increasing the transmission delay, and reducing the efficiency of obtaining the database to be decompressed.
[0005] The present application provides a data decompression method, which is applied to a decompression acceleration card in a computing system, wherein the computing system further includes a central processing unit, a disk, and a compression acceleration card, including:
[0006] Receive and store the database to be decompressed sent by the disk to realize the end-to-end transmission mode; the database to be decompressed is sent by the controller in the disk after the disk receives the first decompression instruction sent by the central processor through the direct memory access data transmission operation; the database to be decompressed is obtained by compressing the database to be compressed by the compression acceleration card;
[0007] Decompress the database to be decompressed to obtain a decompressed database.
[0008] The present application also provides a data decompression method, which is applied to a compression acceleration card in a computing system, wherein the computing system further includes a central processing unit, a disk, and a decompression acceleration card, and the method includes:
[0009] Receiving and storing the database to be compressed sent by the disk to realize the end-to-end transmission mode; the database to be compressed is sent by the controller in the disk initiating the direct memory access data transmission operation after the disk receives the first compression instruction sent by the central processor;
[0010] Compressing the database to be compressed to obtain a compressed database;
[0011] The compressed database is sent to the disk through a direct memory access data transmission operation, so that the disk stores at least one compressed database, and provides a database to be decompressed for a decompression acceleration card, so that the decompression acceleration card decompresses based on the database to be decompressed; the database to be decompressed comes from at least one compressed database; after the disk receives a first decompression instruction sent by a central processing unit, the controller in the disk initiates a direct memory access data transmission operation and sends it to the decompression acceleration card.
[0012] The present application also provides an acceleration card, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned data decompression methods when executing the computer program.
[0013] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned data decompression methods are implemented.
[0014] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned data decompression methods when executed by a processor.
[0015] A data decompression method, an acceleration card, a storage medium and a program product are provided by the present application. The data decompression method in the present application is applied to a decompression acceleration card in a computing system. The computing system also includes a central processing unit and a disk. First, the disk receives a first decompression instruction sent by the central processing unit, so that the disk sends the database to be decompressed to the decompression acceleration card through a direct memory access data transmission operation, and then the decompression acceleration card implements an end-to-end transmission method to receive the database to be decompressed sent by the disk, and then stores the database to be decompressed. It can be seen that compared with the method in the related art that requires the host to obtain the database to be decompressed from the disk, store it in its own memory, and then send the database to be decompressed to the decompression card, the database to be decompressed in the present application does not need to undergo two data transmissions (i.e., generate two transmission paths), but directly completes the end-to-end transmission method through direct memory access, and only needs to undergo one data transmission (i.e., generate one transmission path) to transmit the database to be decompressed to the decompression acceleration card, thereby reducing the transmission delay and improving the efficiency of obtaining the database to be decompressed. In addition, in this embodiment, the database to be decompressed is a compressed database. When the compressed database to be decompressed is transmitted, at the same communication speed and communication distance, the capacity of the database to be decompressed is larger per unit time, thereby further improving the transmission efficiency. Decompressing the database to be decompressed can improve the efficiency of obtaining the database to be decompressed. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0017] Figure 1 A scenario application diagram of a data decompression method provided in this application;
[0018] Figure 2 A schematic diagram of a data decompression method flow provided in an embodiment of the present application;
[0019] Figure 3 A schematic diagram of a data decompression method flow provided in an embodiment of the present application;
[0020] Figure 4 An overall schematic diagram of a computing system provided in an embodiment of the present application;
[0021] Figure 5 A schematic diagram of the structure of a compression acceleration card provided in an embodiment of the present application;
[0022] Figure 6 A schematic diagram of the structure of a decompression acceleration card provided in an embodiment of the present application;
[0023] Figure 7 A schematic diagram of the structure of a data decompression device provided in an embodiment of the present application;
[0024] Figure 8 A schematic diagram of the structure of another data decompression device provided in an embodiment of the present application;
[0025] Fig. 9 A schematic diagram of the structure of an acceleration card provided in this application. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0027] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.
[0028] In the related art, the user initiates a decompression instruction based on the host, sends the decompression instruction to the disk, and then the controller in the disk initiates data transmission, and transmits the database to be decompressed to the host memory through the central processing unit. Further, the controller in the host initiates data transmission again, and transmits the database to be decompressed to the decompression card in the computing system. The decompression card decompresses the database to be decompressed based on the decompression instruction, and queries it after decompression, and obtains data that meets the query requirements to obtain the query results. In the above heterogeneous computing, the database to be decompressed needs to be transmitted to the decompression card after the controller initiates data transmission twice, that is, the database to be decompressed needs to be moved twice. The two transmissions increase the transmission path, and then increase the transmission delay, which reduces the efficiency of obtaining the database to be decompressed. Therefore, in the related art, after the user initiates the decompression instruction, the decompression card needs a certain delay to obtain the database to be decompressed.
[0029] In order to solve the problem that the database to be decompressed in the related art needs to go through two transmission paths, thereby increasing the transmission delay, thereby increasing the problem that the decompression card has low efficiency in obtaining the database to be decompressed, the execution subject decompression acceleration card in this scheme receives and stores the compressed database to be decompressed sent by the disk, realizing an end-to-end transmission method, and then the database to be decompressed only needs to go through one transmission path to be directly sent to the decompression acceleration card, without the need for redundant transmission paths, so that the database to be decompressed is sent to the decompression acceleration card faster, thereby improving the data transmission efficiency, and enabling the decompression acceleration card to obtain the database to be decompressed faster; in this scheme, the disk initiates a direct memory access data transmission operation, and sends the database to be decompressed to the decompression acceleration card, so an end-to-end transmission method is realized, so a transmission path is realized, thereby reducing the transmission delay. In addition, the decompression acceleration card can obtain the database to be decompressed faster, improving the efficiency of obtaining the database to be decompressed. Therefore, compared with the decompression card in the related art, this scheme realizes acceleration, so it is a decompression acceleration card.
[0030] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0031] In conjunction with the specific application environment architecture or the specific hardware architecture on which the execution of the data decompression method depends, the specific application environment architecture or the specific hardware architecture is described herein.
[0032] Figure 1 This is a scenario application diagram of a data decompression method provided by this application. Figure 1 As shown, it includes: a host 101, a disk 102, and a decompression acceleration card 103.
[0033] The host 101 may be a computer, a mobile phone or other device, which is not limited here.
[0034] The host 101 includes a central processing unit and a host memory. The central processing unit is a management device in the computing system and coordinates data processing of the computing system.
[0035] The decompression acceleration card 103 may be a Field Programmable Gate Array (FPGA) board, which can realize decompression and query of the database to be decompressed.
[0036] In this scenario, the user initiates a decompression command through the host 101, and the host 101 generates a first decompression instruction and a second decompression instruction.
[0037] Further, the host 101 sends a first decompression instruction to the disk 102, so that the disk 102 sends the database to be decompressed to the decompression acceleration card 103 through a direct memory access data transmission operation. At the same time, the host 101 sends a second decompression instruction to the decompression acceleration card 103, and then the decompression acceleration card 103 receives the second decompression instruction. The database to be decompressed is a compressed database.
[0038] Furthermore, the decompression acceleration card 103 receives the database to be decompressed and stores it.
[0039] Further, the decompression acceleration card 103 decompresses the database to be decompressed based on the second decompression instruction to obtain a decompressed database, and then queries the decompressed database to obtain a query result.
[0040] Furthermore, the decompression acceleration card 103 sends the query result to the host 101 .
[0041] In the present application, after the host 101 generates a decompression instruction, the decompression acceleration card can implement an end-to-end transmission mode to receive the database to be decompressed sent by the disk, thereby directly obtaining the database to be decompressed from the disk, thereby reducing the transmission path and thus reducing the transmission delay; in this solution, the controller in the disk initiates a direct memory access data transmission operation, so that the decompression acceleration card obtains the database to be decompressed through a transmission path, thereby achieving acceleration and improving the efficiency of obtaining the decompressed database.
[0042] In the present application, the hardware modules in the compression acceleration card (such as the second interface, the second converter, the compression module and the second memory) can realize the compression of the database to be compressed, and the hardware modules in the decompression acceleration card (such as the first interface, the first converter, the decompression module, the interface conversion module, the query module and the first memory) can realize the decompression and query of the database to be decompressed. The present application provides a hardware implementation method for data decompression and query.
[0043] The following are embodiments of the present application, the execution subject of the following embodiments is a decompression acceleration card in a computing system, and the computing system also includes a central processing unit, a disk and a compression acceleration card. The computing system of the present application can be a heterogeneous computing system.
[0044] Figure 2 A data decompression method flow chart provided in an embodiment of the present application. Figure 2 As shown, specifically including:
[0045] S201, receiving and storing the database to be decompressed sent by the disk to realize the end-to-end transmission mode; the database to be decompressed is sent by the controller in the disk after the disk receives the first decompression instruction sent by the central processor to initiate the direct memory access data transmission operation; the database to be decompressed is obtained by compressing the database to be compressed by the compression acceleration card.
[0046] Among them, the peer-to-peer transmission method (Peer to Peer, referred to as P2P) is a technology in which two communication ends directly transmit data.
[0047] The database to be decompressed refers to the database from which decompression is to be performed.
[0048] Among them, Direct Memory Access (DMA) is a computer technology that supports direct data transmission between boards, devices, and systems, which can reduce transmission paths and delays.
[0049] The compression acceleration card is located in the computing system and is used to compress data. Specifically, the compression acceleration card compresses at least one sub-database to be compressed to obtain a compressed database.
[0050] After the sub-database to be compressed is compressed, a compressed database can be obtained, and the obtained compressed database is sent to the disk through a direct memory access data transmission operation, realizing an end-to-end transmission mode, so that the disk may include at least one compressed database. In this application, the database to be decompressed is one or more compressed databases.
[0051] The database to be decompressed may be text, video or image, which is not limited here.
[0052] Among them, a controller is deployed in the disk, and the controller can initiate a direct memory access data transmission operation, so that the database to be decompressed in the disk is directly sent to the decompression acceleration card, so that the decompression acceleration card realizes end-to-end transmission to receive the database to be decompressed.
[0053] Among them, the decompression acceleration card and compression acceleration card in this application can be FPGA boards.
[0054] In one method, after the compression acceleration card compresses the database to be compressed, it obtains a compressed database. If the compressed database is a database to be decompressed, and at this time the decompression acceleration card is receiving a query for the database to be decompressed, the compression acceleration card can use its own controller to initiate a direct memory access data transmission operation, thereby realizing an end-to-end transmission method to send the database to be decompressed directly to the decompression acceleration card, so that the decompression acceleration card receives the database to be decompressed.
[0055] Among them, the first decompression instruction includes the identifier of the database to be decompressed and the disk storage location information of the database to be decompressed, wherein the storage location information of the database to be decompressed is the location information of the database to be decompressed stored on the disk, wherein the disk storage location information represents the disk storage address of the database to be decompressed.
[0056] Specifically, the disk receives the first decompression instruction sent by the central processor, locates the disk storage address where the database to be decompressed is located, and sends the database to be decompressed from the disk storage address to the decompression acceleration card through a direct memory access data transmission operation.
[0057] S202, decompress the database to be decompressed to obtain a decompressed database.
[0058] The decompressed database refers to the decompressed database.
[0059] In one embodiment, the decompression acceleration card may include a decompression module, and the database to be decompressed is decompressed by using the decompression module to obtain a decompressed database.
[0060] In another manner, the decompression acceleration card may include multiple decompression modules, and the multiple decompression modules are used to decompress the database to be decompressed, thereby obtaining the decompressed database.
[0061] It should be noted that after obtaining the decompressed database, if the second decompression instruction includes a query instruction (ie, includes a query target), the decompressed database is queried to obtain a query result.
[0062] In one embodiment, the decompression acceleration card queries the decompression database based on the second decompression instruction to obtain a query result.
[0063] Among them, the second decompression instruction can be used to stimulate the decompression acceleration card to perform the decompression task, wherein the second decompression instruction includes the target to be queried, the query database identifier, the decompression information of the database to be decompressed, the storage information, the parameters required for the decompression database conversion information (such as the capacity of the database to be decompressed, etc.) and other related information, which is not limited here.
[0064] In addition, for the case where the second decompression instruction includes a query instruction, when the user's ultimate goal is to query, in this embodiment, since the database to be decompressed is a compressed database, when the compressed database to be decompressed is transmitted, at the same communication speed and communication distance, more capacity of the database to be decompressed is obtained per unit time, thereby further improving the transmission efficiency. Decompressing the database to be decompressed can improve the efficiency of obtaining the database to be decompressed, and can also improve the efficiency of obtaining the query results. The present embodiment provides a data decompression method. The data decompression method in the present embodiment is applied to a decompression acceleration card in a computing system. The computing system also includes a central processing unit and a disk. First, the disk receives a first decompression instruction sent by the central processing unit, so that the disk sends the database to be decompressed to the decompression acceleration card through a direct memory access data transmission operation, and then the decompression acceleration card implements an end-to-end transmission method to receive the database to be decompressed sent by the disk, and then stores the database to be decompressed. It can be seen that compared with the method in the related art that requires the host to obtain the database to be decompressed from the disk, store it in its own memory, and then send the database to be decompressed to the decompression card, the database to be decompressed in the present application does not need to undergo two data transmissions (i.e., generate two transmission paths), but directly completes the end-to-end transmission method through direct memory access, and only needs to undergo one data transmission (i.e., generate one transmission path) to transmit the database to be decompressed to the decompression acceleration card, thereby reducing the transmission delay and improving the efficiency of obtaining the database to be decompressed.
[0065] In the above further refinement, in this embodiment, the database to be decompressed includes at least one sub-database to be decompressed; the decompression acceleration card also includes a first memory.
[0066] This embodiment is an optional method of receiving and storing a database to be decompressed sent by a disk to implement an end-to-end transmission mode, which specifically includes:
[0067] Step 1: Receive the database to be decompressed to achieve end-to-end transmission, and receive the second decompression instruction sent by the central processor; the second decompression instruction includes storage information; the storage information includes storage address information of at least one sub-database to be decompressed in the first memory.
[0068] The storage information refers to the storage address information of the database to be decompressed stored in the first memory, wherein the database to be decompressed includes at least one sub-database to be decompressed, and the storage information includes the storage address information of at least one sub-database to be decompressed stored in the first memory.
[0069] Step 2: Using the first memory, store at least one to-be-decompressed sub-database to a corresponding storage address in the first memory according to the storage address information.
[0070] Furthermore, the decompression acceleration card stores at least one to-be-decompressed sub-database at a storage address corresponding to the first memory according to corresponding storage address information.
[0071] This embodiment provides a data decompression method. In this embodiment, for each sub-database to be decompressed, according to its corresponding storage address information, it is stored at a storage address corresponding to the first memory, so that the storage is orderly and organized.
[0072] In further detail of the above, in this embodiment, the decompression acceleration card also includes a decompression converter and at least one decompression module.
[0073] This embodiment is an optional method for decompressing the database to be decompressed to obtain the decompressed database, which specifically includes:
[0074] Step 1: Use a decompression converter to receive the decompression information in the second decompression instruction sent by the central processor; the decompression information includes the path information of each sub-database to be decompressed and the corresponding target decompression module identifier; the target decompression module identifier is the decompression module identifier for decompressing the corresponding sub-database to be decompressed.
[0075] Among them, the decompression converter is a one-to-one conversion interface used in the decompression acceleration card.
[0076] It should be noted that if the database to be decompressed includes two sub-databases to be decompressed, namely the first sub-database to be decompressed and the second sub-database to be decompressed, the decompression information includes the path information of the first sub-database to be decompressed (assuming the first path information) and the corresponding target decompression module identifier (assuming the first target decompression module identifier), as well as the path information of the second sub-database to be decompressed (assuming the second path information) and the corresponding target decompression module identifier (assuming the second target decompression module identifier).
[0077] The first target decompression module identifier corresponds to the first target decompression module, wherein the first target decompression module is a decompression module that decompresses the first sub-database to be decompressed.
[0078] Step 2: For any path information, a decompression converter is used to send the path information to a target decompression module identified by a corresponding target decompression module, and the target decompression module is used to obtain a corresponding sub-database to be decompressed from the first memory based on the corresponding path information; the path information includes the identifier of the sub-database to be decompressed and the storage address information corresponding to the sub-database to be decompressed.
[0079] According to the above exemplary embodiment, the first path information is taken as an example for explanation, wherein the first path information includes an identifier of a first sub-database to be decompressed and first storage address information corresponding to the first sub-database to be decompressed.
[0080] The storage address corresponding to the storage address information stores the corresponding sub-database to be decompressed, that is, the first storage address corresponding to the first storage address information stores the first sub-database to be decompressed, wherein the storage address information and the storage address are in the first memory.
[0081] According to the above exemplary embodiment, taking the first path information as an example, the decompression converter sends the first path information to the first target decompression module corresponding to the first target decompression module identifier, and continues to use the first target decompression module to obtain the first sub-database to be decompressed from the first storage address in the first memory based on the first path information. Among them, the first storage address is the storage address corresponding to the first storage address information. Similarly, for the second path information, the decompression converter uses the same method to obtain the corresponding sub-database to be decompressed.
[0082] This embodiment provides a data decompression method. In this embodiment, the decompression converter can send path information to at least one target decompression module according to the decompression information, so that at least one target decompression module can obtain the corresponding sub-database to be decompressed from the first memory based on the corresponding path information, so the decompression converter realizes one-to-many; in addition, each target decompression module can respectively obtain its corresponding sub-database to be decompressed, which is convenient for decompression. In this embodiment, the decompression acceleration includes at least one decompression module, wherein at least the decompression module includes a target decompression module, and the number of target decompression modules is less than or equal to the number of decompression modules, so that each target decompression module can perform its own decompression task, and all decompression tasks can be completed more quickly.
[0083] In the above further refinement, this embodiment is an optional method of using the target decompression module to obtain the corresponding sub-database to be decompressed from the first memory based on the corresponding path information, specifically including:
[0084] Step 1: Using a target decompression module to locate a corresponding storage address from a first memory based on corresponding storage address information.
[0085] Step 2: Obtain the to-be-decompressed sub-database corresponding to the to-be-decompressed sub-database identifier from the storage address; the decompression performance of the target decompression module satisfies the decompression of the to-be-decompressed sub-database.
[0086] Among them, the decompression module in the present application includes at least one decompression module, and the decompression performance of each decompression module can meet the decompression of the corresponding sub-database to be decompressed.
[0087] In the above further refinement, this embodiment is an optional method of decompressing the database to be decompressed to obtain the decompressed database, which specifically includes:
[0088] Each target decompression module is used to decompress the respective sub-database to be decompressed in parallel to obtain each decompressed sub-database respectively; the target decompression module is a decompression module that decompresses the corresponding sub-database to be decompressed.
[0089] According to the above example, this decompression task requires two target decompression modules, namely the first target decompression module and the second target decompression module. The above two target decompression modules can decompress their respective sub-databases to be decompressed in parallel to obtain corresponding decompressed sub-databases, assuming that they are the first decompression sub-database and the second decompression sub-database respectively.
[0090] This embodiment provides a data decompression method. In this embodiment, each target decompression module can decompress its own sub-database to be decompressed in parallel. Due to the parallel decompression, all decompression tasks can be completed faster, thereby improving the decompression efficiency.
[0091] In the further refinement of the above, in this embodiment, the decompression acceleration card also includes an interface conversion module and at least one query module.
[0092] The query module is used to query the decompressed sub-database to find data that meets the query target.
[0093] This embodiment is an optional method of using each target decompression module to decompress the respective to-be-decompressed sub-databases in parallel to obtain the decompressed sub-databases respectively, which specifically includes:
[0094] Step 1: Use each target decompression module to directly send the corresponding decompression sub-database to the interface conversion module bypassing the first memory.
[0095] The interface conversion module is used to perform one-to-many or many-to-one conversion when querying the decompressed sub-database.
[0096] Step 2: Use the interface conversion module to send the corresponding decompressed sub-database to the corresponding query module according to the conversion information; the conversion information includes the link mode of each decompressed sub-database and the query module identifier corresponding to each decompressed sub-database; the conversion information is determined by the interface conversion module.
[0097] Wherein, the interface conversion module includes a linking mode of the decompression sub-database, and the linking mode can be determined based on the throughput of the decompression module and the throughput of the query module. When the throughput of the decompression module is greater than the throughput of a target query module, the interface conversion module determines that the decompressed sub-database output by the decompression module needs to be segmented, generates at least one segmented sub-database, and determines the corresponding first query module for each segmented sub-database, and generates corresponding conversion information based on each segmented sub-database and the corresponding first query module identifier. In the conversion information, the linking mode of the decompressed sub-database output by the decompression module is one-to-many, and the first query module identifier corresponding to each segmented sub-database obtained after the decompression sub-database is segmented. Wherein, the first query module includes the above-mentioned target query module.
[0098] The linking modes of the decompressed sub-databases include one-to-many or many-to-one.
[0099] Step 3: Use the query module to query the corresponding decompressed sub-database to obtain the query result.
[0100] The query result refers to the result obtained by querying from the decompression sub-database that satisfies the query instruction in the second decompression instruction.
[0101] Exemplarily, assuming that the first query module corresponds to the first decompressed sub-database, the first query module queries the first decompressed sub-database to obtain a first query result.
[0102] Step 4: Send the query result to the central processing unit. This embodiment provides a data decompression method. In this embodiment, each target decompression module directly sends the corresponding decompressed sub-database to the interface conversion module, bypassing the first memory. In the related art, the decompressed sub-database is saved to the first memory, and then the query module obtains the decompressed sub-database from the first memory for query. Therefore, in this embodiment, the first memory is bypassed and the decompressed sub-database is directly sent to the corresponding query module without transfer, thereby reducing the data transmission delay.
[0103] In the further refinement of the above, this embodiment is an optional method of using the interface conversion module to send the corresponding decompressed sub-database to the corresponding query module according to the conversion information, including:
[0104] Method 1: If the link mode of the first decompressed sub-database is one-to-many, the interface conversion module is used to segment the first decompressed sub-database according to the conversion information, and each segmented sub-database is sent to multiple first query modules accordingly; the first query module is the query module corresponding to the query module identifier of the first decompressed sub-database in the conversion information; the first decompressed sub-database is any decompressed sub-database in the decompressed sub-database.
[0105] It should be noted that the conversion information may include the number of segments into which the first decompressed sub-database is segmented. For example, the first decompressed sub-database is segmented into two or three segments, which is not limited here.
[0106] Among them, the one-to-many situation is specifically as follows: the interface conversion module determines that the throughput of the decompression module corresponding to the first decompression sub-database is greater than the throughput of a target query module, or the capacity of the decompression sub-database corresponding to the decompression module (i.e., the first decompression sub-database) is greater than the capacity that the above-mentioned target query module can handle, and then the link mode of the decompression sub-database is one-to-many, the interface conversion module generates conversion information, segments the decompression sub-database according to the conversion information, and then uses the interface conversion module to send each segmented sub-database to the corresponding first query module; wherein the first query module includes the above-mentioned target query module.
[0107] The first decompression sub-database is any decompression sub-database in the decompression sub-databases, and the first decompression sub-database is decompressed by the first decompression module.
[0108] The first query module includes multiple first query modules, and each first query module corresponds to a segment sub-database.
[0109] Method 2: If the link mode of the first decompressed sub-database is many-to-one, the interface conversion module is used to send the first decompressed sub-database and the decompressed sub-database bound in the conversion information to the corresponding second query module; the second query module is the query module corresponding to the query module identifier of the first decompressed sub-database and the decompressed sub-database bound in the conversion information.
[0110] Among them, the many-to-one situation is specifically as follows: the interface conversion module determines that the throughput of the decompression module is less than the throughput of a target query module, or the capacity of the decompression sub-database (i.e., the first decompression sub-database) corresponding to the decompression module is less than the capacity that the above-mentioned target query module can process, and based on the above description, generates conversion information for the first decompression sub-database, and the conversion information includes the decompression sub-database bound to the first decompression sub-database (assuming to be decompression sub-database A and decompression sub-database B), and then the interface conversion module sends the first decompression sub-database, decompression sub-database A and decompression sub-database B to the second query module, wherein the second query module is the above-mentioned target query module.
[0111] Among them, there is one second query module.
[0112] Among them, the second query module can meet the query processing of the first decompressed sub-database and the decompressed sub-database bound in the conversion information, so the second query module queries the first decompressed sub-database and the decompressed sub-database bound in the conversion information within the query processing capability, thereby reflecting resource balance and avoiding the query module in the related technology from executing queries beyond the query processing capability.
[0113] In this application, each query module satisfies the corresponding query processing when processing its own sub-database.
[0114] This embodiment also includes:
[0115] Each first query module and / or each second query module queries the corresponding decompressed sub-database in parallel, and the query module performing the query is a working status query module.
[0116] The working status query module refers to the query module that is performing query work.
[0117] It should be noted that the interface conversion module includes a conversion controller, wherein the conversion controller can be connected via an AXI4_Lite line, and the parameters of the conversion controller are configured and issued by the host, wherein the parameters include conversion information, the number of query modules, etc., which are not limited here.
[0118] The present embodiment provides a data decompression method. In the present embodiment, according to the linking mode of the decompressed sub-database, an interface conversion module is used to send the first decompressed sub-database to the first query module or the second query module according to the conversion information, thereby realizing a one-to-many or many-to-one conversion mode, thereby facilitating the query module to query the corresponding decompressed sub-database under its own query capability. In the present embodiment, dynamic conversion can be realized according to different linking modes. Compared with the traditional one-to-one conversion mode in the related art, in the present embodiment, conversion is performed differently according to different linking modes; and for each conversion task, the conversion mode is not fixed, but the conversion mode is flexibly distinguished according to the capacity of each database to be decompressed, thereby reflecting dynamic conversion.
[0119] In further detailing the above, the decompression acceleration card in this embodiment also includes: an interrupt counting module.
[0120] This embodiment is an optional method of using a query module to query the corresponding decompressed sub-database to obtain query results, which specifically includes:
[0121] Step 1: Use each working status query module to query in parallel in their respective sub-databases based on the received query target to obtain the corresponding query sub-result, which is the data that meets the query target; the second decompression instruction includes the query target; the working status query module is the query module for performing the query; the sub-database includes the first decompressed sub-database and its bound decompressed sub-database, or any segmented sub-database after the corresponding first decompressed sub-database is segmented.
[0122] The query target refers to the query requirement. For example, the query target is to search for all cat images in the database to be decompressed.
[0123] The respective sub-databases are sub-databases used for query, and may include the first decompressed sub-database and its bound decompressed sub-database, and any segmented sub-database after the first decompressed sub-database is segmented.
[0124] According to the above exemplary embodiment, it is assumed that the first target decompression module decompresses the first sub-database to be decompressed to obtain a decompressed sub-database A; and the second target decompression module decompresses the second sub-database to be decompressed to obtain a decompressed sub-database B.
[0125] Mode 1: Assume that the conversion information of the decompressed sub-database A is: the link mode of the decompressed sub-database A is many-to-one, and the decompressed sub-database bound to it is the decompressed sub-database B. The interface conversion module is used to send the decompressed sub-database A and the decompressed sub-database B to the second query module. Then, the second query module queries the respective sub-databases, wherein the respective sub-databases include the decompressed sub-database A and the decompressed sub-database B, wherein the decompressed sub-database A is the first decompressed sub-database, and the decompressed sub-database B is the decompressed sub-database bound to the first decompressed sub-database.
[0126] Method 2: Assuming that the conversion information of the decompressed sub-database A is: the link mode of the decompressed sub-database A is one-to-many, the decompressed sub-database A is segmented, assuming that the segments are segmented sub-database A and segmented sub-database B, and the interface conversion module is used to send the segmented sub-database A and the segmented sub-database B to multiple first query modules, specifically, the segmented sub-database A is sent to the first query module A, and the segmented sub-database B is sent to the first query module B. The first query module A queries in the respective sub-databases, wherein the respective sub-databases are the segmented sub-database A; the first query module B queries in the respective sub-databases, wherein the respective sub-databases are the segmented sub-database B.
[0127] In this mode, the link mode of the decompressed sub-database B can be one-to-one, and according to the query module identifier corresponding to the decompressed sub-database B in the conversion information, the interface conversion module is used to send the decompressed sub-database B to the query module corresponding to the query module identifier, assuming that it is the first query module C. The first query module C queries the respective sub-databases, wherein the respective sub-databases are the decompressed sub-database B.
[0128] According to the description in the above-mentioned method 2, the queries performed by the first query module A, the first query module B and the first query module C are working status query modules. At this time, each working status query module queries its corresponding query database based on the query target in parallel, and obtains corresponding query sub-results respectively, which are assumed to be the first query sub-result, the second query sub-result and the third query sub-result respectively.
[0129] Step 2: Use each working status query module to send the corresponding query sub-result to the first memory. If the interrupt counting module is used to generate a total interrupt signal, the first memory obtains the query result; the query result includes at least one query sub-result.
[0130] Specifically, the second decompression instruction also includes a storage identifier after the query, that is, an identifier of the storage address after the query corresponding to the query sub-result.
[0131] Among them, a decompression converter is used to send the corresponding queried storage identifier in the second decompression instruction to the corresponding first query module and / or second query module, and then each query module sends the query sub-result to the storage address corresponding to the storage identifier corresponding to the first memory.
[0132] It should be noted that the first query module A receives the corresponding storage identifier after the query, sends the first query sub-result to the storage address corresponding to the storage identifier after the query in the first memory, and uses the interrupt counting module to generate a total interrupt signal, so as to obtain all query sub-results, that is, obtain the query result.
[0133] It should be noted that the decompressed sub-database is transmitted to the interface conversion module via the AXI_Stream line, and then the interrupt sub-signal is transmitted to the interrupt counting module via the signal line. The AXI_Stream line used in this application can realize data transmission faster. Among them, the signal line can be a normal line, or other available lines, which are not limited here.
[0134] The present embodiment provides a data decompression method. In the present embodiment, each working status query module queries in its own sub-database in parallel. Since it is parallel processing, the speed of data query is further improved. Compared with the method of querying the decompressed sub-databases one by one, it is better, so the query efficiency is improved.
[0135] In the above further refinement, this embodiment is an optional method of using an interrupt counting module to generate a total interrupt signal, specifically including:
[0136] Step 1: After obtaining the corresponding query sub-result for each working status query module, the working status query module is used to generate an interrupt sub-signal and sent to the interrupt counting module.
[0137] The interrupt counting module is a module for counting interrupt sub-signals, and includes a counter.
[0138] The interrupt sub-signal is an interrupt signal generated after the working status query module obtains the corresponding query sub-result, which means that the corresponding working status query module has interrupted the query.
[0139] Step 2: Use the interrupt counting module to generate a total interrupt signal based on the interrupt sub-signals.
[0140] It should be noted that when all the working status query modules generate interrupt sub-signals and send them to the interrupt counting module, the counter in the interrupt counting module counts, and then the interrupt counting module generates a total interrupt signal.
[0141] In the above further refinement, this embodiment is an optional method before using the interrupt counting module to generate a total interrupt signal, specifically including:
[0142] The number of queries is calculated using an interruption counting module, which is based on the capacity of the database to be decompressed, the number of query modules, and the throughput corresponding to the query modules. The capacity of the database to be decompressed and the throughput corresponding to the query modules may be included in the second decompression instruction and sent to the interruption counting module through the decompression converter.
[0143] The capacity of the database to be decompressed can be expressed as DATA total Indicates that the number of modules can be queried using IP num Indicates that the throughput corresponding to the query module can be obtained by IP Throughput .
[0144] Among them, first the interrupt counting module calculates the number of query modules, that is, IP num =T rate / IP Throughput , where T rate Indicates the speed of data transmission, that is, the throughput corresponding to the query module.
[0145] Among them, the query number is Count num Indicates that, as shown in (1), the number of queries is calculated:
[0146] (1)
[0147] Among them, % means finding the remainder.
[0148] This embodiment is an optional method of using an interrupt counting module to generate a total interrupt signal based on an interrupt sub-signal, specifically including:
[0149] An interrupt counting module is used to count the number of interrupt sub-signals received, and a total interrupt signal is generated when the number of interrupt sub-signals is equal to the number of queries.
[0150] Exemplarily, according to the description of the embodiment, assuming that the number of queries is three, after the first query module A obtains the first query sub-result, a first interrupt sub-signal is generated. Similarly, after the remaining first query modules successively obtain the corresponding query sub-results, corresponding interrupt sub-signals are generated. Furthermore, the interrupt counting module receives each interrupt sub-signal and generates a total interrupt signal when the number of interrupt sub-signals is equal to the number of queries (three times).
[0151] In further detailing the above, this embodiment includes:
[0152] An interrupt counting module is used to send a total interrupt signal to the central processing unit through a high-speed serial computer expansion bus. The total interrupt signal is used to prompt the central processing unit that the query of the database to be decompressed has been completed. The total interrupt signal is also used to prompt the central processing unit that the decompression acceleration card is ready for the next decompression task and is ready to query the decompression database of the next decompression task.
[0153] Among them, the high-speed serial computer expansion bus is abbreviated as PCIe, which stands for Peripheral Component Interconnect Express. Among them, PCIe can increase the data transmission speed.
[0154] Specifically, inside the decompression acceleration card, the interrupt counting module transmits the total interrupt signal through the signal line, and the decompression acceleration card transmits the signal externally through the PCIe line.
[0155] It should be noted that, in this embodiment, when the interrupt counting module receives an interrupt sub-signal, it will not immediately feed back to the central processing unit to prompt the central processing unit that the query module corresponding to the interrupt sub-signal has been converted from the working state to the idle state. Instead, when each working state module has obtained the corresponding query sub-result and sent the corresponding interrupt sub-signal to the interrupt counting module, the interrupt counting module generates a total interrupt signal when the number of interrupt sub-signals is equal to the number of queries, and then sends it to the central processing unit through the high-speed serial computer expansion bus. Therefore, it can prompt the central processing unit that the query of the database to be decompressed has been completed, and it is also used to prompt the central processing unit that the decompression acceleration card is ready for the next decompression task. Therefore, the cumbersome process of feeding back to the central processing unit every time an interrupt sub-signal is generated is eliminated. Only one total interrupt signal needs to be fed back to the central processing unit, thereby improving the experience. In addition, the number of queries in this embodiment is not fixed, but is calculated by the interruption counting module based on the capacity of the database to be decompressed, the number of query modules and the throughput corresponding to the query modules. Therefore, the number of queries is flexible and changes dynamically, so it is more in line with the database to be decompressed, which can improve query efficiency and enhance user experience; and using the interruption counting module to calculate the number of queries can alleviate the computing pressure of the host.
[0156] The following is an embodiment of the present application, in which the execution subject is a compression acceleration card, and the computing system also includes a compression acceleration card, a central processing unit, a disk and a decompression acceleration card.
[0157] Figure 3 A data decompression method flow chart provided in an embodiment of the present application. Figure 3 As shown, specifically including:
[0158] S301, receiving and storing a database to be compressed sent by a disk to implement an end-to-end transmission mode; the database to be compressed is sent by a controller in the disk initiating a direct memory access data transmission operation after the disk receives a first compression instruction sent by a central processor.
[0159] S302: compress the database to be compressed to obtain a compressed database.
[0160] S303, sending the compressed database to the disk through a direct memory access data transmission operation, so that the disk stores at least one compressed database, and provides a database to be decompressed for the decompression acceleration card, so that the decompression acceleration card decompresses based on the database to be decompressed; the database to be decompressed comes from at least one compressed database; the database to be decompressed is sent to the decompression acceleration card by a controller in the disk initiating a direct memory access data transmission operation after the disk receives a first decompression instruction sent by the central processing unit.
[0161] In this embodiment, the compression acceleration card also includes a second memory.
[0162] This embodiment includes receiving and storing the database to be compressed sent by the disk to achieve a method of end-to-end transmission, specifically:
[0163] A database to be compressed is received to realize an end-to-end transmission mode, and a second decompression instruction sent by a central processor is received; the second compression instruction includes compressed storage address information of at least one sub-database to be compressed in the database to be compressed in the second memory.
[0164] The second memory is used to store at least one sub-database to be compressed at a corresponding compression storage address in the second memory according to the compression storage address information.
[0165] Among them, in this embodiment: the compression acceleration card also includes at least one compression module.
[0166] Before compressing the database to be compressed to obtain the compressed database, this embodiment includes:
[0167] A compression converter is used to receive configuration information in a second compression instruction sent by a central processor; the configuration information includes compression information of each sub-database to be compressed and a corresponding target compression module identifier; the target compression module identifier is a compression module identifier for compressing the corresponding sub-database to be compressed.
[0168] For any compressed information, a compression converter is used to send the compressed information to a target compression module identified by a corresponding target compression module, and the target compression module is used to obtain the corresponding sub-database to be compressed from the second memory based on the corresponding compressed information; the compressed information includes the identifier of the sub-database to be compressed and the compressed storage address information corresponding to the sub-database to be compressed.
[0169] Among them, the compression converter is a conversion interface used in the compression acceleration card.
[0170] This embodiment includes: an optional method of using the target compression module to obtain the corresponding sub-database to be compressed from the second memory based on the corresponding compression information, specifically:
[0171] A target compression module is used to locate the corresponding compressed storage address from the second memory based on the corresponding compressed storage address information.
[0172] The sub-database to be compressed corresponding to the sub-database to be compressed identifier is obtained from the compression storage address; and the compression performance of the target compression module satisfies the compression of the sub-database to be compressed.
[0173] This embodiment includes: compressing the database to be compressed to obtain an optional method of a compressed database, specifically:
[0174] Each target compression module is used to compress the respective sub-database to be compressed in parallel to obtain each compressed sub-database.
[0175] In this embodiment, the compression acceleration card also includes an interrupt counting module.
[0176] This embodiment is an optional method before sending the compressed database to the disk, including:
[0177] After obtaining each compression sub-database for each target compression module, an interrupt sub-signal is generated using the target compression module and sent to the interrupt counting module.
[0178] An interrupt counting module is used to generate a total interrupt signal based on the interrupt sub-signals.
[0179] This embodiment also includes: before using the interrupt counting module to generate the total interrupt signal based on the interrupt sub-signal, it includes:
[0180] An interrupt counting module is used to calculate the number of compressions; the number of compressions is calculated based on the capacity of the database to be compressed, the number of compression modules, and the corresponding throughput of the compression modules.
[0181] This embodiment also includes: an optional method of using the interrupt counting module to generate a total interrupt signal based on the interrupt sub-signal, specifically:
[0182] The terminal counting module is used to count the number of received interruption sub-signals, and when the number of interruption sub-signals is equal to the number of compression times, a total interruption signal is generated.
[0183] This embodiment also includes:
[0184] An interrupt counting module is used to send a total interrupt signal to the central processing unit through a high-speed serial computer expansion bus. The total interrupt signal is used to prompt the central processing unit that the database to be compressed has been compressed. The total interrupt signal is also used to prompt the central processing unit that the compression acceleration card is ready for the next compression task.
[0185] Figure 4 The overall schematic diagram of a computing system provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the computing system includes a compression acceleration card 401, a decompression acceleration card 402, a host memory 403 and a central processing unit 404. The host includes the host memory 403 and the central processing unit 404.
[0186] The compression acceleration card 401 includes a second interface 4011, a second converter 4012, at least one compression module 4013, a second memory 4014, and an interrupt counting module 4015. The second converter 4012 is a compression converter, which is used in the compression acceleration card 401. The second interface 4011 is externally connected to a high-speed serial computer expansion bus. The second converter 4012 is a compression converter.
[0187] The decompression acceleration card 402 includes a first interface 4021, a first converter 4022, at least one decompression module 4023, an interface conversion module 4024, a first memory 4025, an interrupt counting module 4026, and at least one query module 4027. The first interface 4021 is externally connected to a high-speed serial computer expansion bus. The first converter 4022 is a decompression converter.
[0188] Figure 4 The computing system further includes a main line switch 405, wherein the main line switch 405 is a high-speed serial computer expansion bus switch.
[0189] Figure 4 The computing system further includes a disk 406 in which the compressed database is stored.
[0190] Figure 4 The computing system further includes an AXI4_MM circuit, an AXI_Stream circuit and a control circuit, wherein the control circuit may be an AXI4_Lite circuit, wherein the control circuit is transmitted in the form of a control signal.
[0191] It should be noted that the second interface 4011 also includes an interrupt control processing module, and the interrupt counting module 4015 in the compression acceleration card 401 is connected to the interrupt control processing module through the second interface 4011, and the interrupt control processing module transmits the total interrupt signal to the central processing unit through the main line switch 405. Similarly, the first interface 4021 also includes an interrupt control processing module, and the corresponding total interrupt signal is sent to the main line switch 405 in the same connection method as that in the compression acceleration card 401.
[0192] Figure 4 The high-speed serial computer expansion bus is also included. The high-speed serial computer expansion bus includes a receiving data line and a transmitting data line. The receiving data line is used to receive a high-speed serial receiving signal (PCIe_RX) externally, and the transmitting data line is used to transmit a high-speed serial transmitting signal (PCIe_TX) externally.
[0193] exist Figure 4Specifically, the user can initiate a compression command through the host, and then the central processor 404 initiates a first compression instruction to the disk 406. The disk 406 obtains the database to be compressed from the corresponding disk storage address based on the database identifier to be compressed included in the first compression instruction and the disk storage address information in the disk 406, and uses a direct memory access (DMA) data transmission operation to send the database to be compressed to the second memory 4014 through the main line switch 405 through the second interface 4011 and the AXI4_MM line in sequence, wherein the database to be compressed includes at least one sub-database to be compressed, and the same When the CPU 404 sends the compressed information to the second converter 4012 through the main line switch 405 in sequence through the second interface 4011 and the AXI4_Lite line, the second converter 4012 sends the compressed information therein to the corresponding target compression module through the control signal, and at the same time, the second memory 4014 stores at least one sub-database to be compressed to the compressed storage address corresponding to the second memory 4014 based on the compressed storage address information in the second compression instruction, and further, the target compression module obtains the corresponding sub-database to be compressed from each compressed storage address in the second memory 4014 based on the compressed information. Among them, the target compression module belongs to at least one compression module 4013. The number of target compression modules is less than or equal to the number of compression modules 4013 included in the compression acceleration card 401.
[0194] exist Figure 4 In the figure, S1 is the compression acceleration card 401 receiving the database to be compressed, and the database to be compressed is sent by the disk 406 through a direct memory access (DMA) data transfer operation.
[0195] Furthermore, the target compression module compresses the corresponding sub-database to be compressed, obtains a compressed sub-database, and stores the corresponding compressed sub-database in a corresponding compression storage address.
[0196] At the same time, when the target compression module is compressed, a corresponding interrupt sub-signal is generated, and the above interrupt sub-signal is sent to the interrupt counting module 4015. The interrupt counting module 4015 obtains each interrupt sub-signal. When the number of interrupt sub-signals is equal to the number of compression times, a total interrupt signal is generated. The total interrupt signal is sent to the central processing unit 404 through the second interface 4011 to prompt the central processing unit 404 that the compression acceleration card 401 has completed the compression task.
[0197] At the same time, the controller in the compression acceleration card 401 initiates a direct memory access (DMA) data transfer operation, transfers the compressed sub-database stored in the second memory 4014 to the second interface 4011 through the AXI4_MM line, and transfers it to the disk 406 for storage via the main line switch 405.
[0198] exist Figure 4 In the example, S2 is the compression acceleration card 401 sending the compressed database to the disk 406 through a direct memory access data transfer operation.
[0199] Further, the user initiates a decompression command through the host to generate a first decompression instruction and a second decompression instruction, and the central processor 404 sends the first decompression instruction to the disk 406, and the disk 406 obtains the database to be decompressed from the corresponding disk storage address based on the identifier of the database to be decompressed included in the first decompression instruction and the disk storage address information in the disk 406, and adopts a direct memory access (DMA) data transfer operation to send the database to be decompressed to the first memory 4025 through the main line switch 405 in sequence through the first interface 4021 and the AXI4_MM line, wherein the database to be decompressed includes at least one sub-database to be decompressed, and at the same time, the central processor 404 sends the second decompression instruction to the first converter 4022 through the main line switch 405 in sequence through the first interface 4021 and the AXI4_Lite line. The first converter 4022 sends the path information in the second decompression instruction to the corresponding target decompression module through the AXI4_Lite line based on the decompression information in the second decompression instruction. At the same time, the first memory 4025 stores at least one sub-database to be decompressed to the storage address corresponding to the first memory 4025 based on the storage address information in the second decompression instruction. Further, the target decompression module obtains the corresponding sub-database to be decompressed from each storage address in the first memory 4025 based on the path information. Among them, the target decompression module belongs to at least one decompression module 4023. The number of target decompression modules is less than or equal to the number of decompression modules 4023.
[0200] Further, the target decompression module decompresses the corresponding sub-database to be compressed, obtains the decompressed sub-database, and sends the corresponding decompressed sub-database to the interface conversion module 4024 through the AXI_Stream line. The interface conversion module 4024 determines the conversion information of the first decompressed sub-database, and processes it according to the link mode included in the conversion information of the first decompressed sub-database. If the link mode of the first decompressed sub-database is one-to-many, the interface conversion module 4024 segments the first decompressed sub-database to obtain each segmented sub-database, and sends each segmented sub-database to the corresponding first query module; if the link mode of the first decompressed sub-database is many-to-one, the interface conversion module sends the first decompressed sub-database and the decompressed sub-database bound in the conversion information to the corresponding second query module. The first decompressed sub-database is any decompressed sub-database. The interface conversion module 4024 is connected to at least one query module 4027 through the AXI_Stream line. The first query module and the second query module belong to the query module 4027.
[0201] exist Figure 4 In the example, S3 is the decompression acceleration card 402 receiving the database to be decompressed, where the database to be decompressed is sent by the disk 406 through a direct memory access (DMA) data transfer operation.
[0202] Furthermore, the first query module and / or the second query module queries their respective sub-databases to obtain query sub-results.
[0203] At the same time, after the first query module and / or the second query module queries, a corresponding interrupt sub-signal is generated, and the above interrupt sub-signal is sent to the interrupt counting module 4026. The interrupt counting module 4026 calculates the number of queries and obtains each interrupt sub-signal. When the number of interrupt sub-signals is equal to the number of queries, a total interrupt signal is generated, and the total interrupt signal is sent to the central processing unit 404 through the first interface 4021 to prompt the central processing unit 404 that the decompression acceleration card 402 has completed the decompression task and has completed the query of the decompression database.
[0204] At the same time, the controller in the decompression acceleration card 402 initiates a direct memory access (DMA) data transfer operation, transmits the query sub-result stored in the first memory 4025 to the first interface 4021 through the AXI4_MM line, and transmits it to the central processor 404 via the main line switch 405.
[0205] exist Figure 4 In the example, S4 is the decompression acceleration card 402 sending the query result to the central processor 404 .
[0206] In addition, if the database to be decompressed in the first decompression instruction is the compressed database just compressed by the compression acceleration card 401, the compression acceleration card 401 can transmit the compressed database to the decompression acceleration card 402 through the main line switch by a direct memory access (DMA) data transfer operation to realize an end-to-end transmission method.
[0207] exist Figure 4 In S5, the compression acceleration card 401 sends the compressed database (at this time, serving as a database to be decompressed) to the decompression acceleration card 402 through a direct memory access (DMA) data transmission operation.
[0208] It should be noted that the compression acceleration card 401 and the decompression acceleration card 402 include respective controllers, and the controllers may initiate direct memory access (DMA) data transmission operations to implement an end-to-end transmission mode.
[0209] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.
[0210] In the present application, the above-mentioned hardware module can be used to realize the compression of the compressed database and the query of the decompressed database, and a hardware implementation method is provided.
[0211] Figure 5 This is a schematic diagram of the structure of a compression acceleration card provided in an embodiment of the present application. Figure 5 As shown, the compression acceleration card includes: a PCIe interface 501, a first cross connector 502, at least one compression IP core 503, an interrupt counting module 504 and a second memory 505, wherein the at least one compression IP core 503 is a compression IP core-1 503 and a compression IP core-n 503.
[0212] The PCIe interface 501 includes a PCIe IP layer 5011, a read / write request initiating module 5012, a DMA write module 5013, a DMA read module 5014, a read / write request response module 5015, an AXI4_MM write interface 5016, an AXI4_MM read interface 5017, a bridge module 5018, an AXI4_MM bridge interface 5019, a control line bridge interface 5020, and an interrupt control processing module 5021. The PCIe interface 501 is externally connected to a high-speed serial computer expansion bus.
[0213] Among them, one end of the PCIe IP layer 5011 is respectively connected to the first end of the read / write request initiating module 5012 and one end of the read / write request response module 5015, the second end of the read / write request initiating module 5012 is connected to one end of the DMA write module 5013, and the third end of the read / write request initiating module 5012 is connected to one end of the DMA read module 5014; wherein the other end of the DMA write module 5013 is connected to one end of the AXI4_MM write interface 5016, the other end of the DMA read module 5014 is connected to one end of the AXI4_MM read interface 5017, the other end of the read / write request response module 5015 is connected to one end of the bridge module 5018, and the other end of the bridge module 5018 is respectively connected to one end of the AXI4_MM bridge interface 5019 and one end of the control line bridge interface 5020. In the PCIe interface 501, each module is connected through a signal line.
[0214] The other end of the control line bridge interface 5020 is connected to the first end of the first cross connector 502 through an AXI4_Lite line.
[0215] The other end of the read / write control module 506 and the other end of the AXI4_MM bridge interface 5019 are respectively connected to the first end and the second end of the second memory 505 through the AXI4_MM line. The other end of the read / write control module 506 includes the other end of the DMA write module 5013 and the DMA read module 5014.
[0216] Among them, the second end of the first cross connector 502 is connected to one end of at least one compression IP core 503 through an AXI4_Lite line, the second end of each compression IP core 503 is connected to the first end of the interrupt counting module 504 through a signal line, and the third end of the first cross connector 502 is connected to the second end of the interrupt counting module 504 through an AXI4_Lite line.
[0217] The third terminal of each compression IP core 503 is connected to the third terminal of the second memory 505 through AXI4_MM.
[0218] The third terminal of the interrupt counting module 504 is connected to the interrupt control processing module 5021 via a signal line.
[0219] The PCIe IP layer 5011 may be a processing layer, which is mainly used to parse the PCIe protocol, construct, send and parse the transaction layer packet (TLP), wherein the TLP is a basic data unit. The PCIe IP layer 5011 communicates with external devices through a high-speed serial receiving signal (PCIe_RX) and a high-speed serial transmitting signal (PCIe_TX).
[0220] The read / write request initiation module 5012 can initiate a request packet for reading / writing data, and send the request packet to the PCIe IP layer 5011, thereby starting data transmission with an external device. Exemplarily, when the compression acceleration card reads data from the host memory, the read / write request initiation module 5012 generates a read request packet and sends it to the host; when data needs to be written from the host memory to the second memory 505 in the compression acceleration card, the read / write request initiation module 5012 initiates a write request packet and sends it to the second memory 505.
[0221] Among them, the DMA write module 5013 and the DMA read module 5014 are the read-write control module 506, which is used to control the initiation, timing and data flow control of the DMA data transmission operation read-write transaction process, and complete the DMA data transmission operation (or data movement). The read-write control module 506 is connected to the second memory 505, and the connection type can be the AXI4_MM interface type, and then it can be directly connected to the back-end AXI4 bus-based device.
[0222] Among them, the read-write request response module 5015 is mainly used for: the read-write request response module 5015 passively receives and responds to the read-write request of the host. In the present application, the read-write request response module 5015 can respond according to the register read-write control or memory read-write request initiated by the host. Exemplarily, when the host accesses the compression IP core 503 or the first cross-connector 502 through the PCIe protocol, the read-write request response module 5015 will obtain the request message, parse the request message through the PCIe protocol, and then transmit the parsed request message to the control line bridge interface 5020 through the bridge module 5018 (converted into a format suitable for the AXI4_Lite protocol), and output it as a register read-write control type interface. For example, when the host needs to control the compression IP core 503 to start or read the register value of the compression IP core 503 (for example, When the state of the compression IP core 503 is detected, the control signal is transmitted to the compression IP core 503 through the control line bridge interface 5020; when the end-to-end transmission mode needs to be realized, the corresponding request response is completed through the read-write request response module 5015 and the AXI4_MM bridge interface 5019; the read-write request response module 5015 realizes the conversion of the address between the PCIe protocol and the AXI4 bus protocol, so that the devices on other PCIe main line switches can access the address space connected to this interface through the read-write request response module 5015. In this application, the control line bridge interface 5020 can be used as one of the Master devices of the second memory 505, and can read and write the second memory. In this path, the compression acceleration card, as a responder, does not need to initiate a DMA data transmission operation, but only needs to realize the relevant protocol conversion, and through the base address register space (Base Address Register, referred to as BAR), the storage address of the compression acceleration card can be exposed on the PCIe bus switch, so that the end-to-end transmission mode can be realized, reducing the delay caused by multiple DMA data transmission operations (or data movement) in the related technology. Compared with the related art, in the present application, the storage address of the compression acceleration card can be exposed on the PCIe bus switch, thereby realizing an end-to-end transmission mode.
[0223] The control line bridge interface 5020 may be an AXI_Lite_MM interface.
[0224] The interrupt control processing module 5021 can be used to process the triggered total interrupt signal. When the compression IP core 503 completes the calculation, an interrupt sub-signal will be generated. The interrupt control processing module 5021 can collect the total interrupt signal, and then process the data format and feed it back to the host. The triggered interrupt sub-signal is triggered by the interrupt counting module 504.
[0225] Among them, the interrupt counting module 504 will output an interrupt sub-signal to the outside when any compression IP core 503 completes the compression task. In the related art, when multiple compression IP cores complete the corresponding compression tasks, they will feedback to the host that they have been interrupted, so that the host will continuously process the interrupt program, resulting in a decrease in the computing performance of the central processing unit. In order to solve the above problem, the present application adds an interrupt counting module 504, which can be based on the volume of the compression task (i.e., the capacity of the database to be compressed, i.e., the DATA in the present application) total ) and the number of compressed IP cores 503 for parallel compression (the IP cores in this application can be used num Representation) and computing unit single core processing performance (can be used in the IP in this application Throughput The interrupt counting module 504 generates an interrupt (i.e., generates a total interrupt signal) and feeds it back to the host, so that the host reads the compression result. It can be seen that this method can greatly reduce the number of times that important processors handle interrupt programs, thereby reducing the pressure on the central processing unit and improving the overall performance. The decompression times in this application (which can be represented by Count in this application) are calculated by the compression times, so as to intelligently control the timing of interrupt generation. Under the current compression task, when all the compression IP cores 503 in the working state have completed the compression task (i.e., when the number of collected interrupt sub-signals is equal to the compression times), the interrupt counting module 504 generates an interrupt (i.e., generates a total interrupt signal) and feeds it back to the host, so that the host reads the compression result. num It can be changed dynamically and does not require manual configuration by the user, so the compression efficiency can be improved and the flexibility of the compression accelerator card is increased.
[0226] in, Figure 5 It is also shown in the figure that the high-speed serial computer expansion bus can transmit high-speed serial receiving signals (PCIe_RX) and high-speed serial transmitting signals (PCIe_TX). The decompression accelerator card also includes AXI4_MM lines and control lines; the control line can be an AXI4_Lite line. Figure 5 The irq n here refers to the nth interrupt sub-signal, or the nth interrupt request.
[0227] The compression IP core 503 is a form of the compression module in the present application. It should be noted that the compression IP core 503 can use a preset tool to implement the compression function. The preset tool can be a high-level synthesis tool (High-Level Synthesis, HLS for short).
[0228] The first cross-connector 502 is a compression converter in this application.
[0229] The second memory 505 may be a high bandwidth memory (HBM). The second memory 505 is formed by an integrated circuit, which includes a switch function, so that multiple masters can access it simultaneously.
[0230] Figure 6 This is a schematic diagram of the structure of a decompression acceleration card provided in an embodiment of the present application. Figure 6 As shown, the decompression acceleration card includes: a PCIe interface 601 , a second cross connector 602 , at least one decompression IP core 603 , an interface conversion module 604 , at least one query IP core 605 , a first memory 606 and an interrupt counting module 607 . Figure 6 In the embodiment, the decompression of IP core 603 includes decompression of IP core-1 603 and decompression of IP core-n 603, and the query of IP core 605 includes query of IP core-1 and query of IP core-n.
[0231] The PCIe interface 601 includes an AXI4_MM interface 6011, an AXI4_MM bridge interface 6012, a control line bridge interface 6013, and an interrupt control processing module 6014. The AXI4_MM interface 6011 is a read-write control module. Figure 5 The same as the read-write control module in.
[0232] The PCIe interface 601 further includes a PCIe IP layer, a read / write request initiating module, a DMA writing module, a DMA reading module, a read / write request responding module, and a bridge module ( Figure 6 The connection mode and functions of the above modules are similar to Figure 5 The PCIe interface 501 of the compression acceleration card is the same as that of the PCIe interface 501 of the compression acceleration card, and will not be described here. Figure 6 In the figure, only the AXI4_MM interface 6011, the AXI4_MM bridge interface 6012, the control line bridge interface 6013 and the interrupt control processing module 6014 are shown.
[0233] Among them, the decompression function of the decompression IP core 603 can be developed using the HLS tool, which will not be described here. In the present application, there are the following differences compared to the related art: in the related art, the relevant decompression IP core is a single AXI_MM interface, and the relevant decompression IP core is directly connected to the corresponding memory. When decompression is required, the relevant decompression IP core obtains the database to be decompressed from the corresponding memory, and returns the decompressed database to the corresponding memory after the decompression task is completed. However, in the present application, the decompression IP core 603 obtains the decompressed database after decompression, and the decompressed database can be directly sent to the query IP core 605 through the AXI_Stream line bypassing the first memory 606. During the transmission process, the faster AXI_Stream line is used for implementation.
[0234] Among them, after the decompression IP core 603 outputs the decompressed database, Figure 6 The output FIFO in the interface conversion module 604 is transmitted. The output FIFO or the input FIFO can be used to cache the decompression database to ensure orderly data transmission. FIFO (First In First Out, referred to as FIFO) means first in first out. The decompression IP core 603 is the decompression module in this application.
[0235] Figure 6 The decompression IP matrix 608 is also included, and the decompression IP matrix 608 includes at least one decompression IP core 603 and an output FIFO.
[0236] Figure 6 The query IP matrix 609 is also included, and the query IP matrix 609 includes at least one query IP core 605 and an input FIFO.
[0237] Wherein, interface conversion module 604 can realize "one to many" or "many to one" dynamic link mode through internal conversion function according to the throughput of decompression IP core 603 and query IP core 605, and specific implementation mode can refer to the method description in this application, which will not be repeated here. Interface conversion module 604 in this application can ensure the performance balance between each query IP core 605 on the pipeline, thereby improving overall acceleration. A conversion controller for controlling switching is included in interface conversion module 604, which is configured by host through AXI4_Lite interface. The parameters of the above-mentioned conversion controller.
[0238] in, Figure 6 The interface conversion module 604 also includes an input FIFO corresponding to each query IP core 605. The input FIFO is connected to the interface conversion module 604 in front and connected to the corresponding query IP core 605 in the back. Specifically, the sub-database transmitted from the interface conversion module 604 can be input to the corresponding input FIFO, and then input to the corresponding query IP core 605 by the input FIFO, and then the query IP core 605 queries or filters the sub-database. The sub-database is as described above and will not be repeated here.
[0239] The query IP core 605 is similar in structure to the decompression IP core 603 and can also be developed using the HLS tool. The query IP core 605 can output an interrupt sub-signal and send it to the interrupt counting module 607. The query IP core 605 is a query module in this application.
[0240] The function of the interrupt counting module 607 is consistent with that of the interrupt counting module in the compression acceleration card, which will not be described in detail here. Figure 6It also includes a high-speed serial computer expansion bus that can transmit serial receive signals (PCIe_RX), high-speed serial transmit signals (PCIe_TX), and AXI4_MM lines. The control line can be an AXI4_Lite line, and it also includes an AXI_Stream line and a signal line.
[0241] One end of the AXI4_MM interface 6011 is connected to a first end of the first memory 606 via an AXI4_MM line; one end of the AXI4_MM bridge interface 6012 is connected to a second end of the first memory 606 via an AXI4_MM line.
[0242] Among them, one end of the control line bridge interface 6013 is connected to one end of the second cross connector 602, the other end of the second cross connector 602 is connected to the first end of at least one decompression IP core 603 through the AXI4_Lite line, and the second end of the decompression IP core 603 is connected to one end of the output FIFO through the AXI_Stream line. The other end of the output FIFO is connected to the first end of the interface conversion module 604 through the AXI_Stream line, the second end of the interface conversion module 604 is connected to one end of the input FIFO through the AXI_Stream line, and the other end of each input FIFO is respectively connected to the first end of the corresponding query IP core 605 through the AXI_Stream line. The second end of the query IP core 605 is connected to the first end of the interrupt counting module 607 through the signal line.
[0243] The third terminal of the query IP core 605 is connected to the third terminal of the first memory 606 through an AXI4_MM line.
[0244] Among them, the other end of the second cross connector 602 is connected to the third end of the interface conversion module 604 through the AXI4_Lite line, the other end of the second cross connector 602 is connected to the fourth end of each query IP core 605 through the AXI4_Lite line, and the other end of the second cross connector 602 is connected to the second end of the interrupt counting module 607 through the AXI4_Lite line.
[0245] The second end of the interrupt counting module 607 is connected to one end of the interrupt control processing module 6014 through a signal line.
[0246] Figure 7 The following is a schematic diagram of the structure of a data decompression device provided in an embodiment of the present application. Figure 7 As shown, the present application provides a data decompression device 700, which is located in a decompression acceleration card in a computing system. The computing system also includes a central processing unit, a disk, and a compression acceleration card. The data decompression device 700 includes the following modules:
[0247] The receiving and storing module 701 is used to receive and store the database to be decompressed sent by the disk to realize the end-to-end transmission mode; the database to be decompressed is sent by the controller in the disk after the disk receives the first decompression instruction sent by the central processor to initiate the direct memory access data transmission operation; the database to be decompressed is obtained by compressing the database to be compressed by the compression acceleration card.
[0248] The decompression module 702 is used to decompress the database to be decompressed to obtain a decompressed database.
[0249] Figure 8 This is a schematic diagram of the structure of another data decompression device provided in an embodiment of the present application. Figure 8 As shown, the data decompression device 800 is applied to a compression acceleration card in a computing system. The computing system also includes a central processing unit, a disk and a decompression acceleration card. The data decompression device 800 includes the following modules:
[0250] The receiving and storing module 801 is used to receive and store the database to be compressed sent by the disk to realize the end-to-end transmission mode; the database to be compressed is sent by the controller in the disk initiating a direct memory access data transmission operation after the disk receives the first compression instruction sent by the central processor.
[0251] The compression module 802 is used to compress the database to be compressed to obtain a compressed database.
[0252] The sending module 803 is used to send the compressed database to the disk through a direct memory access data transmission operation, so that the disk stores at least one compressed database, and provides a database to be decompressed for the decompression acceleration card, so that the decompression acceleration card decompresses based on the database to be decompressed; the database to be decompressed comes from at least one compressed database; the database to be decompressed is sent to the decompression acceleration card by a controller in the disk initiating a direct memory access data transmission operation after the disk receives a first decompression instruction sent by the central processing unit.
[0253] The description of the features in the embodiment corresponding to the data decompression device can refer to the relevant description of the embodiment corresponding to the data decompression method, which will not be repeated here.
[0254] Fig. 9 This is a schematic diagram of the structure of an acceleration card provided in this application. Fig. 9 As shown, the acceleration card 900 provided in this embodiment includes: at least one processor 901 and a memory 902. Optionally, the acceleration card 900 also includes a communication component 903. The processor 901, the memory 902 and the communication component 903 are connected via a bus 904.
[0255] In a specific implementation process, at least one processor 901 executes the computer execution instructions stored in the memory 902, so that at least one processor 901 executes the above-mentioned data decompression method embodiment.
[0256] The specific implementation process of the processor 901 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0257] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0258] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.
[0259] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.
[0260] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned data decompression method embodiments when running.
[0261] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0262] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above data decompression method embodiments are implemented.
[0263] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned data decompression method embodiments are implemented.
[0264] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0265] The above is a detailed introduction to a data decompression method provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.
Claims
1. A data decompression method, characterized in that: The method is applied to a decompression acceleration card in a computing system, wherein the computing system further includes a central processing unit, a disk and a compression acceleration card, including: Receive and store the database to be decompressed sent by the disk to realize the end-to-end transmission mode; the database to be decompressed is sent by the controller in the disk initiating a direct memory access data transmission operation after the disk receives the first decompression instruction sent by the central processor; the database to be decompressed is obtained by compressing the database to be compressed by the compression acceleration card; The database to be decompressed is decompressed to obtain a decompressed database.
2. The method according to claim 1, characterized in that The database to be decompressed includes at least one sub-database to be decompressed; the decompression acceleration card also includes a first memory; The receiving and storing of the database to be decompressed sent by the disk to realize the end-to-end transmission mode includes: Receive a database to be decompressed to achieve an end-to-end transmission mode, and receive a second decompression instruction sent by the central processor; the second decompression instruction includes storage information; the storage information includes storage address information of at least one sub-database to be decompressed in the first memory; The first memory is used to store at least one of the to-be-decompressed sub-databases at a corresponding storage address in the first memory according to the storage address information.
3. The method according to claim 1, characterized in that The decompression acceleration card also includes a decompression converter and at least one decompression module; Before decompressing the database to be decompressed to obtain the decompressed database, the method includes: A decompression converter is used to receive decompression information in the second decompression instruction sent by the central processor; the decompression information includes path information of each sub-database to be decompressed and a corresponding target decompression module identifier; the target decompression module identifier is a decompression module identifier for decompressing the corresponding sub-database to be decompressed; For any path information, a decompression converter is used to send the path information to a target decompression module identified by a corresponding target decompression module, and the target decompression module is used to obtain a corresponding sub-database to be decompressed from a first memory based on the corresponding path information; the path information includes the sub-database identification to be decompressed and the storage address information corresponding to the sub-database to be decompressed.
4. The method according to claim 3, characterized in that The step of using the target decompression module to obtain the corresponding sub-database to be decompressed from the first memory based on the corresponding path information includes: Using the target decompression module to locate the corresponding storage address from the first memory based on the corresponding storage address information; The sub-database to be decompressed corresponding to the sub-database to be decompressed identifier is obtained from the storage address; and the decompression performance of the target decompression module satisfies the decompression of the sub-database to be decompressed.
5. The method according to claim 2, characterized in that: The step of decompressing the database to be decompressed to obtain a decompressed database includes: Each target decompression module is used to decompress the respective sub-database to be decompressed in parallel to obtain each decompressed sub-database respectively; the target decompression module is a decompression module that decompresses the corresponding sub-database to be decompressed.
6. The method according to claim 5, characterized in that The decompression acceleration card also includes an interface conversion module and at least one query module; After the target decompression modules are used to decompress the respective sub-databases to be decompressed in parallel to obtain the decompressed sub-databases respectively, the method further includes: Using each of the target decompression modules to directly send the corresponding decompression sub-database to the interface conversion module bypassing the first memory; The interface conversion module is used to send the corresponding decompressed sub-database to the corresponding query module according to the conversion information; the conversion information includes the link mode of each decompressed sub-database and the query module identifier corresponding to each decompressed sub-database; the conversion information is determined by the interface conversion module; Using the query module to query the corresponding decompressed sub-database to obtain query results; The query result is sent to the central processing unit.
7. The method according to claim 6, characterized in that The step of using the interface conversion module to send the corresponding decompressed sub-database to the corresponding query module according to the conversion information includes: If the link mode of the first decompressed sub-database is one-to-many, the interface conversion module is used to segment the first decompressed sub-database according to the conversion information, and each segmented sub-database is sent to a plurality of first query modules accordingly; the first query module is a query module corresponding to the query module identifier of the first decompressed sub-database in the conversion information; the first decompressed sub-database is any one of the decompressed sub-databases; If the link mode of the first decompressed sub-database is many-to-one, the interface conversion module is used to send the first decompressed sub-database and the decompressed sub-database bound in the conversion information to the corresponding second query module; the second query module is a query module corresponding to the query module identifier of the first decompressed sub-database and the decompressed sub-database bound in the conversion information; The method further comprises: Each of the first query modules and / or each of the second query modules queries the corresponding decompressed sub-database in parallel, and the query module performing the query is a working status query module.
8. The method according to claim 7, characterized in that The decompression acceleration card also includes: an interrupt counting module; The query module is used to query the corresponding decompressed sub-database to obtain the query result, including: Each working state query module is used to query in parallel in the respective sub-database based on the received query target to obtain a corresponding query sub-result, wherein the query sub-result is data that satisfies the query target; the second decompression instruction includes the query target; the working state query module is a query module for performing the query; the sub-database includes the corresponding first decompressed sub-database and its bound decompressed sub-database, or any segmented sub-database after the first decompressed sub-database is segmented; Each working status query module is used to send the corresponding query sub-result to the first memory. If the interrupt counting module is used to generate a total interrupt signal, the first memory obtains the query result; the query result includes at least one query sub-result.
9. The method according to claim 8, characterized in that The method of using an interrupt counting module to generate a total interrupt signal includes: After obtaining the corresponding query sub-result for each working state query module, the working state query module is used to generate an interrupt sub-signal and sent to the interrupt counting module; An interrupt counting module is used to generate a total interrupt signal based on the interrupt sub-signals.
10. The method according to claim 8, characterized in that Before the interrupt counting module is used to generate the total interrupt signal, the method further includes: The interruption counting module is used to calculate the number of queries; the number of queries is calculated based on the capacity of the database to be decompressed, the number of query modules and the throughput corresponding to the query modules; The method of using an interrupt counting module to generate a total interrupt signal based on the interrupt sub-signals includes: The interruption counting module is used to count the number of received interruption sub-signals, and when the number of interruption sub-signals is equal to the number of queries, a total interruption signal is generated.
11. The method according to claim 9, characterized in that The method further comprises: The interrupt counting module is used to send the total interrupt signal to the central processing unit through a high-speed serial computer expansion bus. The total interrupt signal is used to prompt the central processing unit that the query of the database to be decompressed has been completed. The total interrupt signal is also used to prompt the central processing unit that the decompression acceleration card is ready for the next decompression task.
12. A data decompression method, characterized in that: The method is applied to a compression acceleration card in a computing system, wherein the computing system further includes a central processing unit, a disk and a decompression acceleration card, and the method includes: Receiving and storing the database to be compressed sent by the disk to realize the end-to-end transmission mode; the database to be compressed is sent by the controller in the disk initiating the direct memory access data transmission operation after the disk receives the first compression instruction sent by the central processor; Compressing the database to be compressed to obtain a compressed database; The compressed database is sent to the disk through a direct memory access data transmission operation, so that the disk stores at least one of the compressed databases, and provides a database to be decompressed for the decompression acceleration card, so that the decompression acceleration card decompresses based on the database to be decompressed; the database to be decompressed is derived from at least one of the compressed databases; the database to be decompressed is sent to the decompression acceleration card by a controller in the disk initiating a direct memory access data transmission operation after the disk receives a first decompression instruction sent by a central processing unit.
13. An accelerator card, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the data decompression method according to any one of claims 1 to 12 when executing the computer program.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data decompression method according to any one of claims 1 to 12.
15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data decompression method according to any one of claims 1 to 12 are implemented.
Citation Information
Patent Citations
Data transmission method and system, FPGA (Field Programmable Gate Array) heterogeneous accelerator card and storage medium
CN108776648A
Data processing method, device, system and equipment of heterogeneous computing platform and medium
CN113704301A
Multi-database accelerated query method, device and equipment based on FPGA (Field Programmable Gate Array) and medium
CN116383240A
Row data decompression method and system and FPGA (Field Programmable Gate Array) heterogeneous data accelerator card
CN117155400A
Data writing method and device, data reading method and device, equipment and medium
CN117785803A