Data Transmission Method, Device, Medium and Program Product

Abstract

The present invention provides a data transmission method, device, medium and program product, which can be applied to the fields of data security and computer technology. The data transmission method includes: in response to a user's data query request, dividing the data query request into multiple query subtasks; based on the data indexes of the multiple query subtasks respectively, determining the corresponding relationship between the multiple query subtasks and multiple data partitions; based on the corresponding relationship, dividing one or more query subtasks corresponding to the same data partition into the same query subtask group; sequentially sending the multiple query subtasks in the query subtask group to a programmable device in sequence, so that the programmable device queries in a storage unit based on the data index in the query subtask to obtain a query sub-result of the query subtask; and based on the query sub-results of the multiple query subtasks from the programmable device, determining a query result of the data query request.

Description

Technical Field

[0001] The present invention relates to the fields of data security and computer technology, and particularly relates to a data transmission method, device, medium, and program product. Background Art

[0002] With the advent of the big data era, accurate and secure data query from databases storing massive amounts of data has become a major challenge. To improve the efficiency and speed of querying massive data, related technologies usually adopt software methods to encrypt and decrypt data and accelerate software queries.

[0003] However, in traditional software encryption / decryption and software-accelerated database query methods, both software encryption / decryption and software-accelerated database query belong to the category of software acceleration technologies, and ultimately need to be executed by a central processing unit (CPU). This will consume a large amount of CPU computing resources in the server, affect other software programs in the system, and affect the stable operation of the server. Summary of the Invention

[0004] In view of the above problems, the present invention provides a data transmission method, device, medium, and program product.

[0005] According to a first aspect of the present invention, there is provided a data transmission method applied to a server side, including: in response to a user's data query request, dividing the data query request into multiple query subtasks; based on the data indexes of the multiple query subtasks respectively, determining the corresponding relationships between the multiple query subtasks and multiple data partitions included in a storage unit; based on the corresponding relationships, dividing one or more query subtasks corresponding to the same data partition into the same query subtask group; sequentially sending the multiple query subtasks in the query subtask group to a programmable device, so that the programmable device queries in the storage unit based on the data indexes in the query subtasks to obtain query sub-results of the query subtasks, where the storage unit is divided into multiple data partitions; and based on the query sub-results of the multiple query subtasks from the programmable device, determining the query result of the data query request.

[0006] The second aspect of the present invention provides a data transmission method, which is applied to a programmable device and includes: in response to a plurality of query subtasks sent by a server, querying in a storage unit based on the data index in the query subtasks to obtain query sub-results of the query subtasks, wherein the plurality of query subtasks are obtained by the server dividing a data query request in response to a user's data query request, and the plurality of query subtasks are sequentially sent by the server in an ordered form; and sending the query sub-results of the query subtasks to the server so that the server determines the query result of the data query request based on the query sub-results of the respective query subtasks; wherein the server sends the plurality of query subtasks to the programmable device in sequence in the following manner: determining the correspondence between the plurality of query subtasks and a plurality of data partitions based on the respective data indexes of the plurality of query subtasks; dividing one or more query subtasks corresponding to the same data partition into the same query subtask group based on the correspondence; and sending the plurality of query subtasks in the query subtask group to the programmable device in sequence.

[0007] The third aspect of the present invention provides a data transmission device applied to a server, including: a task division module for dividing a data query request into a plurality of query subtasks in response to a user's data query request; a relationship determination module for determining the correspondence between the plurality of query subtasks and a plurality of data partitions based on the respective data indexes of the plurality of query subtasks; a task grouping module for dividing one or more query subtasks corresponding to the same data partition into the same query subtask group based on the correspondence; a task sending module for sending the plurality of query subtasks in the query subtask group to the programmable device in sequence so that the programmable device queries in the storage unit based on the data index in the query subtasks to obtain query sub-results of the query subtasks; and a result determination module for determining the query result of the data query request based on the query sub-results of the respective query subtasks from the programmable device.

[0008] The fourth aspect of the present invention provides a data transmission device applied to a programmable device, including: a data query module for querying in a storage unit based on the data index in the query subtasks in response to a plurality of query subtasks sent by a server to obtain query sub-results of the query subtasks, wherein the plurality of query subtasks are obtained by the server dividing a data query request in response to a user's data query request, and the plurality of query subtasks are sequentially sent by the server in an ordered form; and a result sending module for sending the query sub-results of the query subtasks to the server so that the server determines the query result of the data query request based on the query sub-results of the respective query subtasks.

[0009] A fifth aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0010] A sixth aspect of the present invention further provides a computer-readable storage medium, on which a computer program or instruction is stored, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0011] A seventh aspect of the present invention further provides a computer program product, comprising a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0012] According to an embodiment of the present invention, the server is used to divide a data query request into multiple query subtasks, and sequentially send the query subtasks to a programmable device. The programmable device is used for pipelined query, and the step with the largest time delay in the data query process is migrated to an external device for execution, thereby reducing the data processing amount and complexity of the server and improving the efficiency of data query and data transmission. In addition, when the programmable device is used to implement data query, the programmable device only performs point-to-point communication with the storage unit and the server, so the problem of data leakage can be avoided and data security can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Through the following description of the embodiments of the present invention with reference to the drawings, the above content and other objects, features and advantages of the present invention will become clearer. In the drawings:

[0014] Figure 1 The application scenario diagram of the data transmission method, device, medium and program product according to the embodiment of the present invention is shown.

[0015] Figure 2 The flowchart of the data transmission method applied to the server according to the embodiment of the present invention is shown.

[0016] Figure 3 The schematic diagram of the task sending timing and task execution timing of the query subtask group according to the embodiment of the present invention is shown.

[0017] Figure 4 The flowchart of the data transmission method applied to the programmable device according to the embodiment of the present invention is shown.

[0018] Figure 5 The structural diagram of the programmable device according to the embodiment of the present invention is shown.

[0019] Figure 6 The flowchart of the data transmission process of a query subtask group according to the embodiment of the present invention is shown.

[0020] Figure 7 The block diagram of a data transmission device applied to a server according to an embodiment of the present invention is shown.

[0021] Figure 8 The block diagram of a data transmission device applied to a programmable device according to an embodiment of the present invention is shown.

[0022] Figure 9 The block diagram of an electronic device suitable for implementing a data transmission method according to an embodiment of the present invention is shown. Detailed implementation manners

[0023] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, many specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present invention.

[0024] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "including", "comprising" and the like used herein indicate the presence of the described features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0026] In the case of using expressions such as "at least one of A, B, and C", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but is not limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C).

[0027] In addition, since there is an online upgrade problem with the query software during the process of implementing accelerated query through software technology, it is difficult for the query software to be used offline, and there is a security risk that the data query process can be obtained by a third party.

[0028] An embodiment of the present invention provides a data transmission method, which is applied to a server and includes: in response to a user's data query request, dividing the data query request into multiple query subtasks; sequentially sending the multiple query subtasks to a programmable device in order, so that the programmable device queries in a storage unit based on the data index in the query subtask to obtain a query sub-result of the query subtask; and determining a query result of the data query request based on the query sub-results of the multiple query subtasks from the programmable device.

[0029] Figure 1 FIG. shows an application scenario diagram of a data transmission method, device, medium, and program product according to an embodiment of the present invention.

[0030] As Figure 1 shown, the application scenario 100 according to this embodiment may include a client 101, a server 102, a network 103, a programmable device 104, and a storage unit 105.

[0031] The user logs in to the client 101 and communicates with the server 102 through the network 103, and sends a data query request to the server 102.

[0032] The server 102 is connected to the programmable device 104, and transmits the user's data query request to the programmable device 104 in a point-to-point manner and obtains a query result from the programmable device 104. The server 102 sends the query result to the client 101 through the network 103.

[0033] The programmable device 104 is connected to the storage unit 105. When receiving the data query request sent by the server 102, it obtains data from the storage unit 105, queries the query result of the data query request therefrom, and sends the query result to the server 102.

[0034] It should be understood that Figure 1 the numbers of the client, server, network, programmable device, and storage unit in

[0035] are merely illustrative. According to actual needs, there may be any number of clients, servers, networks, programmable devices, and storage units. Figure 1 The following will be based on Figures 2 to 6 the scenario described above, and will describe in detail the data transmission method of the embodiment of the invention through

[0036] Figure 2 FIG. shows a flowchart of a data transmission method applied to a server according to an embodiment of the present invention.

[0037] As Figure 2 shown, the data transmission method of this embodiment includes operation S210 to operation S250.

[0038] In operation S210, in response to a user's data query request, the data query request is divided into multiple query subtasks.

[0039] The data query request submitted by the user may include restrictions such as data sources and filtering conditions. The server can construct a corresponding Structured Query Language (SQL) based on the data sources and filtering conditions selected by the user, and based on the SQL statement, divide the data query request into multiple query subtasks.

[0040] In one embodiment, the SQL statement can be parsed to determine an Abstract Syntax Tree (AST) corresponding to the SQL statement, and based on the AST, the syntax and semantics are determined to determine the division method of the data query request.

[0041] For example, in the case where it is determined according to the AST that the SQL statement represents that data needs to be queried from multiple different partitions of a data table, the division can be based on the partitions, so that each query subtask only needs to query data from a single partition.

[0042] In operation S220, based on the data indexes of the multiple query subtasks respectively, the corresponding relationships between the multiple query subtasks and the multiple data partitions included in the storage unit are determined.

[0043] The corresponding relationships between the multiple query subtasks and the multiple data partitions can be determined based on the data indexes of the multiple query subtasks respectively. Among them, in the case where it is determined according to the data index that data needs to be read from a certain data partition and the query sub-result of the current query subtask is retrieved from the data, it can be determined that there is a corresponding relationship between the current query sub-result and the task partition.

[0044] In operation S230, based on the corresponding relationships, one or more query subtasks corresponding to the same data partition are divided into the same query subtask group.

[0045] Based on the corresponding relationships, one or more query subtasks corresponding to each data partition can be determined, and one or more query subtasks corresponding to the same data partition are divided into the same query subtask group.

[0046] In operation S240, the multiple query subtasks in the query subtask group are sequentially sent to the programmable device in order, so that the programmable device queries in the storage unit based on the data indexes in the query subtasks to obtain the query sub-results of the query subtasks.

[0047] Divide the data query request into multiple query subtasks. The data index of each query subtask may include restrictions such as the data source and filtering conditions of the query subtask.

[0048] During the processing of each query subtask, there may be multiple processing steps. A programmable device may include multiple processing components, and the processing components correspond one by one to the processing steps. Thus, after the programmable device receives the query subtasks sent in sequence, multiple processing components can process multiple query subtasks in a pipeline form, enabling multiple processing components to work continuously without waiting for the previous query subtask to complete execution.

[0049] Based on the data index, determine the data source of the query subtask. After the programmable device sequentially executes multiple processing steps of the query subtask, the query sub-result can be obtained.

[0050] In operation S250, based on the query sub-results of multiple query subtasks from the programmable device, determine the query result of the data query request.

[0051] Summarize and splice the multiple query sub-results from the programmable device to obtain the query result of the query task request, and feedback the query result to the user. According to the embodiments of the present invention, use the server to divide the data query request into multiple query subtasks, and sequentially send the query subtasks to the programmable device. Use the programmable device for pipeline query, and migrate the step with the largest time delay in the data query process to an external device, thereby reducing the data processing volume and complexity of the server and improving the efficiency of data query and data transmission. In addition, during the process of using the programmable device to implement data query, the programmable device only performs point-to-point communication with the storage unit and the server, so the problem of data leakage can be avoided and data security can be improved.

[0052] After dividing the data query request into multiple query subtasks, the multiple query subtasks can be directly sent to the programmable device in sequence according to the order, so that the programmable device can sequentially execute multiple query subtasks to obtain the query sub-results of multiple query subtasks. However, in the case of a large data query request, the number of divided query subtasks is huge. Sending multiple query subtasks directly to the programmable device in a sequential form is likely to result in an excessive number of query sub-results, which is difficult to manage and count. Therefore, the multiple query subtasks can be divided into multiple query subtask groups, and the data query and data transmission can be performed with the query subtask group as the processing granularity.

[0053] Each query subtask group includes S query subtasks, where S represents the number of processing steps required for the programmable device to perform a query in the storage unit to obtain a query sub-result, and S is a positive integer; a plurality of query subtasks set in sequential form are sequentially sent to the programmable device, including: sending the first query subtask in the query subtask group to the programmable device for the programmable device to execute, where the query subtask group includes S query subtasks among a plurality of query subtasks; when the programmable device executes to the s-th step, the s-th query subtask is sent to the programmable device, where s = 2, ……, S.

[0054] Figure 3 FIG. shows a schematic diagram of the task sending timing and task execution timing of the query subtask group according to an embodiment of the present invention.

[0055] As Figure 3 shown, taking S = 4 as an example, each query subtask is processed in four steps in the programmable device to obtain a query sub-result. Therefore, except for the case where the remaining query sub-results in the last query subtask group are less than 4 after dividing into multiple query subtask groups, each of the remaining query subtask groups includes four query subtasks.

[0056] Among them, when the number of query subtasks corresponding to a data partition is greater than S, the multiple query subtasks corresponding to the data partition can be divided into multiple query subtask groups. Except for the case where the remaining query sub-results in the last query subtask group are less than S after dividing into multiple query subtask groups, each of the remaining query subtask groups includes four query subtasks.

[0057] Since each processing component in the programmable device cannot process multiple query subtasks simultaneously, therefore, by sending query subtasks in a pipeline form, it can be ensured that after each processing component finishes processing the current query subtask, it starts to process the corresponding processing step of the next query subtask.

[0058] When dividing the processing steps, through a pre-planned method, try to make the respective processing durations of each processing step the same. Therefore, it is only necessary to control that after the first processing step of the current query subtask ends, the next query subtask is sent to the programmable device for processing to ensure the smooth flow of the pipeline.

[0059] Therefore, it can be determined that for each step executed by the programmable device, the server sends a query subtask to the programmable device. That is, when the programmable device executes to the s-th step, the s-th query subtask is sent to the programmable device.

[0060] In this embodiment, multiple query subtasks are sent in a sequential form. The programmable device can complete the data query and processing of 4 query subtasks in the query subtask group after 7 time cycles. If the 4 query subtasks are executed sequentially in a serial manner, 16 time cycles are required. Therefore, by processing multiple query subtasks in a pipeline form, the processing efficiency of the query subtask group can be significantly improved. By controlling the sending time of the query subtasks, it can ensure the normal operation of each processing component in the programmable device. For the query subtasks, it can also ensure that multiple processing steps in the query subtasks are executed sequentially and continuously, ensuring the accuracy of data transmission and data query.

[0061] Since multiple query subtasks are obtained by dividing a data query request, the query sub-results of each query subtask also need to be combined to obtain the query result of the data query request. Also, since multiple query subtasks are sent to the programmable device in units of query subtask groups, after obtaining the query sub-results of multiple query subtasks in each query subtask group respectively, multiple query sub-results can be combined to obtain the query result of the query subtask group, and the query result of the query subtask group can be returned to the server as the processing unit.

[0062] Therefore, the server needs to determine the query result of the data query request based on the query sub-results of multiple query subtasks from the programmable device.

[0063] In response to the server receiving the second query result and one or more first query results sent by the programmable device, the second query result and one or more first query results are concatenated to obtain the query result, where the first query result is determined by the programmable device based on the query sub-results of S query subtasks in query subtask groups other than the last query subtask group, and the second query result is determined by the programmable device based on the query sub-results of one or more query subtasks in the last query subtask group.

[0064] The query subtasks obtained by dividing the data query request are combined into multiple query subtask groups. In multiple query subtask groups, except for the last query subtask group, other query subtask groups include S query subtasks. According to different situations of the total number of query subtasks, the last query subtask group may include S or fewer query subtasks.

[0065] Since multiple query subtasks in each query subtask group are executed sequentially, for each query subtask group, after the S-th query subtask is executed, it can be determined that multiple query subtasks in the query subtask group have all been executed, and the first query result can be determined based on the query sub-results of multiple query subtasks.

[0066] For the last query subtask group, since the last query subtask in the last query subtask group is the last one among the multiple query subtasks of the data query request, that is, after one or more query subtasks in the last query subtask group are executed, there are no unexecuted query subtasks. Therefore, one or more query sub-results obtained by the programmable device at this time can be determined as the second query result for each query subtask.

[0067] Each time the programmable device determines a first query result or a second query result, it sends it to the server. After receiving the second query result, the server can determine that the data of the programmable device has been sent completely. Therefore, operations such as splicing and combining the received second query result and one or more first query results can be performed to obtain the query result.

[0068] In the case where the amount of data required for the data query request is large, there will be a situation where the number of query subtask groups is too large, or the first query result or the second query result obtained by each query subtask group is too large. Directly transmitting the first query result or the second query result will reduce the data query efficiency due to a large amount of transmission delay.

[0069] Therefore, before sending the first query result and the second query result, the programmable device can compress the first query result and the second query result respectively to obtain the first compression result and the second compression result, and send the first compression result and the second compression result to the server, so as to reduce the size of the transmitted data through data compression and improve the data transmission efficiency.

[0070] After receiving the first compression result and the second compression result, the server can use a data decompression method that matches the data compression method used for data compression with the programmable device to decompress the second compression result and one or more first compression results to obtain the second query result and one or more first query results. Among them, the second compression result is obtained by the programmable device using the compression component to compress the second query result, and the first compression result is obtained by the programmable device using the compression component to compress the first query result; and based on the task order of the query subtasks corresponding to one or more first query results, splice the second query result and one or more first query results to obtain the query result.

[0071] After completing the decompression of the data and restoring the first query result and the second query result, the multiple query results can be spliced or combined according to the task order of the second query result and the query subtasks corresponding to each first query result to obtain the query result.

[0072] According to an embodiment of the present invention, decompressing the compressed data and splicing and combining to obtain a query result can ensure the accuracy of the data on the premise of reducing the channel load during the transmission process.

[0073] Through the above method, the response to the data query request can be quickly completed, and the query result of the data query request can be obtained. However, for high-frequency data query requests, even if the response speed is fast, repeated data transmission will cause resource waste. Therefore, after the server obtains the query result, the query result can be temporarily stored in the cache of the server, and the cache eviction policy, such as First In First Out (FIFO), Least Recently Used (LRU), and Least Frequently Used (LFU), can be used to periodically update the server cache, so as to respond to the user's data query request. When it is determined that the query result corresponding to the data query request is stored in the server cache, obtain the query result from the server cache; and send the query result to the client and save the query result in the server cache.

[0074] According to an embodiment of the present invention, part of the query result is stored using the server cache, so that when the query request hits the query result, the query result can be directly obtained from the server cache, further improving the efficiency of data transmission.

[0075] By setting a programmable device, some steps of data query can be transferred to the hardware outside the server, thereby reducing the processing load of the server. Since the processing processes of obtaining data from the storage unit and storing data in the storage unit are opposite, when the programmable device can perform data query, the programmable device can also be used to store data in the storage unit to further reduce the server load.

[0076] During the data storage process, in response to a data storage request, the data to be stored included in the data storage request is sent to the programmable device, so that the programmable device uses the data encryption component to encrypt the data to be stored to obtain encrypted data, and sends the encrypted data to the storage unit for storage.

[0077] The data storage request includes the data to be stored and the target address for storage. The data to be stored and the storage address are sent to the programmable device, and the programmable device processes the data encryption, data sending and other processes required during the data storage process, so as to store the encrypted data to be stored at the target address of the storage unit, further reducing the processing process of the server and improving the hardware utilization rate of the programmable device.

[0078] According to an embodiment of the present invention, the data transmission method further includes: based on the data indexes of multiple query subtasks, dividing a storage unit into multiple data partitions such that each data partition includes at least a query sub-result of one query subtask; sending the data partition result to a programmable device so that the programmable device determines a target partition from the multiple data partitions based on the data partition result and the data indexes in the query subtasks; and obtaining the query sub-result of the query subtask from the target partition.

[0079] According to an embodiment of the present invention, before executing multiple query subtasks, the storage unit can be divided into multiple data partitions according to the data indexes of the multiple query subtasks, and one or more query subtasks included in each data partition can be determined. When dividing the query subtasks into query subtask groups, the query subtasks on the same data partition can be preferentially divided into the same query subtask group, so that when the programmable device processes the query subtask group, it only needs to obtain the storage content of the data partition from the storage unit to obtain the query sub-results of the multiple query subtasks. Thus, for a query subtask group, the number of data partitions to be queried can be reduced, and the query efficiency can be improved.

[0080] A large amount of data can be stored in the storage unit. Even after data partitioning, there will still be a situation where some data partitions are too large. In this case, to execute a query subtask, the content of the data partition needs to be obtained from the storage unit. When the content of the data partition is frequently queried, the query efficiency will be reduced.

[0081] Therefore, the above problem can be solved by means of caching. Based on the access frequency of the programmable device to the storage unit, the data with an access frequency higher than the access threshold is cached in a solid-state drive; when it is determined that the query sub-result is stored on the solid-state drive, the programmable device is used to obtain the query sub-result from the solid-state drive based on the data index via the programmable device.

[0082] The solid-state drive can be arranged between the storage unit and the programmable device. After receiving a query subtask, the programmable device can determine whether the access frequency of the data source of the query subtask is higher than the access threshold. If it is determined to be higher than the access threshold, the query sub-result can be directly obtained from the solid-state drive.

[0083] According to an embodiment of the present invention, by arranging a solid-state drive between the programmable device and the storage unit, the amount of data processing in the intermediate process of data transmission can be further reduced by adding external hardware, and the data query and data transmission efficiency can be improved.

[0084] Figure 4 The flowchart of the data transmission method applied to a programmable device according to an embodiment of the present invention is shown.

[0085] As Figure 4 shown, the data transmission method of this embodiment includes operations S410 to S420.

[0086] In operation S410, in response to multiple query subtasks sent by the server, based on the data indexes in the query subtasks, a query is performed in the storage unit to obtain query sub-results of the query subtasks.

[0087] Among them, the multiple query subtasks are obtained by the server dividing the data query request in response to a user's data query request, and the multiple query subtasks are sent by the server in sequence.

[0088] The programmable device sequentially executes the multiple query subtasks based on the order in which the query subtasks are received, and obtains query sub-results of each of the multiple query subtasks.

[0089] In operation S420, the query sub-results of the query subtasks are sent to the server so that the server determines the query result of the data query request based on the query sub-results of each of the multiple query subtasks.

[0090] The server can send the multiple query subtasks to the programmable device in sequence in the following manner: based on the data indexes of each of the multiple query subtasks, determine the correspondence between the multiple query subtasks and multiple data partitions; based on the correspondence, divide one or more query subtasks corresponding to the same data partition into the same query subtask group; and send the multiple query subtasks in the query subtask group to the programmable device in sequence.

[0091] The programmable device can use the query subtask group as the processing granularity. After obtaining the query sub-results of each of the multiple query subtasks in the query subtask group, splice the multiple query sub-results and send them to the server to reduce the number of data transmissions between the programmable device and the server and improve the utilization rate of the data transmission efficiency channel.

[0092] Therefore, using a programmable device to implement data query and data transmission can reduce the data processing volume of the server and improve the processing efficiency and response speed of the server.

[0093] According to an embodiment of the present invention, performing a query in the storage unit based on the data indexes in the query subtasks to obtain query sub-results of the query subtasks includes: using a data transmission component to obtain stored data in the storage unit from the storage unit based on the data indexes; using a data decryption component to decrypt the stored data to obtain plaintext data; and based on the plaintext data, determining the query sub-results and storing the query sub-results in a cache component.

[0094] The data index can be used to represent the location of the data source including the query subtask. The location includes the logical location and the physical location. According to the location of the data source, the stored data can be obtained from the storage unit, where the stored data includes the query sub-results of the query subtask.

[0095] Since the data on the storage unit is stored by a programmable device, and the programmable device encrypts the data during the data storage process, the obtained stored data is encrypted data. The stored data can be decrypted by using the decryption method corresponding to the encryption process. Among them, a data decryption component can be set in the programmable device to decrypt the stored data to obtain plaintext data.

[0096] After obtaining the plaintext data, the filtering conditions in the data index can be used to determine the query sub-results from the plaintext data and store them in the cache component.

[0097] The cache component can be used to temporarily store the query sub-results, and after all the query subtasks in the query subtask group are executed, the multiple query sub-results are sent to the server at one time.

[0098] According to an embodiment of the present invention, each query subtask group includes S query subtasks, where S represents the number of processing steps required for the programmable device to query the query sub-results in the storage unit, and S is a positive integer; sending the query sub-results of the query subtask to the server includes: in response to the completion of the execution of the S-th query subtask in the query subtask group, reading the query sub-results of the S query subtasks in the query subtask group from the cache component; uploading the first query result obtained by concatenating the query sub-results of the S query subtasks to the server and clearing the cache component; in response to the completion of the execution of the last query subtask in the query request, uploading the second query result obtained by concatenating one or more query sub-results stored in the cache component to the server, so that the server can splice the received one or more first query results and the second query result to obtain the query result.

[0099] For the remaining query subtask groups except the last one, when the S-th query subtask in the query subtask group is executed, the remaining S - 1 query sub-results of the query subtask group are stored in the cache component. Therefore, the first to the S - 1-th query sub-results can be read from the cache component, concatenated with the S-th query sub-result, and the first query result is obtained and uploaded to the server.

[0100] Similarly, for the last query subtask group, in response to the completion of the execution of the last query subtask in the query request, the query sub-results stored in the current cache component are obtained from the cache component, concatenated with the last query sub-result, and the second query result is obtained and uploaded to the server.

[0101] According to an embodiment of the present invention, by providing a data transmission component, a data decryption component, and a cache component on a programmable component, it is possible to implement, in the form of integrating hardware units, the execution of steps in the data query process using independent hardware, thereby realizing the migration of the data query process from the server side and reducing the data processing load on the server side.

[0102] According to an embodiment of the present invention, uploading a first query result obtained by splicing the query sub-results of each of S query sub-tasks to the server includes: using the compression component of the programmable device to compress the first query result to obtain a first compressed result; and uploading the first compressed result to the server; uploading a second query result obtained by splicing one or more query sub-results saved in the cache component to the server includes: using the compression component to compress the second query result to obtain a second compressed result; and uploading the second compressed result to the server.

[0103] The compression of the first query result and the second query result can be achieved by providing a compression component in the programmable device. After obtaining the first query result or the second query result, it is timely compressed to obtain the first compressed result or the second compressed result, and the first compressed result or the second compressed result is uploaded to the server.

[0104] Figure 5 Shows a structural diagram of a programmable device according to an embodiment of the present invention.

[0105] As Figure 5 shown, the programmable device 104 includes a data transmission component 1041, a data encryption component 1042, a data decryption component 1043, a data query component 1044, a cache component 1045, and a compression component 1046.

[0106] According to an embodiment of the present invention, the programmable device 104 may include a Field-Programmable Gate Array (FPGA), an Application Specific Integrated Circuit, a microcontroller, etc.

[0107] The data transmission component 1041 is used for point-to-point communication with the server, the storage unit, and the solid-state drive to complete operations such as data reading and data uploading.

[0108] The data encryption component 1042 is used for encrypting the data to be stored sent by the server and sending it to the storage unit for storage through the data transmission component 1041.

[0109] The data decryption component 1043 is used to decrypt the stored data obtained from the storage unit or the solid-state drive to obtain the plaintext data. The data query component 1044 is used to obtain the query sub-results from the plaintext data based on the data index of the query sub-task.

[0110] The cache component 1045 is used to temporarily store the query sub-results that have been executed when multiple query sub-tasks in the current query sub-task group have not been executed yet.

[0111] The compression component 1046 is used to compress the query results determined by multiple query sub-results.

[0112] Figure 6 The flowchart shows the data transmission process of a query sub-task group according to an embodiment of the present invention.

[0113] As Figure 6 shown, the client sends a data query request to the server. In response to receiving the data query request, the server divides the data query request into multiple query sub-tasks, and sequentially sends four query sub-tasks in a query sub-task group to the data transmission component of the programmable device. After receiving the query sub-task, the data transmission component allocates different components in the programmable device to execute the corresponding steps according to the processing steps of the query sub-task.

[0114] For example, in the case where the first step of the processing step is to obtain data, the data transmission component obtains data from the storage unit based on the query sub-task. In the case where the second step of the processing step is data decryption, the data transmission component transmits the data obtained from the storage unit to the data decryption component, and the data decryption component decrypts it to obtain the plaintext data. In the case where the third step of the processing step is data retrieval, the data query component filters and screens from the plaintext data based on the filtering conditions in the data index. In the case where the fourth step of the processing step is to determine the result, the data query component transmits the query sub-results obtained by filtering and screening to the cache component for temporary storage.

[0115] After all 4 query sub-tasks in the query sub-task group are executed, the first query result is determined according to the query sub-result of the 4th query sub-task and the query results of the 1st to 3rd query sub-tasks stored in the cache component. And the first query result is compressed by the compression component and sent to the server. The server stores the compressed result in the server cache.

[0116] In the case where multiple query sub-task groups are all executed, the server decompresses the multiple compressed results in the server cache, splices the decompressed results to obtain the query result of the data query request. The query result is stored in the server cache and returned to the client.

[0117] Based on the above data transmission method, the present invention further provides a data transmission device applied to a server. The following will be combined with Figure 7 to describe the device in detail.

[0118] Figure 7 Fig. shows a structural block diagram of a data transmission device applied to a server according to an embodiment of the present invention.

[0119] As Figure 7 shown, the data transmission device 700 in this embodiment includes a task division module 710, a relationship determination module 720, a task grouping module 730, a task sending module 740, and a result determination module 750.

[0120] The task division module 710 is configured to divide a data query request into multiple query subtasks in response to a user's data query request. In one embodiment, the task division module 710 may be configured to perform the operation S210 described above, which will not be elaborated here.

[0121] The relationship determination module 720 is configured to determine the corresponding relationship between multiple query subtasks and multiple data partitions based on the data indexes of the multiple query subtasks. In one embodiment, the relationship determination module 720 may be configured to perform the operation S220 described above, which will not be elaborated here.

[0122] The task grouping module 730 is configured to divide one or more query subtasks corresponding to the same data partition into the same query subtask group based on the corresponding relationship. In one embodiment, the task grouping module 730 may be configured to perform the operation S230 described above, which will not be elaborated here.

[0123] The task sending module 740 is configured to sequentially send multiple query subtasks in the query subtask group to a programmable device, so that the programmable device queries in a storage unit based on the data indexes in the query subtasks to obtain query sub-results of the query subtasks. In one embodiment, the task sending module 740 may be configured to perform the operation S240 described above, which will not be elaborated here.

[0124] The result determination module 750 is configured to determine a query result of the data query request based on the query sub-results of the multiple query subtasks from the programmable device. In one embodiment, the result determination module 750 may be configured to perform the operation S250 described above, which will not be elaborated here.

[0125] According to an embodiment of the present invention, the task sending module 740 includes a first task sending sub-module and a second task sending sub-module.

[0126] The first task sending sub-module is used to send the first query sub-task in the query sub-task group to the programmable device for the programmable device to execute, where the query sub-task group includes S query sub-tasks among multiple query sub-tasks.

[0127] The second task sending sub-module is used to send the s-th query sub-task to the programmable device when the programmable device executes to the s-th step, where s = 2,..., S.

[0128] According to an embodiment of the present invention, the result determination module 750 includes a result determination sub-module.

[0129] The result determination sub-module is used to, in response to receiving the second query result and one or more first query results sent by the programmable device, splice the second query result and one or more first query results to obtain a query result, where the first query result is determined by the programmable device based on the respective query sub-results of S query sub-tasks in the query sub-task groups other than the last query sub-task group, and the second query result is determined by the programmable device based on the respective query sub-results of one or more query sub-tasks in the last query sub-task group.

[0130] According to an embodiment of the present invention, the result determination sub-module includes a result decompression unit and a result splicing unit.

[0131] The result decompression unit is used to decompress the second compressed result and one or more first compressed results to obtain the second query result and one or more first query results, where the second compressed result is obtained by the programmable device using a compression component to compress the second query result, and the first compressed result is obtained by the programmable device using a compression component to compress the first query result.

[0132] The result splicing unit is used to splice the second query result and one or more first query results based on the task order of the query sub-tasks corresponding to the one or more first query results to obtain a query result.

[0133] According to an embodiment of the present invention, the data transmission device 700 further includes a result acquisition module and a result sending module.

[0134] The result acquisition module is used to, in response to a user's data query request, when it is determined that the query result corresponding to the data query request is stored in the server cache, obtain the query result from the server cache.

[0135] The result sending module is used to send the query result to the client and save the query result in the server cache.

[0136] According to an embodiment of the present invention, the data transmission device 700 further includes a data storage module.

[0137] A data storage module, configured to, in response to a data storage request, send the data to be stored included in the data storage request to a programmable device, so that the programmable device uses a data encryption component to encrypt the data to be stored to obtain encrypted data, and send the encrypted data to a storage unit for storage.

[0138] According to an embodiment of the present invention, the data transmission device 700 further includes a storage division module, a partition sending module, and a result query module.

[0139] The storage division module is configured to divide the storage unit into multiple data partitions based on the data indexes of multiple query subtasks, so that each data partition includes at least a query sub-result of one query subtask.

[0140] The partition sending module is configured to send the data partition result to the programmable device, so that the programmable device determines a target partition from multiple data partitions based on the data partition result and the data indexes in the query subtasks.

[0141] The result query module is configured to obtain the query sub-result of the query subtask from the target partition.

[0142] According to an embodiment of the present invention, the data transmission device 700 further includes a data cache module and a result acquisition module.

[0143] The data cache module is configured to cache the data with an access frequency higher than an access threshold to a solid-state drive based on the access frequency of the programmable device to the storage unit.

[0144] The result acquisition module is configured to, when it is determined that the query sub-result is stored on the solid-state drive, use the programmable device to obtain the query sub-result from the solid-state drive based on the data index via the programmable device.

[0145] Based on the above data transmission method, the present invention further provides a data transmission device applied to a programmable device. The following will be combined with Figure 8 to describe the device in detail.

[0146] Figure 8 FIG. shows a structural block diagram of a data transmission device applied to a programmable device according to an embodiment of the present invention.

[0147] As Figure 8 shown, the data transmission device 800 in this embodiment includes a data query module 810 and a result sending module 820.

[0148] The data query module 810 is configured to respond to multiple query subtasks sent by the server, and perform queries in the storage unit based on the data indexes in the query subtasks to obtain query sub-results of the query subtasks. Among them, the multiple query subtasks are obtained by the server dividing the data query request in response to the user's data query request, and the multiple query subtasks are sent by the server sequentially in an ordered form. In an embodiment, the data query module 810 may be used to execute the operation S410 described above, which will not be elaborated here.

[0149] The result sending module 820 is configured to send the query sub-results of the query subtasks to the server, so that the server can determine the query result of the data query request based on the query sub-results of each of the multiple query subtasks. In an embodiment, the result sending module 820 may be used to execute the operation S420 described above, which will not be elaborated here.

[0150] According to an embodiment of the present invention, the data query module 810 includes a data acquisition sub-module, a data decryption sub-module, and a data storage sub-module.

[0151] The data acquisition sub-module is configured to use the data transmission component to acquire the stored data in the storage unit from the storage unit based on the data index.

[0152] The data decryption sub-module is configured to use the data decryption component to decrypt the stored data to obtain the plaintext data.

[0153] The data storage sub-module is configured to determine the query sub-result based on the plaintext data and store the query sub-result in the cache component.

[0154] According to an embodiment of the present invention, the result sending module 820 includes a result reading sub-module, a first result uploading sub-module, and a second result uploading sub-module.

[0155] The result reading sub-module is configured to, in response to the completion of the execution of the S-th query subtask in the query subtask group, read the query sub-results of each of the S query subtasks in the query subtask group from the cache component.

[0156] The first result uploading sub-module is configured to upload the first query result obtained by splicing the query sub-results of each of the S query subtasks to the server and clear the cache component.

[0157] The second result uploading sub-module is configured to, in response to the completion of the execution of the last query subtask in the query request, upload the second query result obtained by splicing one or more query sub-results saved in the cache component to the server, so that the server can splice the received one or more first query results and the second query result to obtain the query result.

[0158] According to an embodiment of the present invention, the first result uploading sub-module includes a first compression unit and a first uploading unit.

[0159] The first compression unit is configured to use a compression component of a programmable device to compress a first query result to obtain a first compressed result.

[0160] The first uploading unit is configured to upload the first compressed result to a server.

[0161] According to an embodiment of the present invention, the second result uploading sub-module includes a second compression unit and a second uploading unit.

[0162] The second compression unit is configured to use a compression component to compress a second query result to obtain a second compressed result.

[0163] The second uploading unit is configured to upload the second compressed result to a server.

[0164] According to an embodiment of the present invention, any multiple of the task division module 710, the relationship determination module 720, the task grouping module 730, the task sending module 740, the result determination module 750, the data query module 810, and the result sending module 820 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the task division module 710, the relationship determination module 720, the task grouping module 730, the task sending module 740, the result determination module 750, the data query module 810, and the result sending module 820 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system in a package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the task division module 710, the relationship determination module 720, the task grouping module 730, the task sending module 740, the result determination module 750, the data query module 810, and the result sending module 820 may be at least partially implemented as a computer program module, and when the computer program module runs, it can execute the corresponding functions.

[0165] Figure 9 Shows a block diagram of an electronic device suitable for implementing a data transmission method according to an embodiment of the present invention.

[0166] As Figure 9As shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. The processor 901 can include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 901 can also include on-board memory for caching purposes. The processor 901 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0167] In the RAM 903, various programs and data required for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The processor 901 performs various operations of the method flow according to an embodiment of the present invention by executing the program in the ROM 902 and / or the RAM 903. It should be noted that the program can also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 can also perform various operations of the method flow according to an embodiment of the present invention by executing the program stored in the one or more memories.

[0168] According to an embodiment of the present invention, the electronic device 900 can further include an input / output (I / O) interface 905, and the input / output (I / O) interface 905 is also connected to the bus 904. The electronic device 900 can further include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read from it can be installed into the storage section 908 as needed.

[0169] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium stores one or more programs, and when the one or more programs are executed, the methods according to the embodiments of the present invention are implemented.

[0170] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than the ROM 902 and RAM 903.

[0171] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program codes for executing the method shown in the flowchart. When the computer program product runs on a computer system, the program codes are used to enable the computer system to implement the method provided by the embodiments of the present invention.

[0172] When the computer program is executed by the processor 901, the above functions defined in the system / apparatus of the embodiments of the present invention are executed. According to an embodiment of the present invention, the above-described systems, apparatuses, modules, units, etc. may be implemented by computer program modules.

[0173] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and is downloaded and installed through the communication part 909, and / or installed from the removable medium 911. The program codes included in the computer program may be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0174] In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiments of the present invention are performed. According to an embodiment of the present invention, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0175] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedures and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0177] Those skilled in the art can understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0178] The embodiments of the present invention have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.

Claims

1. A data transmission method, applied to a server, characterized in that: The method comprises: In response to a data query request from a user, dividing the data query request into a plurality of query subtasks; Determining, based on the respective data indexes of the plurality of query subtasks, a correspondence between the plurality of query subtasks and a plurality of data partitions included in the storage unit; Based on the corresponding relationship, one or more query subtasks corresponding to the same data partition are divided into the same query subtask group; Sending the multiple query subtasks in the query subtask group to the programmable device in sequence, so that the programmable device performs a query on the storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask; and The query result of the data query request is determined based on the query sub-results of each of the plurality of query sub-tasks from the programmable device.

2. The method according to claim 1, characterized in that Each query subtask group includes S query subtasks, where S represents the number of processing steps required for the programmable device to query the storage unit to obtain the query subresult, and S is a positive integer; The sending the plurality of query subtasks in the query subtask group to the programmable device in sequence comprises: Sending the first query subtask in the query subtask group to the programmable device for execution by the programmable device, wherein the query subtask group includes S query subtasks in the plurality of query subtasks; When the programmable device executes the s-th step, the s-th query subtask is sent to the programmable device, s=2, ..., S.

3. The method according to claim 2, characterized in that The determining the query result of the data query request based on the query sub-results of each of the plurality of query sub-tasks from the programmable device comprises: In response to receiving a second query result and one or more first query results sent by the programmable device, the second query result and one or more first query results are concatenated to obtain the query result, wherein the first query result is determined by the programmable device based on the query sub-results of each of the S query subtasks in the query subtask group other than the last query subtask group, and the second query result is determined by the programmable device based on the query sub-results of each of the one or more query subtasks in the last query subtask group.

4. The method according to claim 3, characterized in that The step of concatenating the second query result with one or more of the first query results to obtain the query result includes: decompressing the second compression result and one or more first compression results to obtain the second query result and one or more first query results, wherein the second compression result is obtained by the programmable device compressing the second query result using the compression component, and the first compression result is obtained by the programmable device compressing the first query result using the compression component; and Based on the task order of the query subtasks corresponding to the one or more first query results, the second query result and one or more first query results are concatenated to obtain the query result.

5. The method according to claim 1, characterized in that The method further comprises: In response to the user's data query request, if it is determined that the query result corresponding to the data query request is stored in the server cache, obtaining the query result from the server cache; and The query result is sent to the client, and the query result is stored in the server cache.

6. The method according to claim 1, characterized in that The method further comprises: In response to a data storage request, the data to be stored included in the data storage request is sent to the programmable device, so that the programmable device uses a data encryption component to encrypt the data to be stored, obtains encrypted data, and sends the encrypted data to the storage unit for storage.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Based on the data indexes of the plurality of query subtasks, the storage unit is divided into a plurality of data partitions, so that each data partition includes at least one query subresult of the query subtask; Sending the data partitioning result to the programmable device so that the programmable device determines a target partition from the multiple data partitions based on the data partitioning result and the data index in the query subtask; A query subresult of the query subtask is obtained from the target partition.

8. The method according to claim 7, characterized in that The method further comprises: Based on the access frequency of the programmable device to the storage unit, cache the data with an access frequency higher than an access threshold in the solid state drive; When it is determined that the query sub-result is stored on the solid state drive, the query sub-result is obtained from the solid state drive via the programmable device based on the data index using the programmable device.

9. A data transmission method, applied to a programmable device, characterized in that: The method comprises: In response to multiple query subtasks sent by the server, query the storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask, wherein the multiple query subtasks are obtained by the server in response to the user's data query request by dividing the data query request, and the multiple query subtasks are sent by the server in sequence; and Sending the query sub-results of the query sub-tasks to the server, so that the server determines the query result of the data query request based on the query sub-results of each of the plurality of query sub-tasks; The server sends the multiple query subtasks to the programmable device in sequence in the following manner: Determining a correspondence between the plurality of query subtasks and the plurality of data partitions based on respective data indexes of the plurality of query subtasks; Based on the corresponding relationship, one or more query subtasks corresponding to the same data partition are divided into the same query subtask group; The multiple query subtasks in the query subtask group are sent to the programmable device in sequence.

10. The method according to claim 9, characterized in that The querying in the storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask includes: Using a data transmission component, based on a data index, acquiring the storage data in the storage unit from the storage unit; Decrypting the stored data using a data decryption component to obtain plaintext data; and Based on the plaintext data, the query sub-result is determined and the query sub-result is stored in a cache component.

11. The method according to claim 10, characterized in that Each query subtask group includes S query subtasks, where S represents the number of processing steps required for the programmable device to query the storage unit to obtain the query subresult, and S is a positive integer; The sending the query sub-result of the query sub-task to the server includes: In response to the Sth query subtask in the query subtask group being executed, reading respective query subresults of the S query subtasks in the query subtask group from the cache component; Upload the first query result obtained by concatenating the query sub-results of the S query sub-tasks to the server, and clear the cache component; In response to the completion of the execution of the last query subtask in the query request, a second query result obtained by splicing one or more query subresults stored in the cache component is uploaded to the server, so that the server can splice the query result based on the received one or more first query results and second query results.

12. The method according to claim 11, characterized in that The uploading to the server a first query result obtained by splicing the query sub-results of the S query sub-tasks, comprises: Using a compression component of the programmable device, compressing the first query result to obtain a first compressed result; and Uploading the first compression result to the server; The step of uploading the second query result obtained by splicing one or more query sub-results stored in the cache component to the server includes: Using the compression component, compressing the second query result to obtain a second compressed result; and The second compression result is uploaded to the server.

13. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 12.

14. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images