Data transmission method, device, medium and program product

Abstract

The invention provides a data transmission method and device, a medium and a program product, which can be applied to the technical fields of data security and computers. The data transmission method comprises the following steps: in response to a data query request of a user, dividing the data query request into a plurality of query subtasks; determining a corresponding relationship between the plurality of query subtasks and the plurality of data partitions based on the respective data indexes of the plurality of query subtasks; on the basis of the corresponding relation, one or more query subtasks corresponding to the same data partition are divided into the same query subtask group; a plurality of query subtasks in the query subtask group are sequentially sent to the programmable device according to the sequence, so that the programmable device performs query in a storage unit based on data indexes in the query subtasks, and query subresults of the query subtasks are obtained; and determining a query result of the data query request based on respective query sub-results of the plurality of query sub-tasks from the programmable device.

Description

Technical Field

[0001] The present invention relates to the fields of data security and computer technology, and in particular to a data transmission method, device, medium and program product. Background Art

[0002] With the advent of the big data era, accurate and secure data query from databases storing massive amounts of data has become a major challenge. In order to improve the efficiency and speed of querying massive data, related technologies usually use software methods to encrypt and decrypt data and accelerate software queries.

[0003] However, in traditional software encryption and decryption and software accelerated database query methods, both software encryption and decryption and software accelerated database query belong to the category of software acceleration technology, and ultimately require a central processing unit (CPU) to execute, which will consume a large amount of CPU computing resources in the server, affecting other software programs in the system and affecting the stable operation of the server. Summary of the invention

[0004] In view of the above problems, the present invention provides a data transmission method, device, medium and program product.

[0005] According to a first aspect of the present invention, a data transmission method is provided, which is applied to a server, and includes: in response to a user's data query request, dividing the data query request into multiple query subtasks; determining the correspondence between the multiple query subtasks and the multiple data partitions included in a storage unit based on the data indexes of the multiple query subtasks; based on the correspondence, dividing one or more query subtasks corresponding to the same data partition into the same query subtask group; sending the multiple query subtasks in the query subtask group to a programmable device in sequence, so that the programmable device queries the storage unit based on the data indexes in the query subtasks to obtain query subresults of the query subtasks, wherein the storage unit is divided into multiple data partitions; and determining the query result of the data query request based on the query subresults of the multiple query subtasks from the programmable device.

[0006] The second aspect of the present invention provides a data transmission method, which is applied to a programmable device, including: in response to multiple query subtasks sent by a server, based on the data index in the query subtask, querying in a storage unit to obtain query subresults of the query subtask, wherein the multiple query subtasks are obtained by the server in response to the user's data query request by dividing the data query request, and the multiple query subtasks are sent by the server in sequence; and sending the query subresults of the query subtask to the server, so that the server determines the query result of the data query request based on the query subresults of each of the multiple query subtasks; wherein the server sends the multiple query subtasks in sequence to the programmable device in the following manner: based on the data indexes of each of the multiple query subtasks, determining the correspondence between the multiple query subtasks and the multiple data partitions; based on the correspondence, dividing one or more query subtasks corresponding to the same data partition into the same query subtask group; and sending the multiple query subtasks in the query subtask group to the programmable device in sequence.

[0007] The third aspect of the present invention provides a data transmission device applied to a server, including: a task division module, used to respond to a user's data query request and divide the data query request into multiple query subtasks; a relationship determination module, used to determine the correspondence between multiple query subtasks and multiple data partitions based on the data indexes of each of the multiple query subtasks; a task grouping module, used to divide one or more query subtasks corresponding to the same data partition into the same query subtask group based on the correspondence; a task sending module, used to send multiple query subtasks in the query subtask group to a programmable device in sequence, so that the programmable device can query in a storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask; a result determination module, used to determine the query result of the data query request based on the query subresults of each of the multiple query subtasks from the programmable device.

[0008] The fourth aspect of the present invention provides a data transmission device applied to a programmable device, including: a data query module, which is used to respond to multiple query subtasks sent by a server, and query in a storage unit based on the data index in the query subtask to obtain query subresults of the query subtask, wherein the multiple query subtasks are obtained by the server in response to the user's data query request by dividing the data query request, and the multiple query subtasks are sent by the server in sequence; a result sending module, which is used to send the query subresults of the query subtask to the server, so that the server can determine the query result of the data query request based on the query subresults of each of the multiple query subtasks.

[0009] A fifth aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0010] The sixth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0011] The seventh aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0012] According to an embodiment of the present invention, a data query request is divided into multiple query subtasks by a server, and the query subtasks are sent to a programmable device in sequence, and a pipeline query is performed by a programmable device, and the step with the largest delay in the data query process is migrated to an external device, thereby reducing the data processing volume and complexity of the server, and improving the efficiency of data query and data transmission. In addition, in the process of implementing data query using a programmable device, the programmable device only communicates point-to-point with a storage unit and a server, thereby avoiding the problem of data leakage and improving data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0014] Figure 1 An application scenario diagram of a data transmission method, device, medium, and program product according to an embodiment of the present invention is shown.

[0015] Figure 2 A flow chart of a data transmission method applied to a server according to an embodiment of the present invention is shown.

[0016] Figure 3 A schematic diagram showing a task sending sequence and a task execution sequence of a query subtask group according to an embodiment of the present invention is shown.

[0017] Figure 4 A flow chart of a data transmission method applied to a programmable device according to an embodiment of the present invention is shown.

[0018] Figure 5 A structural diagram of a programmable device according to an embodiment of the present invention is shown.

[0019] Figure 6 A flow chart of a data transmission process of a query subtask group according to an embodiment of the present invention is shown.

[0020] Figure 7 A structural block diagram of a data transmission device applied to a server according to an embodiment of the present invention is shown.

[0021] Figure 8 A structural block diagram of a data transmission device applied to a programmable device according to an embodiment of the present invention is shown.

[0022] Fig. 9 A block diagram of an electronic device suitable for implementing a data transmission method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0023] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0024] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0025] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0026] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0027] In addition, since the query software has the problem of online upgrade in the process of accelerating query through software technology, the query software is difficult to use offline, and there is a security risk that the data query process may be obtained by a third party.

[0028] An embodiment of the present invention provides a data transmission method, which is applied to a server, and includes: in response to a user's data query request, dividing the data query request into multiple query subtasks; sending the multiple query subtasks to a programmable device in sequence, so that the programmable device queries a storage unit based on a data index in the query subtask to obtain a query subresult of the query subtask; and determining a query result of the data query request based on the query subresults of each of the multiple query subtasks from the programmable device.

[0029] Figure 1 An application scenario diagram of a data transmission method, device, medium, and program product according to an embodiment of the present invention is shown.

[0030] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a client 101 , a server 102 , a network 103 , a programmable device 104 and a storage unit 105 .

[0031] The user logs in to the client 101 and communicates with the server 102 through the network 103 , and sends a data query request to the server 102 .

[0032] The server 102 is connected to the programmable device 104 , transmits the user's data query request to the programmable device 104 in a point-to-point manner and obtains the query result from the programmable device 104 , and the server 102 sends the query result to the client 101 through the network 103 .

[0033] The programmable device 104 is connected to the storage unit 105 , and upon receiving a data query request sent by the server 102 , obtains data from the storage unit 105 , obtains a query result of the data query request therefrom, and sends the query result to the server 102 .

[0034] It should be understood that Figure 1 The number of clients, servers, networks, programmable devices, and storage units in the embodiment is only for illustration. Any number of clients, servers, networks, programmable devices, and storage units may be provided according to implementation requirements.

[0035] The following will be based on Figure 1 The scene described by Figure 2~Figure 6 The data transmission method according to the embodiment of the present invention is described in detail.

[0036] Figure 2 A flow chart of a data transmission method applied to a server according to an embodiment of the present invention is shown.

[0037] like Figure 2 As shown, the data transmission method of this embodiment includes operations S210 to S250.

[0038] In operation S210 , in response to a data query request from a user, the data query request is divided into a plurality of query subtasks.

[0039] The data query request submitted by the user may include restrictions such as data source and filtering conditions. The server can construct the corresponding structured query language (SQL) based on the data source and filtering conditions selected by the user, and divide the data query request into multiple query subtasks based on the SQL statement.

[0040] In one embodiment, the SQL statement may be parsed to determine an Abstract Syntax Tree (AST) corresponding to the SQL statement, and the syntax and semantics may be determined based on the AST to determine a division method for the data query request.

[0041] For example, when it is determined according to AST that the SQL statement representation needs to query data from multiple different partitions of the data table, the partitioning can be performed so that each query subtask only needs to query data from a single partition.

[0042] In operation S220, a correspondence relationship between the plurality of query subtasks and the plurality of data partitions included in the storage unit is determined based on the respective data indexes of the plurality of query subtasks.

[0043] The correspondence between multiple query subtasks and multiple data partitions can be determined based on the data indexes of each of the multiple query subtasks. When it is determined that data needs to be read from a certain data partition based on the data index, and the query subresult of the current query subtask is determined by searching the data, it can be determined that there is a correspondence between the current query subresult and the task partition.

[0044] In operation S230 , based on the corresponding relationship, one or more query subtasks corresponding to the same data partition are divided into the same query subtask group.

[0045] Based on the corresponding relationship, one or more query subtasks corresponding to each data partition may be determined, and one or more query subtasks corresponding to the same data partition may be divided into the same query subtask group.

[0046] In operation S240, multiple query subtasks in the query subtask group are sent to the programmable device in sequence, so that the programmable device performs a query in the storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask.

[0047] The data query request is divided into multiple query subtasks, and the data index of each query subtask may include restrictions such as the data source and filtering conditions of the query subtask.

[0048] The processing process of each query subtask may include multiple processing steps, and the programmable device may include multiple processing components, and the processing components correspond to the processing steps one by one, so that after the programmable device receives query subtasks sent in sequence, multiple processing components can process multiple query subtasks in the form of a pipeline, so that multiple processing components can work continuously without waiting for the previous query subtask to be executed.

[0049] The data source of the query subtask is determined based on the data index, and the query subresult is obtained after the multiple processing steps of the query subtask are executed in sequence using a programmable device.

[0050] In operation S250 , a query result of the data query request is determined based on the query sub-results of each of the plurality of query sub-tasks from the programmable device.

[0051] Summarize and splice multiple query sub-results from the programmable device to obtain the query result of the query task request, and feed back the query result to the user. According to an embodiment of the present invention, a data query request is divided into multiple query sub-tasks by a server, and the query sub-tasks are sent to the programmable device in sequence, and a pipeline query is performed by a programmable device, and the step with the largest delay in the data query process is migrated to an external device, thereby reducing the data processing volume and complexity of the server, and improving the efficiency of data query and data transmission. In addition, in the process of implementing data query using a programmable device, the programmable device only communicates point-to-point with the storage unit and the server, so that the problem of data leakage can be avoided and data security can be improved.

[0052] After dividing the data query request into multiple query subtasks, the multiple query subtasks can be directly sent to the programmable device in sequence, so that the programmable device can execute the multiple query subtasks in sequence and obtain the query subresults of the multiple query subtasks. However, in the case of a large data query request, the number of query subtasks obtained by division is huge. If the multiple query subtasks are directly sent to the programmable device in sequence, there will be too many query subresults, which are difficult to manage and count. Therefore, by dividing the multiple query subtasks into multiple query subtask groups, data query and data transmission can be performed with the query subtask group as the processing granularity.

[0053] Each query subtask group includes S query subtasks, S represents the number of processing steps required for the programmable device to query the storage unit to obtain the query subresult, and S is a positive integer; multiple query subtasks arranged in a sequential form are sent to the programmable device in sequence, including: sending the first query subtask in the query subtask group to the programmable device for execution by the programmable device, wherein the query subtask group includes S query subtasks in the multiple query subtasks; when the programmable device executes to the s-th step, sending the s-th query subtask to the programmable device, s=2,...,S.

[0054] Figure 3 A schematic diagram showing a task sending sequence and a task execution sequence of a query subtask group according to an embodiment of the present invention is shown.

[0055] like Figure 3 As shown, taking S=4 as an example, each query subtask is processed in four steps in the programmable device to obtain a query subresult. Except for the case where the number of remaining query subresults in the last query subtask group is less than 4 after multiple query subtask groups are divided, the remaining query subtask groups all include four query subtasks.

[0056] Among them, when the number of query subtasks corresponding to a data partition is greater than S, the multiple query subtasks corresponding to the data partition can be divided into multiple query subtask groups. Except for the case where the number of remaining query subresults in the last query subtask group is less than S after the multiple query subtask groups are divided, the remaining query subtask groups all include four query subtasks.

[0057] Since each processing component in a programmable device cannot process multiple query subtasks simultaneously, sending query subtasks in a pipeline form enables each processing component to start processing the corresponding processing steps of the next query subtask after processing the current query subtask.

[0058] When dividing the processing steps, the processing time of each processing step can be made the same as much as possible through pre-planning. Therefore, it is only necessary to control that the next query subtask is sent to the programmable device for processing after the first processing step of the current query subtask is completed, so as to ensure the smooth flow of the pipeline.

[0059] Therefore, it can be determined that the server sends a query subtask to the programmable device every time the programmable device executes a step, that is, when the programmable device executes the sth step, the sth query subtask is sent to the programmable device.

[0060] In this embodiment, multiple query subtasks are sent in a sequential form, and the programmable device can complete the data query and processing of 4 query subtasks in the query subtask group after 7 time cycles. If the 4 query subtasks are executed in sequence in a serial manner, 16 time cycles are required. Therefore, by processing multiple query subtasks in a pipeline form, the processing efficiency of the query subtask group can be significantly improved. By controlling the sending time of the query subtask, it is possible to ensure that each processing component in the programmable device operates normally. For the query subtask, it is also possible to ensure that the multiple processing steps in the query subtask are executed in sequence and continuously, thereby ensuring the accuracy of data transmission and data query.

[0061] Since multiple query subtasks are obtained by dividing the data query request, the query sub-results of each query subtask also need to be combined to obtain the query result of the data query request. Since multiple query subtasks are sent to the programmable device with the query subtask group as the granularity, after obtaining the query sub-results of each of the multiple query subtasks in each query subtask group, the multiple query sub-results can be combined to obtain the query result of the query subtask group, and returned to the server with the query result of the query subtask group as the processing granularity.

[0062] Therefore, the server needs to determine the query result of the data query request based on the query sub-results of each of the multiple query sub-tasks from the programmable device.

[0063] In response to the server receiving the second query result and one or more first query results sent by the programmable device, the second query result and the one or more first query results are concatenated to obtain a query result, wherein the first query result is determined by the programmable device based on the query sub-results of each of the S query subtasks in the query subtask group other than the last query subtask group, and the second query result is determined by the programmable device based on the query sub-results of each of the one or more query subtasks in the last query subtask group.

[0064] The query subtasks obtained by dividing the data query request are combined to obtain multiple query subtask groups. Among the multiple query subtask groups, except for the last query subtask group, the other query subtask groups include S query subtasks. Depending on the total number of query subtasks, the last query subtask group may include S or less than S query subtasks.

[0065] Since the multiple query subtasks in each query subtask group are executed sequentially, for each query subtask group, after the Sth query subtask is executed, it can be determined that the multiple query subtasks in the query subtask group have been executed, and the first query result can be determined based on the query subresults of the multiple query subtasks.

[0066] For the last query subtask group, since the last query subtask in the last query subtask group is the last of multiple query subtasks in the data query request, that is, after one or more query subtasks in the last query subtask group are executed, there are no unexecuted query subtasks, so the query subresults of each of the one or more query subtasks obtained by the programmable device query at this time can be determined as the second query result.

[0067] Each time the programmable device determines a first query result or a second query result, it sends it to the server. After receiving the second query result, the server can determine that the data of the programmable device has been sent, so the received second query result can be spliced ​​and combined with one or more first query results to obtain the query result.

[0068] When the amount of data required for a data query request is large, there may be too many query subtask groups, or the first query result or the second query result obtained by each query subtask group is too large. Directly transmitting the first query result or the second query result will reduce the data query efficiency due to a large amount of transmission delay.

[0069] Therefore, before sending the first query result and the second query result, the programmable device can compress the first query result and the second query result respectively to obtain the first compressed result and the second compressed result, and send the first compressed result and the second compressed result to the server to reduce the size of the transmitted data through data compression and improve the data transmission efficiency.

[0070] After the server receives the first compression result and the second compression result, it can use a data decompression method that matches the data compression method used by the programmable device when compressing data to decompress the second compression result and one or more first compression results to obtain a second query result and one or more first query results, wherein the second compression result is obtained by the programmable device compressing the second query result using a compression component, and the first compression result is obtained by the programmable device compressing the first query result using a compression component; and based on the task order of the query subtasks corresponding to the one or more first query results, the second query result and one or more first query results are spliced ​​to obtain the query result.

[0071] After the data is decompressed and the first query result and the second query result are restored, the multiple query results can be spliced ​​or combined according to the task order of the query subtasks corresponding to the second query result and each first query result to obtain the query result.

[0072] According to the embodiment of the present invention, the compressed data is decompressed and concatenated to obtain the query result, which can ensure the accuracy of the data while reducing the channel load during the transmission process.

[0073] Through the above method, the response to the data query request can be completed quickly, and the query result of the data query request can be obtained. However, for high-frequency data query requests, even if the response speed is fast, repeated data transmission will also waste resources. Therefore, after the server obtains the query result, the query result can be temporarily stored in the server cache, and the cache elimination strategy, such as First In First Out (FIFO), Least Recently Used (LRU), and Least Frequently Used (LFU), is used to regularly update the server cache, so as to respond to the user's data query request, and obtain the query result from the server cache when it is determined that the query result corresponding to the data query request is stored in the server cache; and the query result is sent to the client, and the query result is saved in the server cache.

[0074] According to an embodiment of the present invention, part of the query results are stored in the server cache, so that when the query request hits the query result, the query result can be directly obtained from the server cache, further improving the efficiency of data transmission.

[0075] By setting up a programmable device, some steps of data query can be transferred to hardware outside the server, thereby reducing the processing load of the server. Since the processing procedures of obtaining data from and storing data in the storage unit are opposite, when the programmable device is capable of data query, the programmable device can also be used to store data in the storage unit to further reduce the server load.

[0076] During the data storage process, in response to a data storage request, the data to be stored included in the data storage request is sent to the programmable device, so that the programmable device uses a data encryption component to encrypt the data to be stored, obtains encrypted data, and sends the encrypted data to the storage unit for storage.

[0077] The data storage request includes the data to be stored and the target address of storage. The data to be stored and the storage address are sent to the programmable device, which handles the data encryption, data transmission and other processes required in the data storage process, thereby storing the encrypted data to be stored in the target address of the storage unit, further reducing the processing flow of the server and improving the hardware utilization of the programmable device.

[0078] According to an embodiment of the present invention, the data transmission method also includes: dividing the storage unit into multiple data partitions based on the data indexes of each of the multiple query subtasks, so that each data partition includes at least a query subresult of a query subtask; sending the data partition results to the programmable device so that the programmable device determines a target partition from the multiple data partitions based on the data partition results and the data indexes in the query subtask; and obtaining the query subresults of the query subtask from the target partition.

[0079] According to an embodiment of the present invention, before executing multiple query subtasks, the storage unit can be divided into multiple data partitions according to the data indexes of the multiple query subtasks, and one or more query subtasks included in each data partition can be determined. When the query subtasks are divided into query subtask groups, the query subtasks on the same data partition can be preferentially divided into the same query subtask group, so that when the programmable device processes the query subtask group, it only needs to obtain the storage content of the data partition from the storage unit to obtain the query subresults of the multiple query subtasks, so that for a query subtask group, the number of data partitions to be queried can be reduced, thereby improving the query efficiency.

[0080] A large amount of data can be stored in a storage unit. Even if data partitioning is performed, some data partitions may be too large. In this case, in order to execute the query subtask, it is necessary to obtain the content of the data partition from the storage unit. If the content of the data partition is queried frequently, the query efficiency will be reduced.

[0081] Therefore, the above problem can be solved by means of caching. Based on the access frequency of the programmable device to the storage unit, the data with an access frequency higher than the access threshold is cached in the solid-state hard disk; when it is determined that the query sub-results are stored on the solid-state hard disk, the programmable device is used to obtain the query sub-results from the solid-state hard disk via the programmable device based on the data index.

[0082] The solid-state hard disk can be set between the storage unit and the programmable device. After receiving the query subtask, the programmable device can determine whether the access frequency of the data source of the query subtask is higher than the access threshold. If it is determined to be higher than the access threshold, the query subresult can be directly obtained from the solid-state hard disk.

[0083] According to an embodiment of the present invention, by setting a solid state drive between the programmable device and the storage unit, the amount of data processing in the intermediate process of data transmission can be further reduced by adding external hardware, thereby improving the efficiency of data query and data transmission.

[0084] Figure 4 A flow chart of a data transmission method applied to a programmable device according to an embodiment of the present invention is shown.

[0085] like Figure 4 As shown, the data transmission method of this embodiment includes operations S410 to S420.

[0086] In operation S410, in response to a plurality of query subtasks sent by the server, a query is performed in a storage unit based on a data index in the query subtask to obtain a query subresult of the query subtask.

[0087] The multiple query subtasks are obtained by dividing the data query request in response to the user's data query request by the server, and the multiple query subtasks are sent sequentially by the server.

[0088] The programmable device executes the multiple query subtasks in sequence based on the order in which the query subtasks are received, and obtains the query subresults of the multiple query subtasks respectively.

[0089] In operation S420, the query sub-results of the query sub-tasks are sent to the server, so that the server determines the query result of the data query request based on the query sub-results of the plurality of query sub-tasks.

[0090] The server can send multiple query subtasks in sequence to the programmable device in the following manner: based on the data indexes of the multiple query subtasks, determine the correspondence between the multiple query subtasks and the multiple data partitions; based on the correspondence, divide one or more query subtasks corresponding to the same data partition into the same query subtask group; and send the multiple query subtasks in the query subtask group to the programmable device in sequence.

[0091] The programmable device can use the query subtask group as the processing granularity. After obtaining the query subresults of multiple query subtasks in the query subtask group, the multiple query subresults are spliced ​​and sent to the server, so as to reduce the number of data transmissions between the programmable device and the server and improve the utilization rate of the channel for data transmission efficiency.

[0092] Therefore, using programmable devices to implement data query and data transmission can reduce the amount of data processing on the server side and improve the processing efficiency and response speed of the server side.

[0093] According to an embodiment of the present invention, based on the data index in the query subtask, a query is performed in the storage unit to obtain a query subresult of the query subtask, including: using a data transmission component to obtain the storage data in the storage unit from the storage unit based on the data index; using a data decryption component to decrypt the storage data to obtain plaintext data; and based on the plaintext data, determining the query subresult and storing the query subresult in the cache component.

[0094] The data index can be used to indicate the location of the data source including the query subtask, where the location includes a logical location and a physical location. Storage data can be obtained from the storage unit according to the location of the data source, where the storage data includes the query subresults of the query subtask.

[0095] Since the data on the storage unit is stored by a programmable device, the programmable device encrypts the data during the data storage process, so the acquired storage data is encrypted data, and the storage data can be decrypted using a decryption method corresponding to the encryption process. In particular, a data decryption component can be set in the programmable device to decrypt the storage data to obtain plaintext data.

[0096] After obtaining the plaintext data, the filter conditions in the data index can be used to determine the query sub-results from the plaintext data and store them in the cache component.

[0097] The cache component can be used to temporarily store query sub-results, and after all query sub-tasks in the query sub-task group are executed, multiple query sub-results are sent to the server at one time.

[0098] According to an embodiment of the present invention, each query subtask group includes S query subtasks, S represents the number of processing steps required for the programmable device to query in the storage unit to obtain the query subresult, and S is a positive integer; the query subresults of the query subtasks are sent to the server, including: in response to the completion of the execution of the Sth query subtask in the query subtask group, reading the query subresults of each of the S query subtasks in the query subtask group from the cache component; uploading a first query result obtained by splicing the query subresults of each of the S query subtasks to the server, and clearing the cache component; in response to the completion of the execution of the last query subtask in the query request, uploading a second query result obtained by splicing one or more query subresults stored in the cache component to the server, so that the server can splice the query result based on the received one or more first query results and second query results.

[0099] For the remaining query subtask groups except the last query subtask group, when the Sth query subtask in the query subtask group is executed, the cache component stores the remaining S-1 query subresults in the query subtask group. Therefore, the 1st to S-1th query subresults can be read from the cache component and spliced ​​with the Sth query subresult to obtain the first query result and upload it to the server.

[0100] Similarly, for the last query subtask group, in response to the completion of the execution of the last query subtask in the query request, the query subresult stored in the current cache component is obtained from the cache component, and is spliced ​​with the last query subresult to obtain the second query result and upload it to the server.

[0101] According to an embodiment of the present invention, by setting a data transmission component, a data decryption component and a cache component on a programmable component, it is possible to implement the steps in the data query process using independent hardware in the form of an integrated hardware unit, thereby enabling the data query process to be migrated from the server side and reducing the data processing load on the server side.

[0102] According to an embodiment of the present invention, a first query result obtained by concatenating the query sub-results of each of S query sub-tasks is uploaded to a server, including: using a compression component of a programmable device to compress the first query result to obtain a first compressed result; and uploading the first compressed result to the server; a second query result obtained by concatenating one or more query sub-results stored in a cache component is uploaded to the server, including: using the compression component to compress the second query result to obtain a second compressed result; and uploading the second compressed result to the server.

[0103] The compression of the first query result and the second query result can be achieved by setting a compression component in a programmable device. After obtaining the first query result or the second query result, it is compressed in time to obtain the first compressed result or the second compressed result, and the first compressed result or the second compressed result is uploaded to the server.

[0104] Figure 5 A structural diagram of a programmable device according to an embodiment of the present invention is shown.

[0105] like Figure 5 As shown, the programmable device 104 includes a data transmission component 1041 , a data encryption component 1042 , a data decryption component 1043 , a data query component 1044 , a cache component 1045 and a compression component 1046 .

[0106] According to an embodiment of the present invention, the programmable device 104 may include a Field-Programmable Gate Array (FPGA), an application-specific integrated circuit, a microcontroller, and the like.

[0107] The data transmission component 1041 is used to perform point-to-point communication with the server, storage unit and solid-state hard disk to complete operations such as data reading and data uploading.

[0108] The data encryption component 1042 is used to encrypt the data to be stored sent by the server, and send it to the storage unit for storage through the data transmission component 1041.

[0109] The data decryption component 1043 is used to decrypt the storage data obtained from the storage unit or the solid state drive to obtain plain text data. The data query component 1044 is used to obtain query sub-results from the plain text data based on the data index of the query sub-task.

[0110] The cache component 1045 is used to temporarily store the query sub-results that have been executed when multiple query sub-tasks in the current query sub-task group have not been executed yet.

[0111] The compression component 1046 is used to compress the query result determined by the multiple query sub-results.

[0112] Figure 6 A flow chart of a data transmission process of a query subtask group according to an embodiment of the present invention is shown.

[0113] like Figure 6 As shown, the client sends a data query request to the server. In response to receiving the data query request, the server divides the data query request into multiple query subtasks, and sends the four query subtasks in a query subtask group to the data transmission component of the programmable device in sequence. After the data transmission component receives the query subtask, it allocates different components in the programmable device to execute the corresponding steps according to the processing steps of the query subtask.

[0114] For example, when the first step of the processing step is to obtain data, the data transmission component obtains data from the storage unit based on the query subtask. When the second step of the processing step is data decryption, the data transmission component transmits the data obtained from the storage unit to the data decryption component, which decrypts the data to obtain the plaintext data. When the third step of the processing step is data retrieval, the data query component filters the plaintext data based on the filter conditions in the data index. When the fourth step of the processing step is to determine the result, the data query component transmits the query sub-results obtained by filtering to the cache component for temporary storage.

[0115] After all four query subtasks in the query subtask group are executed, the first query result is determined according to the query subresult of the fourth query subtask and the query results of the first to third query subtasks stored in the cache component. The first query result is compressed by the compression component and sent to the server. The server stores the compressed result in the server cache.

[0116] When multiple query subtask groups are executed, the server decompresses the multiple compressed results in the server cache, concatenates the decompressed results to obtain the query result of the data query request, stores the query result in the server cache, and returns it to the client.

[0117] Based on the above data transmission method, the present invention also provides a data transmission device applied to the server. Figure 7 The device is described in detail.

[0118] Figure 7 A structural block diagram of a data transmission device applied to a server according to an embodiment of the present invention is shown.

[0119] like Figure 7 As shown, the data transmission device 700 of this embodiment includes a task division module 710 , a relationship determination module 720 , a task grouping module 730 , a task sending module 740 and a result determination module 750 .

[0120] The task division module 710 is used to respond to the user's data query request and divide the data query request into multiple query subtasks. In one embodiment, the task division module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0121] The relationship determination module 720 is used to determine the corresponding relationship between the multiple query subtasks and the multiple data partitions based on the data indexes of the multiple query subtasks. In one embodiment, the relationship determination module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0122] The task grouping module 730 is used to group one or more query subtasks corresponding to the same data partition into the same query subtask group based on the corresponding relationship. In one embodiment, the task grouping module 730 can be used to perform the operation S230 described above, which will not be described in detail here.

[0123] The task sending module 740 is used to send the multiple query subtasks in the query subtask group to the programmable device in sequence, so that the programmable device performs a query in the storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask. In one embodiment, the task sending module 740 can be used to perform the operation S240 described above, which will not be repeated here.

[0124] The result determination module 750 is used to determine the query result of the data query request based on the query sub-results of each of the multiple query sub-tasks from the programmable device. In one embodiment, the result determination module 750 can be used to perform the operation S250 described above, which will not be repeated here.

[0125] According to an embodiment of the present invention, the task sending module 740 includes a first task sending submodule and a second task sending submodule.

[0126] The first task sending submodule is used to send the first query subtask in the query subtask group to the programmable device so that the programmable device executes it, wherein the query subtask group includes S query subtasks in the plurality of query subtasks.

[0127] The second task sending submodule is used to send the s-th query subtask to the programmable device when the programmable device executes the s-th step, s=2,...,S.

[0128] According to an embodiment of the present invention, the result determination module 750 includes a result determination submodule.

[0129] A result determination submodule is used for, in response to receiving a second query result and one or more first query results sent by the programmable device, concatenating the second query result and the one or more first query results to obtain a query result, wherein the first query result is determined by the programmable device based on the query subresults of each of S query subtasks in a query subtask group other than the last query subtask group, and the second query result is determined by the programmable device based on the query subresults of each of one or more query subtasks in the last query subtask group.

[0130] According to an embodiment of the present invention, the result determination submodule includes a result decompression unit and a result splicing unit.

[0131] A result decompression unit is used to decompress the second compression result and one or more first compression results to obtain a second query result and one or more first query results, wherein the second compression result is obtained by compressing the second query result by the programmable device using the compression component, and the first compression result is obtained by compressing the first query result by the programmable device using the compression component.

[0132] The result concatenation unit is used to concatenate the second query result and one or more first query results based on the task order of the query subtasks corresponding to the one or more first query results to obtain the query result.

[0133] According to the embodiment of the present invention, the data transmission device 700 further includes a result acquisition module and a result sending module.

[0134] The result acquisition module is used to respond to the user's data query request and acquire the query result from the server cache when it is determined that the query result corresponding to the data query request is stored in the server cache.

[0135] The result sending module is used to send the query results to the client and save the query results in the server cache.

[0136] According to an embodiment of the present invention, the data transmission device 700 further includes a data storage module.

[0137] The data storage module is used to respond to a data storage request and send the data to be stored included in the data storage request to the programmable device, so that the programmable device uses the data encryption component to encrypt the data to be stored, obtain the encrypted data, and send the encrypted data to the storage unit for storage.

[0138] According to an embodiment of the present invention, the data transmission device 700 further includes a storage partitioning module, a partition sending module and a result query module.

[0139] The storage partitioning module is used to divide the storage unit into multiple data partitions based on the data indexes of the multiple query subtasks, so that each data partition includes at least a query subresult of a query subtask.

[0140] The partition sending module is used to send the data partitioning result to the programmable device so that the programmable device can determine the target partition from multiple data partitions based on the data partitioning result and the data index in the query subtask.

[0141] The result query module is used to obtain the query sub-results of the query sub-task from the target partition.

[0142] According to an embodiment of the present invention, the data transmission device 700 further includes a data cache module and a result acquisition module.

[0143] The data cache module is used to cache data with an access frequency higher than an access threshold into a solid-state hard disk based on the access frequency of the programmable device to the storage unit.

[0144] The result acquisition module is used to obtain the query sub-result from the solid state drive via the programmable device based on the data index by using the programmable device when it is determined that the query sub-result is stored on the solid state drive.

[0145] Based on the above data transmission method, the present invention also provides a data transmission device applied to a programmable device. Figure 8 The device is described in detail.

[0146] Figure 8 A structural block diagram of a data transmission device applied to a programmable device according to an embodiment of the present invention is shown.

[0147] like Figure 8 As shown, the data transmission device 800 of this embodiment includes a data query module 810 and a result sending module 820 .

[0148] The data query module 810 is used to respond to multiple query subtasks sent by the server, query in the storage unit based on the data index in the query subtask, and obtain the query subresult of the query subtask, wherein the multiple query subtasks are obtained by the server in response to the user's data query request by dividing the data query request, and the multiple query subtasks are sent by the server in sequence. In one embodiment, the data query module 810 can be used to perform the operation S410 described above, which will not be repeated here.

[0149] The result sending module 820 is used to send the query sub-results of the query sub-task to the server, so that the server can determine the query result of the data query request based on the query sub-results of the multiple query sub-tasks. In one embodiment, the result sending module 820 can be used to perform the operation S420 described above, which will not be repeated here.

[0150] According to an embodiment of the present invention, the data query module 810 includes a data acquisition submodule, a data decryption submodule and a data storage submodule.

[0151] The data acquisition submodule is used to acquire the storage data in the storage unit from the storage unit based on the data index by using the data transmission component.

[0152] The data decryption submodule is used to decrypt the stored data using the data decryption component to obtain plaintext data.

[0153] The data storage submodule is used to determine the query sub-results based on the plaintext data and store the query sub-results in the cache component.

[0154] According to an embodiment of the present invention, the result sending module 820 includes a result reading submodule, a first result uploading submodule and a second result uploading submodule.

[0155] The result reading submodule is used to read the query subresults of the S query subtasks in the query subtask group from the cache component in response to the Sth query subtask in the query subtask group being executed.

[0156] The first result uploading submodule is used to upload the first query result obtained by splicing the query sub-results of the S query subtasks to the server and clear the cache component.

[0157] The second result uploading submodule is used to upload the second query result obtained by splicing one or more query sub-results saved in the cache component to the server in response to the completion of the execution of the last query subtask in the query request, so that the server can splice the query result based on the received one or more first query results and second query results.

[0158] According to an embodiment of the present invention, the first result uploading submodule includes a first compression unit and a first uploading unit.

[0159] The first compression unit is used to compress the first query result using a compression component of a programmable device to obtain a first compression result.

[0160] The first uploading unit is used to upload the first compression result to the server.

[0161] According to an embodiment of the present invention, the second result uploading submodule includes a second compression unit and a second uploading unit.

[0162] The second compression unit is used to compress the second query result using the compression component to obtain a second compressed result.

[0163] The second uploading unit is used to upload the second compression result to the server.

[0164] According to an embodiment of the present invention, any multiple modules among the task division module 710, the relationship determination module 720, the task grouping module 730, the task sending module 740, the result determination module 750, the data query module 810 and the result sending module 820 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the task division module 710, the relationship determination module 720, the task grouping module 730, the task sending module 740, the result determination module 750, the data query module 810 and the result sending module 820 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the task division module 710, the relationship determination module 720, the task grouping module 730, the task sending module 740, the result determination module 750, the data query module 810 and the result sending module 820 can be at least partially implemented as a computer program module, which can perform the corresponding function when it is executed.

[0165] Fig. 9 A block diagram of an electronic device suitable for implementing a data transmission method according to an embodiment of the present invention is shown.

[0166] like Fig. 9As shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0167] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0168] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0169] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0170] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0171] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0172] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 901. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0173] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0174] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0175] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0176] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0177] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0178] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data transmission method, applied to a server, characterized in that: The method comprises: In response to a data query request from a user, dividing the data query request into a plurality of query subtasks; Determining, based on the respective data indexes of the plurality of query subtasks, a correspondence between the plurality of query subtasks and a plurality of data partitions included in the storage unit; Based on the corresponding relationship, one or more query subtasks corresponding to the same data partition are divided into the same query subtask group; Sending the multiple query subtasks in the query subtask group to the programmable device in sequence, so that the programmable device performs a query on the storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask; and The query result of the data query request is determined based on the query sub-results of each of the plurality of query sub-tasks from the programmable device.

2. The method according to claim 1, characterized in that Each query subtask group includes S query subtasks, where S represents the number of processing steps required for the programmable device to query the storage unit to obtain the query subresult, and S is a positive integer; The sending the plurality of query subtasks in the query subtask group to the programmable device in sequence comprises: Sending the first query subtask in the query subtask group to the programmable device for execution by the programmable device, wherein the query subtask group includes S query subtasks in the plurality of query subtasks; When the programmable device executes the s-th step, the s-th query subtask is sent to the programmable device, s=2, ..., S.

3. The method according to claim 2, characterized in that The determining the query result of the data query request based on the query sub-results of each of the plurality of query sub-tasks from the programmable device comprises: In response to receiving a second query result and one or more first query results sent by the programmable device, the second query result and one or more first query results are concatenated to obtain the query result, wherein the first query result is determined by the programmable device based on the query sub-results of each of the S query subtasks in the query subtask group other than the last query subtask group, and the second query result is determined by the programmable device based on the query sub-results of each of the one or more query subtasks in the last query subtask group.

4. The method according to claim 3, characterized in that The step of concatenating the second query result with one or more of the first query results to obtain the query result includes: decompressing the second compression result and one or more first compression results to obtain the second query result and one or more first query results, wherein the second compression result is obtained by the programmable device compressing the second query result using the compression component, and the first compression result is obtained by the programmable device compressing the first query result using the compression component; and Based on the task order of the query subtasks corresponding to the one or more first query results, the second query result and one or more first query results are concatenated to obtain the query result.

5. The method according to claim 1, characterized in that The method further comprises: In response to the user's data query request, if it is determined that the query result corresponding to the data query request is stored in the server cache, obtaining the query result from the server cache; and The query result is sent to the client, and the query result is stored in the server cache.

6. The method according to claim 1, characterized in that The method further comprises: In response to a data storage request, the data to be stored included in the data storage request is sent to the programmable device, so that the programmable device uses a data encryption component to encrypt the data to be stored, obtains encrypted data, and sends the encrypted data to the storage unit for storage.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Based on the data indexes of the plurality of query subtasks, the storage unit is divided into a plurality of data partitions, so that each data partition includes at least one query subresult of the query subtask; Sending the data partitioning result to the programmable device so that the programmable device determines a target partition from the multiple data partitions based on the data partitioning result and the data index in the query subtask; A query subresult of the query subtask is obtained from the target partition.

8. The method according to claim 7, characterized in that The method further comprises: Based on the access frequency of the programmable device to the storage unit, cache the data with an access frequency higher than an access threshold in the solid state drive; When it is determined that the query sub-result is stored on the solid state drive, the query sub-result is obtained from the solid state drive via the programmable device based on the data index using the programmable device.

9. A data transmission method, applied to a programmable device, characterized in that: The method comprises: In response to multiple query subtasks sent by the server, query the storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask, wherein the multiple query subtasks are obtained by the server in response to the user's data query request by dividing the data query request, and the multiple query subtasks are sent by the server in sequence; and Sending the query sub-results of the query sub-tasks to the server, so that the server determines the query result of the data query request based on the query sub-results of each of the plurality of query sub-tasks; The server sends the multiple query subtasks to the programmable device in sequence in the following manner: Determining a correspondence between the plurality of query subtasks and the plurality of data partitions based on respective data indexes of the plurality of query subtasks; Based on the corresponding relationship, one or more query subtasks corresponding to the same data partition are divided into the same query subtask group; The multiple query subtasks in the query subtask group are sent to the programmable device in sequence.

10. The method according to claim 9, characterized in that The querying in the storage unit based on the data index in the query subtask to obtain the query subresult of the query subtask includes: Using a data transmission component, based on a data index, acquiring the storage data in the storage unit from the storage unit; Decrypting the stored data using a data decryption component to obtain plaintext data; and Based on the plaintext data, the query sub-result is determined and the query sub-result is stored in a cache component.

11. The method according to claim 10, characterized in that Each query subtask group includes S query subtasks, where S represents the number of processing steps required for the programmable device to query the storage unit to obtain the query subresult, and S is a positive integer; The sending the query sub-result of the query sub-task to the server includes: In response to the Sth query subtask in the query subtask group being executed, reading respective query subresults of the S query subtasks in the query subtask group from the cache component; Upload the first query result obtained by concatenating the query sub-results of the S query sub-tasks to the server, and clear the cache component; In response to the completion of the execution of the last query subtask in the query request, a second query result obtained by splicing one or more query subresults stored in the cache component is uploaded to the server, so that the server can splice the query result based on the received one or more first query results and second query results.

12. The method according to claim 11, characterized in that The uploading to the server a first query result obtained by splicing the query sub-results of the S query sub-tasks, comprises: Using a compression component of the programmable device, compressing the first query result to obtain a first compressed result; and Uploading the first compression result to the server; The step of uploading the second query result obtained by splicing one or more query sub-results stored in the cache component to the server includes: Using the compression component, compressing the second query result to obtain a second compressed result; and The second compression result is uploaded to the server.

13. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 12.

14. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images