Method and system for improving network resource utilization rate of distributed database
By compressing the result sets of a distributed database, the problem of large result sets consuming network resources is solved, achieving efficient utilization of network resources and improved interactive performance.
Patent Information
- Application Number
- CN202410989021.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-26
AI Technical Summary
In distributed in-memory databases, when client query requests return large result sets, they consume a large amount of network resources, resulting in low network resource utilization.
The result set is compressed, and a preset threshold is used to determine whether compression is necessary. If compression is needed, the compressed result set is sent to the client; otherwise, the original result set is sent directly.
It effectively saves network resources and improves network resource utilization and interactive performance.
Smart Images

Figure CN119031043A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and system for improving the utilization rate of distributed database network resources. Background Art
[0002] When a distributed memory database responds to a query request from a client, the server transmits the result set to the client in sequence upon receiving the request. In many cases, the server will return many records that meet the request conditions for one request from the client, which will occupy a large amount of bandwidth.
[0003] Currently, the server receives the query request from the client, finds the results that meet the request conditions in the data, and returns the result set to the client in sequence. When the result set is large, it occupies a large amount of network resources. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for improving the utilization rate of distributed database network resources, aiming to solve the problem of more efficient utilization of network resources when a client search requests a server to return a large result set.
[0005] In order to solve the above technical problems, the present invention provides a method for improving the utilization rate of distributed database network resources, comprising the following steps:
[0006] Get query request from client;
[0007] According to the query request, the target data is obtained from the database;
[0008] Generate a result set based on the target data;
[0009] Send the result set to the client.
[0010] Preferably, sending the result set to the client specifically includes the following steps:
[0011] Compress the result set to obtain a compressed result;
[0012] Send the compressed result to the client.
[0013] Preferably, compressing the result set to obtain a compressed result; and sending the compressed result to the client specifically comprises the following steps:
[0014] Determine whether the result set needs to be compressed based on the preset threshold;
[0015] If the result set needs to be compressed, the result set is compressed and sent to the client;
[0016] If the result set does not need to be compressed, the result set is sent to the client.
[0017] Preferably, judging whether the result set needs to be compressed according to a preset threshold value specifically includes the following steps:
[0018] Define the result set as R, the result set size as k, and the preset threshold as S;
[0019] When the result set size k ≥ threshold S, it is determined that the result set needs to be compressed;
[0020] When the result set size k is less than the threshold value S, it is determined that the result set does not need to be compressed.
[0021] Preferably, compressing the result set and sending the compressed result set to the client specifically includes the following steps:
[0022] Compress the result set to obtain a compressed result set;
[0023] Determine whether compression is successful based on the size of the result set and the size of the result set after compression.
[0024] If the compression is determined to be successful, the compressed result set is sent to the client and the compressed result set is decompressed.
[0025] If it is determined that the compression fails, the compressed result set is sent to the client.
[0026] Preferably, judging whether compression is successful according to the size of the result set and the size of the result set after compression processing specifically includes the following steps:
[0027] Define the size of the result set after compression as v;
[0028] When the result set size v after compression is less than the result set size k, the compression is considered successful;
[0029] When the result set size v after compression processing equals the result set size k, it is determined that the compression has failed.
[0030] Preferably, the method further comprises the following steps:
[0031] When compression succeeds, a compression success flag f(1) is added to the result set after compression processing;
[0032] When compression fails, a compression failure flag f(0) is added to the result set after compression processing;
[0033] The client decompresses the compressed result set with the compression success flag f(1).
[0034] Preferably, the result set includes more than 1 million pieces of target data.
[0035] The present invention provides a system for improving the utilization rate of distributed database network resources, comprising:
[0036] A query request acquisition module, used to obtain a query request from a client;
[0037] A target data acquisition module is used to acquire target data from a database according to a query request;
[0038] An integration module is used to generate a result set based on target data;
[0039] The sending module is used to send the result set to the client.
[0040] Compared with the prior art, the present invention has the following beneficial effects:
[0041] The present invention utilizes compression technology to compress a large result set, thereby effectively saving network resources;
[0042] The present invention utilizes compression technology to improve network resource utilization;
[0043] The present invention utilizes compression technology to improve interactive performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.
[0045] Figure 1 It is a schematic diagram of the communication process between the memory database server and the client;
[0046] Figure 2 It is a schematic diagram of the existing query implementation process;
[0047] Figure 3 It is a flow chart of a method for improving the utilization rate of distributed database network resources in the present invention. DETAILED DESCRIPTION
[0048] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.
[0049] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.
[0050] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0051] The present invention is further described in detail below in conjunction with the accompanying drawings:
[0052] The present invention provides a method for improving the utilization rate of distributed database network resources, comprising the following steps:
[0053] Get query request from client;
[0054] According to the query request, the target data is obtained from the database;
[0055] Generate a result set based on the target data;
[0056] Send the result set to the client.
[0057] Preferably, sending the result set to the client specifically includes the following steps:
[0058] Compress the result set to obtain a compressed result;
[0059] Send the compressed result to the client.
[0060] Preferably, compressing the result set to obtain a compressed result; and sending the compressed result to the client specifically comprises the following steps:
[0061] Determine whether the result set needs to be compressed based on the preset threshold;
[0062] If the result set needs to be compressed, the result set is compressed and sent to the client;
[0063] If the result set does not need to be compressed, the result set is sent to the client.
[0064] Preferably, judging whether the result set needs to be compressed according to a preset threshold value specifically includes the following steps:
[0065] Define the result set as R, the result set size as k, and the preset threshold as S;
[0066] When the result set size k ≥ threshold S, it is determined that the result set needs to be compressed;
[0067] When the result set size k is less than the threshold value S, it is determined that the result set does not need to be compressed.
[0068] Preferably, compressing the result set and sending the compressed result set to the client specifically includes the following steps:
[0069] Compress the result set to obtain a compressed result set;
[0070] Determine whether compression is successful based on the size of the result set and the size of the result set after compression.
[0071] If the compression is determined to be successful, the compressed result set is sent to the client and the compressed result set is decompressed.
[0072] If it is determined that the compression fails, the compressed result set is sent to the client.
[0073] Preferably, judging whether compression is successful according to the size of the result set and the size of the result set after compression processing specifically includes the following steps:
[0074] Define the size of the result set after compression as v;
[0075] When the result set size v after compression is less than the result set size k, the compression is considered successful;
[0076] When the result set size v after compression processing equals the result set size k, it is determined that the compression has failed.
[0077] Preferably, the method further comprises the following steps:
[0078] When compression succeeds, a compression success flag f(1) is added to the result set after compression processing;
[0079] When compression fails, a compression failure flag f(0) is added to the result set after compression processing;
[0080] The client decompresses the compressed result set with the compression success flag f(1).
[0081] Preferably, the result set includes more than 1 million pieces of target data.
[0082] The present invention provides a system for improving the utilization rate of distributed database network resources, comprising:
[0083] A query request acquisition module, used to obtain a query request from a client;
[0084] A target data acquisition module is used to acquire target data from a database according to a query request;
[0085] An integration module is used to generate a result set based on target data;
[0086] The sending module is used to send the result set to the client.
[0087] The present invention greatly improves the utilization rate of network resources of distributed memory databases in scenarios with large result sets;
[0088] The present invention can improve the interactive performance in large result set scenarios.
[0089] In order to better illustrate the technical effect of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process:
[0090] Example 1, a method for improving the utilization rate of distributed database network resources:
[0091] The communication process between the memory database server and the client, such as Figure 1 As shown in the figure, when the distributed memory database responds to the query request of the client, the server sends the result set to the client in sequence after receiving the request from the client. In many cases, the server will return many records that meet the request conditions for one request from the client.
[0092] The existing query implementation process, such as Figure 2 As shown, the following steps are included:
[0093] 1. The client application initiates a query request;
[0094] 2. The server receives the query request and searches the data for results that meet the conditions;
[0095] 3. The server forms a result set with the results that meet the conditions;
[0096] 4. Feed the result set back to the client application.
[0097] From the above process, we can see that the command statement for the query request initiated by the client consumes limited network resources. Steps 2 and 3 are completed on the server side, without consuming network resources. Step 4 feeds the result set back to the client. This step increases linearly with the different sorting of the result set. In actual cases, there are cases where more than 1 million records are returned.
[0098] In this embodiment, a determination of the result set size is added to the 4th step on the basis of the original solution. For example, Figure 3 As shown, the result set is defined as R; the result set size is defined as k, and a threshold S is set. When k≥S, the result set is compressed, and a compression success flag is set as f(1), and the compressed size is v. When v<k, the compressed result set R(v) is transmitted to the client application; when v = k, it means the compression is invalid, and a compression failure flag is set as f(0), and the result set R(k) is transmitted to the client application. When the client receives the result set, it detects the compression flag. When it is f(1), the result set is decompressed.
[0099] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules, modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units, modules or components can be combined or integrated into another device, or some features can be ignored or not executed.
[0100] The units may or may not be physically separated. The components shown as units may be a physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0101] In addition, the functional units in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.
[0102] Specifically, according to the embodiments disclosed by the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium. The computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present invention are executed. It should be noted that the above computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above.
[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0104] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A method for improving the utilization rate of distributed database network resources, characterized in that: The following steps are involved: Get query request from client; According to the query request, the target data is obtained from the database; Generate a result set based on the target data; Send the result set to the client.
2. The method for improving the utilization rate of distributed database network resources according to claim 1, characterized in that: Sending the result set to the client includes the following steps: Compress the result set to obtain a compressed result; Send the compressed result to the client.
3. The method for improving the utilization rate of distributed database network resources according to claim 2, characterized in that: Compress the result set to obtain a compression result; and send the compression result to the client, specifically including the following steps: Determine whether the result set needs to be compressed based on the preset threshold; If the result set needs to be compressed, the result set is compressed and sent to the client; If the result set does not need to be compressed, the result set is sent to the client.
4. The method for improving the utilization rate of distributed database network resources according to claim 3, characterized in that: Based on the preset threshold, determine whether the result set needs to be compressed, which specifically includes the following steps: Define the result set as R, the result set size as k, and the preset threshold as S; When the result set size k ≥ threshold S, it is determined that the result set needs to be compressed; When the result set size k is less than the threshold value S, it is determined that the result set does not need to be compressed.
5. The method for improving the utilization rate of distributed database network resources according to claim 4, characterized in that: The result set is compressed and sent to the client, which specifically includes the following steps: Compress the result set to obtain a compressed result set; Determine whether compression is successful based on the size of the result set and the size of the result set after compression. If the compression is judged to be successful, the compressed result set is sent to the client and the compressed result set is decompressed; If it is determined that the compression fails, the compressed result set is sent to the client.
6. The method for improving the utilization rate of distributed database network resources according to claim 5, characterized in that: According to the size of the result set and the size of the result set after compression, whether the compression is successful is determined, which specifically includes the following steps: Define the size of the result set after compression as v; When the result set size v after compression is less than the result set size k, the compression is considered successful; When the result set size v after compression processing equals the result set size k, it is determined that the compression has failed.
7. The method for improving the utilization rate of distributed database network resources according to claim 6, characterized in that: The following steps are also included: When compression succeeds, a compression success flag f(1) is added to the result set after compression processing; When compression fails, a compression failure flag f(0) is added to the result set after compression processing; The client decompresses the compressed result set with the compression success flag f(1).
8. The method for improving the utilization rate of distributed database network resources according to claim 1, characterized in that: The result set includes more than 1 million pieces of target data.
9. A system for improving the utilization rate of distributed database network resources, used to implement the method for improving the utilization rate of distributed database network resources as claimed in any one of claims 1 to 8, characterized in that: include: A query request acquisition module, used to obtain a query request from a client; A target data acquisition module is used to acquire target data from a database according to a query request; An integration module is used to generate a result set based on target data; The sending module is used to send the result set to the client.
