Data query method, device, equipment and medium
By establishing connections and creating threads during the data query process, and using HTTP SSE technology to obtain cursors and send data in real time, we solved the database performance issues of paging queries and memory overflow issues of full queries, and achieved efficient data transmission and query.
Patent Information
- Application Number
- CN202111547088.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-16
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2041-12-16
AI Technical Summary
In the existing technology, the paging data query method causes database performance problems and multiple network requests, resulting in increased response time; pulling data in full may cause memory overflow exceptions and take a long time.
By establishing a connection with the second software service, creating a thread and using HTTP SSE technology to initiate a data query request, obtaining a cursor and traversing the database to obtain the target data, and sending it to the second software service when the quantity threshold is reached, multiple repeated queries are avoided.
It improves data query efficiency, avoids memory overflow problems, reduces the exponential growth of response time, and optimizes database performance.
Smart Images

Figure CN114218279B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data query method, device, equipment and medium. Background Art
[0002] When upstream software services that provide data queries provide large-scale queries to downstream software services that retrieve data, they typically use either a paging-based query or a full all-at-once query. The paging-based query method triggers multiple network requests, and each subsequent paging request requires the upstream service to reconnect to the database layer to read data. The database also needs to repeat the same operations as the previous query request to execute the data query. Paging queries using skip or offset methods can also cause database performance issues as the number of pages increases. The total time required to retrieve all data can also be lengthy, increasing with the number of downstream services. A full all-at-once query requires the upstream service to fetch all the required data from the database. Reading all the data into memory can cause exceptions such as memory overflow in the application service, and the query time can be very long, even increasing exponentially with the number of downstream services. Summary of the Invention
[0003] In view of this, the purpose of this application is to provide a method that can improve data query efficiency and avoid memory overflow. The specific solution is as follows:
[0004] In a first aspect, the present application provides a data query method, applied to a first software service, comprising:
[0005] Obtaining a first data query request sent by a second software service;
[0006] establishing a first connection with the second software service;
[0007] Create a new thread to obtain a first thread;
[0008] Initiating a second data query request corresponding to the first data query request to a database using the first thread, and obtaining a cursor corresponding to the second data query request returned by the database;
[0009] traversing the cursor using the first thread to obtain corresponding target data from the database;
[0010] During the traversal process, whenever the amount of the acquired target data reaches a quantity threshold, the target data is sent to the second software service through the first connection until the traversal is completed.
[0011] Optionally, the using the first thread to initiate a second data query request corresponding to the first data query request to the database includes:
[0012] The first thread is used to initiate a second data query request corresponding to the first data query request and carrying cursor timeout prohibition information to the database.
[0013] Optionally, sending the target data to the second software service through the first connection includes:
[0014] creating a new thread to obtain a second thread, and using the second thread to send the target data to the second software service through the first connection;
[0015] Alternatively, a thread pool is used to send the target data to the second software service through the first connection.
[0016] Optionally, the cursor is first searched from the internal cache of the database based on the second data query request, and if found in the internal cache, the cursor is read from the internal cache and returned.
[0017] Optionally, establishing the first connection with the second software service includes:
[0018] A first connection is established with the second software service based on HTTP SSE technology.
[0019] Optionally, before using the first thread to initiate a second data query request corresponding to the first data query request to the database, the method further includes:
[0020] determining a second connection between the first software service and the database;
[0021] Accordingly, the initiating of a second data query request corresponding to the first data query request to the database by using the first thread includes:
[0022] The first thread is used to initiate a second data query request corresponding to the first data query request to the database through the second connection.
[0023] Optionally, determining the second connection between the first software service and the database includes:
[0024] searching for an available connection from existing connections between the first software service and the database;
[0025] If an available connection is found, the available connection is determined as the second connection;
[0026] Otherwise, it is determined whether the number of existing connections to the database reaches a preset connection number threshold; if not, a new connection is created to obtain a second connection.
[0027] In a second aspect, the present application discloses a data query device, applied to a first software service, comprising:
[0028] A query request acquisition module, configured to acquire a first data query request sent by a second software service;
[0029] A first connection establishing module, configured to establish a first connection with the second software service;
[0030] A first thread creation module, configured to create a new thread to obtain a first thread;
[0031] a query request sending module, configured to initiate a second data query request corresponding to the first data query request to the database using the first thread;
[0032] A cursor acquisition module, configured to acquire a cursor corresponding to the second data query request returned by the database;
[0033] a target data acquisition module, configured to utilize the first thread to traverse the cursor to acquire corresponding target data from the database;
[0034] The target data sending module is used to send the target data to the second software service through the first connection whenever the amount of the acquired target data reaches a quantity threshold during the traversal process, until the traversal is completed.
[0035] In a third aspect, the present application discloses an electronic device, comprising:
[0036] Memory, used to store computer programs;
[0037] The processor is used to execute the computer program to implement the aforementioned data query method.
[0038] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, which implements the aforementioned data query method when executed by a processor.
[0039] In a fifth aspect, the present application discloses a computer program product, which implements the aforementioned data query method when executed.
[0040] As can be seen, the present application first obtains a first data query request sent by a second software service, establishes a first connection with the second software service, and creates a new thread to obtain a first thread. The first thread is then used to initiate a second data query request corresponding to the first data query request to the database, and obtains a cursor corresponding to the second data query request returned by the database. The first thread is then used to traverse the cursor to obtain the corresponding target data from the database. During the traversal process, whenever the amount of target data obtained reaches a threshold, the target data is sent to the second software service via the first connection until the traversal is completed. That is, in the present application, the first software service obtains the cursor returned by the database, traverses the cursor to obtain the corresponding data from the database, and when the obtained data meets a certain amount, sends it to the second software service via the connection with the second software service. In this way, data can be queried by sending a data query request to the database only once, avoiding the database from performing multiple repeated tasks, avoiding the problem of exponentially increasing response time as the number of services requesting data increases, improving data query efficiency, and avoiding the problem of memory overflow caused by pulling all the data. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.
[0042] Figure 1 A schematic diagram of the system framework applicable to the data query solution provided in this application;
[0043] Figure 2 A flow chart of a data query method disclosed in this application;
[0044] Figure 3 A specific flow chart of data query in paging mode provided by this application;
[0045] Figure 4 A specific flowchart of full-data query provided for this application;
[0046] Figure 5 A specific data query flow chart disclosed in this application;
[0047] Figure 6 This is a structural diagram of a data query device disclosed in this application;
[0048] Figure 7 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0050] The software service that provides data query is an upstream service. When providing large-scale queries to the software service that obtains data, that is, the downstream service, it currently generally uses two methods: paging and pulling, or pulling the required data in full at one time. In the paging and pulling method, the paging query will initiate multiple network requests. At the same time, when the next paging query request is made, the upstream service needs to re-establish a new connection with the database layer to read the data. The database also needs to repeat the series of operations of the previous query request to execute the data query. As the page number of the paging increases, it may also cause performance problems of the database. At the same time, the total time to obtain all the data will also be very long, and it will also increase with the increase of downstream services. Pulling all the required data in full at one time means that the upstream service needs to pull the full amount of data from the database. Reading the full amount of data into the memory may cause abnormalities such as memory overflow of the application service, and the pulling time may be very long or increase exponentially with the increase of downstream services. To this end, the present application provides a data query method that can improve data query efficiency and avoid memory overflow.
[0051] The system framework used in the data query solution of this application can be found in Figure 1 As shown, it may specifically include: a second electronic device 101, a first electronic device 102, and a communication connection between the second electronic device 101 and the first electronic device 102 via a network 103. The first electronic device and the second electronic device may both be terminal devices or servers. For example, the first electronic device may be a server running a first software service, and the second electronic device may be a terminal device running a second software service. Figure 1 As shown, the second electronic device sends a data query request to the first electronic device through the network, and the first electronic device returns the queried target data to the second electronic device.
[0052] In some embodiments, a SaaS application may be implemented through the collaboration of the first software service and the second software service, and the SaaS application may be used to provide functions such as office work and chatting.
[0053] See also Figure 2 As shown, the embodiment of the present application discloses a data query method, which is applied to a first software service, including:
[0054] Step S11: Obtain a first data query request sent by the second software service.
[0055] It can be understood that in the embodiment of the present application, the first software service is an upstream service that provides data query, and the second service is a downstream service that needs to query data.
[0056] The first data query request carries a query condition.
[0057] Step S12: Establishing a first connection with the second software service.
[0058] In some embodiments, the embodiments of the present application may establish a first connection with the second software service based on HTTP (Hyper Text Transfer Protocol) SSE (Server-Sent Events) technology. That is, a data channel is established between the first software service and the second software service based on HTTP SSE technology, so that after the data is queried, the data is transmitted using the data channel.
[0059] HTTP is the protocol used to transmit hypertext from a World Wide Web server to a local browser. SSE is a lightweight alternative to WebSocket that uses HTTP. Strictly speaking, HTTP doesn't support server push, but when a server announces to a client that it intends to send streaming information, the client maintains the connection, which is how SSE works.
[0060] Step S13: Create a new thread to obtain a first thread.
[0061] That is, the embodiment of the present application creates a new thread for data query, thereby improving response efficiency.
[0062] Step S14: using the first thread to initiate a second data query request corresponding to the first data query request to the database, and obtaining a cursor corresponding to the second data query request returned by the database.
[0063] Among them, the first data query request is a data query request sent by the second software service to the first software service, which can be based on HTTP SSE technology or TCP protocol. The second data query request is a request initiated by the first software service to the database, which can be based on TCP protocol.
[0064] In some embodiments, the present application can utilize the first thread to initiate a second data query request to the database corresponding to the first data query request, carrying information prohibiting a cursor timeout. For example, if the cursor timeout is set to ten minutes, then the time required to retrieve data from the database cannot exceed 10 minutes.
[0065] Such an operation can avoid cursor timeout when traversing the cursor to obtain the corresponding target data from the database, and can obtain the corresponding target data in time.
[0066] The cursor is a cursor that the database first searches from a cache (internal cache) based on the second data query request, and if found in the cache, is read from the cache and returned.
[0067] That is, in the embodiment of the present application, when the database receives a second data query request, it first searches the internal cache to see if there is a cursor corresponding to the second data query request, that is, a cursor that meets the query conditions. If so, it is read from the internal cache. In this way, the use of the cache can further improve query efficiency. If the cursor is not found in the internal cache, the database searches data in the database that is different from the internal cache to analyze, optimize, and execute the query statement corresponding to the second data query request.
[0068] Furthermore, in an embodiment of the present application, before using the first thread to initiate a second data query request corresponding to the first data query request to the database, a second connection between the first software service and the database is determined, and then the first thread is used to initiate a second data query request corresponding to the first data query request to the database through the second connection.
[0069] The step of determining the second connection between the first software service and the database specifically includes: searching for an available connection from existing connections between the first software service and the database; if an available connection is found, determining the available connection as the second connection; otherwise, determining whether the number of existing connections to the database reaches a preset connection threshold; if not, creating a new connection to obtain the second connection; if so, waiting for the connection to be released.
[0070] That is, when determining the communication connection with the database, the embodiment of the present application gives priority to existing available connections to save connection resources. If there are no available connections, it is determined whether the number of connections between the first service and the database reaches the quantity threshold. If not, a new connection is created.
[0071] It should be pointed out that a cursor is a method of processing data. In order to view or process the data in the result set, the cursor provides the ability to browse the data forward or backward in the result set one or more rows at a time.
[0072] Step S15: traverse the cursor using the first thread to obtain corresponding target data from the database.
[0073] Step S16: During the traversal process, whenever the amount of the acquired target data reaches a quantity threshold, the target data is sent to the second software service through the first connection until the traversal is completed.
[0074] In one embodiment, a new thread may be created to obtain a second thread, and the second thread is used to send the target data to the second software service through the first connection.
[0075] In another embodiment, a thread pool is used to send the target data to the second software service through the first connection.
[0076] The thread pool may be established in advance to avoid frequent creation of threads and waste of resources.
[0077] It should be noted that the embodiment of the present application uses different threads to perform cursor traversal and data transmission respectively, which can send data to the second software service in real time, avoid cursor timeout, and further improve data query efficiency. At the same time, the division of labor between different threads can also improve the success rate of data query.
[0078] In some embodiments, the quantity threshold can be configured based on actual needs. For example, it can be set to a single piece of data, meaning that once a single piece of data is acquired, it is sent to the second software service. Alternatively, it can be set to multiple pieces of data, meaning that when multiple pieces of data are acquired, they are all sent to the second software service. The quantity threshold can be preset or changed based on actual circumstances. For example, the quantity threshold can be negatively correlated with the urgency of the first data query request (the urgency of the related business). That is, the higher the urgency of the first data query request, indicating a higher demand for real-time data transmission, the smaller the quantity threshold, thereby improving the real-time nature of data transmission.
[0079] As can be seen, the embodiment of the present application first obtains a first data query request sent by a second software service, establishes a first connection with the second software service, and creates a new thread to obtain a first thread. Then, the first thread is used to initiate a second data query request corresponding to the first data query request to the database, and obtains a cursor corresponding to the second data query request returned by the database. Then, the first thread is used to traverse the cursor to obtain the corresponding target data from the database. During the traversal process, whenever the amount of the target data obtained reaches a threshold, the target data is sent to the second software service via the first connection until the traversal is completed. That is, in the present application, the first software service obtains the cursor returned by the database, traverses the cursor to obtain the corresponding data from the database, and when the obtained data meets a certain amount, it is sent to the second software service via the connection with the second software service. In this way, it is possible to query the data by sending a data query request to the database only once, avoiding the database from performing multiple repeated tasks, avoiding the problem of exponentially increasing response time as the number of services requesting data increases, improving data query efficiency, and avoiding the problem of memory overflow caused by pulling all the data.
[0080] Below, in order to further illustrate the beneficial effects of the present application scheme compared with the prior art, a further comparative description of the prior art and the present application scheme is given. Figure 3 As shown, Figure 3 A specific flow chart of performing data query in a paging manner is provided for an embodiment of the present application. The second software service initiates a query request for page n, the first software service initiates a query request for page n to the database, the database returns the data of page n to the first software service, and it takes m time. The first software service wants the second software service to return the data of page n, and it may take 2m. In this way, according to the total number of pages queried n, the above steps are repeated n times. It can be seen that paging query will initiate multiple query requests. At the same time, when the next paging query request is made, the first software service needs to re-establish a new connection with the database layer to read the data. The database also needs to repeat a series of operations of the previous query request to execute the data query. As the page number of the paging increases, it may also cause performance problems of the database. At the same time, the total time to obtain all the data will be very long, and it will also increase with the increase of software services requesting data. See. Figure 4 As shown, Figure 4A specific flowchart of full data query is provided for an embodiment of the present application. To perform a full pull, the second software service sends a full query request to the first software service, the first software service sends a full query request to the database, the database returns the full data, and the time taken is k. The first software service returns the full data to the second software service, and the time taken may be 2k. Pulling all the required data at a time means that the first software service provider also needs to pull the full data from the database. Reading the full data into the memory may cause abnormalities such as memory overflow in the application service, and the pulling time may be very long or increase exponentially with the increase in the number of services requesting data. See Figure 5 As shown, Figure 5 A specific data query flow chart disclosed in an embodiment of the present application. When the second software service initiates a request to the first software service, the first software service transmits data to the second software service based on HTTP SSE technology. After the request reaches the first software service, the first software service opens a new thread to obtain data from the database through a cursor, and traverses the obtained cursor to obtain all required data, and writes data to the second software service while traversing. Since reading cursor data and sending data to the second software service are performed simultaneously, it is equivalent to the work of a pipeline loop. Only one request needs to be initiated to query the required data. Avoid exceptions such as memory overflow caused by the service reading data during data transmission, and avoid the problem of exponential increase in interface response time as the number of downstream services requesting data increases.
[0081] It is understandable that, whether using a paging query or a full request query, as the number of services requesting data increases, data is requested level by level, and the second software service at the last level interacts with the first software service, retrieving data from the database through the first software service. This means that data query time increases exponentially as the number of downstream services requesting data increases. The solution provided in this application, however, establishes a data channel between the first and second software services using HTTP SSE technology. The first software service retrieves data from the database using a cursor, traverses the obtained cursor to obtain all required data, and writes data to the second software service while traversing. This avoids a single full fetch, forming a data stream. If the time required for paging or full queries on n downstream services requesting data is n*t, then the time required to provide data queries to downstream services requesting data using an SSE-based and database cursor-based approach is only 1*t.
[0082] See also Figure 6 As shown, the embodiment of the present application discloses a data query device, which is applied to a first software service, including:
[0083] A query request obtaining module 11 is configured to obtain a first data query request sent by a second software service;
[0084] A first connection establishing module 12, configured to establish a first connection with the second software service;
[0085] A first thread creation module 13, configured to create a new thread to obtain a first thread;
[0086] A query request sending module 14 is configured to use the first thread to initiate a second data query request corresponding to the first data query request to the database;
[0087] A cursor acquisition module 15 is configured to acquire a cursor corresponding to the second data query request returned by the database;
[0088] a target data acquisition module 16, configured to utilize the first thread to traverse the cursor to acquire corresponding target data from the database;
[0089] The target data sending module 17 is configured to send the target data to the second software service through the first connection whenever the amount of the acquired target data reaches a threshold value during the traversal process, until the traversal is completed.
[0090] As can be seen, the embodiment of the present application first obtains a first data query request sent by a second software service, establishes a first connection with the second software service, and creates a new thread to obtain a first thread. Then, the first thread is used to initiate a second data query request corresponding to the first data query request to the database, and obtains a cursor corresponding to the second data query request returned by the database. Then, the first thread is used to traverse the cursor to obtain the corresponding target data from the database. During the traversal process, whenever the amount of the target data obtained reaches a threshold, the target data is sent to the second software service via the first connection until the traversal is completed. That is, in the present application, the first software service obtains the cursor returned by the database, traverses the cursor to obtain the corresponding data from the database, and when the obtained data meets a certain amount, it is sent to the second software service via the connection with the second software service. In this way, it is possible to query the data by sending a data query request to the database only once, avoiding the database from performing multiple repeated tasks, avoiding the problem of exponentially increasing response time as the number of services requesting data increases, improving data query efficiency, and avoiding the problem of memory overflow caused by pulling all the data.
[0091] The first connection establishing module 12 is specifically configured to establish a first connection with the second software service based on HTTP SSE technology.
[0092] Furthermore, the query request sending module 14 is specifically configured to use the first thread to initiate a second data query request corresponding to the first data query request and carrying cursor timeout prohibition information to the database.
[0093] In one embodiment, the target data sending module 17 is specifically configured to create a new thread to obtain a second thread, and use the second thread to send the target data to the second software service through the first connection.
[0094] In another embodiment, the target data sending module 17 is specifically configured to use a thread pool to send the target data to the second software service through the first connection.
[0095] In addition, the cursor is first searched from the internal cache of the database based on the second data query request, and if found in the internal cache, the cursor is read from the internal cache and returned.
[0096] Furthermore, the apparatus further includes a second connection determining module configured to determine a second connection between the first software service and the database before initiating a second data query request corresponding to the first data query request to the database using the first thread;
[0097] Correspondingly, the query request sending module 14 is specifically configured to utilize the first thread to initiate a second data query request corresponding to the first data query request to the database through the second connection.
[0098] Furthermore, the second connection determination module is specifically configured to search for an available connection from existing connections between the first software service and the database; if an available connection is found, determine the available connection as the second connection; otherwise, determine whether the number of existing connections to the database reaches a preset connection number threshold; if not, create a new connection to obtain the second connection.
[0099] See also Figure 7 As shown, an embodiment of the present application discloses an electronic device 20, including a processor 21 and a memory 22; wherein the memory 22 is used to store a computer program; the processor 21 is used to execute the computer program, and the data query method disclosed in the above embodiment.
[0100] For the specific process of the above data query method, please refer to the corresponding content disclosed in the above embodiments, which will not be repeated here.
[0101] Furthermore, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.
[0102] In addition, the electronic device 20 also includes a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26; wherein the power supply 23 is used to provide an operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input / output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0103] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the data query method disclosed in the aforementioned embodiment.
[0104] For the specific process of the above data query method, please refer to the corresponding content disclosed in the above embodiments, which will not be repeated here.
[0105] The embodiments of the present application also disclose a computer program product, which, when executed, implements the data query method disclosed in the aforementioned embodiments.
[0106] For the specific process of the above data query method, please refer to the corresponding content disclosed in the above embodiments, which will not be repeated here.
[0107] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.
[0108] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0109] The above is a detailed introduction to a data query method, device, equipment and medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A data query method, characterized in that: Applied to the First Software Service, including: Obtaining a first data query request sent by a second software service; establishing a first connection with the second software service; Create a new thread to obtain a first thread; Initiating a second data query request corresponding to the first data query request to a database using the first thread, and obtaining a cursor corresponding to the second data query request returned by the database; the cursor is first searched from an internal cache of the database based on the second data query request, and if the cursor is found in the internal cache, the cursor is read from the internal cache and returned; traversing the cursor using the first thread to obtain corresponding target data from the database; During the traversal process, whenever the amount of the target data obtained reaches a quantity threshold, the target data is sent to the second software service through the first connection until the traversal is completed; wherein different threads are used to perform cursor traversal and data sending respectively, and the quantity threshold is negatively correlated with the urgency of the first data query request.
2. The data query method according to claim 1, characterized in that: The using the first thread to initiate a second data query request corresponding to the first data query request to the database includes: The first thread is used to initiate a second data query request corresponding to the first data query request and carrying cursor timeout prohibition information to the database.
3. The data query method according to claim 1, characterized in that: The sending the target data to the second software service through the first connection includes: creating a new thread to obtain a second thread, and using the second thread to send the target data to the second software service through the first connection; Alternatively, a thread pool is used to send the target data to the second software service through the first connection.
4. The data query method according to claim 1, characterized in that: The establishing of the first connection with the second software service includes: A first connection is established with the second software service based on HTTP SSE technology.
5. The data query method according to any one of claims 1 to 4, characterized in that: Before using the first thread to initiate a second data query request corresponding to the first data query request to the database, the method further includes: determining a second connection between the first software service and the database; Accordingly, the initiating of a second data query request corresponding to the first data query request to the database by using the first thread includes: The first thread is used to initiate a second data query request corresponding to the first data query request to the database through the second connection.
6. The data query method according to claim 5, characterized in that: Determining a second connection between the first software service and the database includes: searching for an available connection from existing connections between the first software service and the database; If an available connection is found, the available connection is determined as the second connection; Otherwise, it is determined whether the number of existing connections to the database reaches a preset connection number threshold; if not, a new connection is created to obtain a second connection.
7. A data query device, characterized in that: Applied to the First Software Service, including: A query request acquisition module, configured to acquire a first data query request sent by a second software service; A first connection establishing module, configured to establish a first connection with the second software service; A first thread creation module, configured to create a new thread to obtain a first thread; a query request sending module, configured to initiate a second data query request corresponding to the first data query request to the database using the first thread; a cursor acquisition module, configured to acquire a cursor corresponding to the second data query request returned by the database; the cursor is first searched from the internal cache of the database based on the second data query request, and if found in the internal cache, the cursor is read from the internal cache and returned; a target data acquisition module, configured to utilize the first thread to traverse the cursor to acquire corresponding target data from the database; The target data sending module is configured to send the target data to the second software service through the first connection whenever the amount of the target data obtained reaches a quantity threshold during the traversal process, until the traversal is completed; wherein different threads are used to perform cursor traversal and data sending respectively, and the quantity threshold is negatively correlated with the urgency of the first data query request.
8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the data query method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed by a processor, implements the data query method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Data query method and device, and electronic equipment
CN107545050A
Data query method and device, computer device and storage medium
CN109885589A
Query request processing method, device and equipment and key value query system
CN113360525A