A slip data query method, a query system and a storage medium
By asynchronously acquiring and caching document identification information, the memory consumption problem during large-scale database queries was solved, query performance was improved, and efficient document data querying was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN WOJIA CLOUD NETWORK TECH CO LTD
- Filing Date
- 2023-09-06
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies consume a lot of memory and significantly reduce query performance when querying large amounts of database data.
An asynchronous method for acquiring and caching document identification information is adopted. The synchronous server receives the query request and queries the data page for the first time. The asynchronous server asynchronously acquires and caches the document identification information of multiple data pages, and uses caching middleware for sorting and storage. After the client specifies the target page number, the synchronous server retrieves the target document identification information from the cache and queries the database.
It effectively reduces memory consumption when querying data and improves query performance, especially when dealing with large amounts of data.
Smart Images
Figure CN117171404B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method, system and storage medium for querying document data. Background Technology
[0002] The existing database stores document data lists in a paginated format, with each pagination containing multiple document data entries. These document data refer to files or electronic records that document and transmit various business information within the company. Document data plays a crucial role in recording, tracking, and verifying business activities.
[0003] When querying document data in a database, pagination queries are typically used. This involves using the same query conditions and sorting method to query the document data in each data page. The server retrieves a set of data from n data pages in the database, and then iterates through these n data page sets to retrieve the document data from the nth page.
[0004] When the amount of data in the database is large, the number of data in each data page is also larger. Using existing query technology can easily cause large memory consumption and a significant decrease in query performance during the data query process. Summary of the Invention
[0005] This application provides a method, system, and storage medium for querying document data; it can reduce memory consumption during data querying and improve query performance.
[0006] This application provides a method for querying document data, characterized by including:
[0007] Obtain a document query request, wherein the document query request includes query filtering conditions;
[0008] When querying document data in the database's preset data pages based on the aforementioned query filtering conditions, document identification information corresponding to multiple data pages is asynchronously obtained and cached; wherein, the database contains multiple data pages, and each data page contains multiple document data and document identification information corresponding to the document data;
[0009] Get the target page number for data pagination;
[0010] Determine the target document identifier information corresponding to the target page number from the cached document identifier information;
[0011] Based on the target document identification information, query the document data in the database.
[0012] Furthermore, the document query request includes:
[0013] Retrieve multiple query conditions;
[0014] The multiple query conditions are assembled into the query filter conditions.
[0015] Furthermore, the method is applied to a server, which includes a synchronization server;
[0016] Querying document data in the database's preset data pagination based on the aforementioned query filtering conditions includes:
[0017] After receiving the query filter conditions, the synchronization server obtains the document data of the first data page in the database, wherein the first data page is the data page queried for the first time in the database;
[0018] The synchronization server uses the query filter conditions to query the document data of the first data page and sends the query results back to the client.
[0019] Furthermore, the server also includes an asynchronous server;
[0020] The asynchronous acquisition and caching of document identification information corresponding to multiple data pages includes:
[0021] The asynchronous server receives the asynchronous message sent by the synchronous server when querying the document data of the first data page;
[0022] The asynchronous server queries and caches the document identification information of multiple data pages in the database based on the asynchronous message, and sends the cached document identification information to the synchronous server.
[0023] Furthermore, the document identification information corresponding to the cached multiple data pages includes:
[0024] The sorting rules for document identification information are determined based on the page numbers of the data pagination.
[0025] Based on the sorting rules, the document identification information corresponding to multiple data pages is cached in the cache middleware.
[0026] Furthermore, the target page number for obtaining data pagination includes:
[0027] Retrieve the pagination display data corresponding to the data pagination in the database;
[0028] The pagination display data is sent to the client, and the target page number for data pagination determined by the client based on the pagination display data is received.
[0029] Furthermore, determining the target document identifier information corresponding to the target page number from the cached document identifier information includes:
[0030] Determine the target data pagination corresponding to the target page number;
[0031] The target document identifier information corresponding to the target data pagination is determined from the cached document identifier information.
[0032] This application also provides a document data query system, including:
[0033] The first acquisition unit is used to acquire a data query request, wherein the data query request includes query filtering conditions;
[0034] The caching unit is used to obtain and cache document identification information corresponding to multiple data pages when querying document data in a preset data page of the database based on the query filtering conditions; wherein, the database contains multiple data pages, and each data page contains multiple document data and document identification information corresponding to the document data;
[0035] The second acquisition unit is used to acquire the target page number for data pagination;
[0036] The determining unit is used to determine the target document identification information corresponding to the target page number from the cached document identification information;
[0037] The query unit is used to query document data in the database based on the target document identification information.
[0038] This application also provides an electronic device, including:
[0039] Central processing unit, memory, and input / output interfaces;
[0040] The memory is either a short-term storage memory or a persistent storage memory;
[0041] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the methods described above.
[0042] This application also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method described above.
[0043] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:
[0044] In this embodiment, a document query request is obtained, which includes query filtering conditions. When querying document data in a preset data page of the database based on the query filtering conditions, document identification information corresponding to multiple data pages is asynchronously obtained and cached. The target page number of the data page is obtained. The target document identification information corresponding to the target page number is determined from the cached document identification information. Document data in the database is queried based on the target document identification information.
[0045] As can be seen, by determining the target document identifier from the cached document identifier information, and then querying the document data corresponding to the target document identifier from the database, without having to query the document data too much from the database, the memory consumption during data query is effectively reduced and the query performance is improved. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.
[0047] Figure 1 This is a flowchart illustrating a document data query process disclosed in an embodiment of this application;
[0048] Figure 2 This is a query interaction diagram of document data disclosed in an embodiment of this application;
[0049] Figure 3 This is a diagram of a document data query system disclosed in an embodiment of this application;
[0050] Figure 4 This is a schematic diagram of an electronic device disclosed in an embodiment of this application. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0052] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0053] In the following description, expressions such as "one specific implementation" or "one specific example" are used, which describe a subset of all possible embodiments. However, it is understood that "one specific implementation" or "one specific example" can be the same subset or a different subset of all possible embodiments, and can be combined with each other without conflict. In the following description, the term "plural" means at least two. The statement that a certain value reaches a threshold (if it exists) as used in this application may, in some specific examples, include cases where the former is greater than the latter.
[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0055] Existing methods for querying document data in a database typically employ pagination. This involves using the same query conditions and sorting method to retrieve document data from each page. The server retrieves a set of n data pages from the database, then iterates through these n pages to retrieve the document data from the nth page. However, when the database contains a large amount of document data, and each page contains more documents, existing query techniques can easily lead to high memory consumption and a significant decrease in query performance. Therefore, this application provides a document data query method that reduces memory consumption and improves query performance. Figure 1 As shown, the specific steps include the following:
[0056] 101. Obtain document query request.
[0057] In this embodiment, a document query request can be obtained, such as by receiving a document query request sent by a client through a server. The document query request includes query filtering conditions, which are mandatory filtering conditions that filter or exclude data based on specific criteria. These query filtering conditions can be organizational information, date, or material number, etc. In practical applications, many similar filtering conditions exist, but are not limited here. The client is the terminal device used by the user, such as a mobile phone or computer, etc., and is not specifically limited here.
[0058] 102. When querying data in a preset data pagination based on query filtering conditions, asynchronously obtain and cache the document identification information corresponding to multiple data paginations.
[0059] It is understood that this document query request is used to retrieve document data from the database. After receiving the document query request, the server can query the document data in the database based on the query filter conditions in the document query request. The database contains multiple data pages, each containing multiple document data and corresponding document identification information. It is understood that the document identification information can be a document primary key ID or a document sequence number; the specific requirement is not limited here. Each document data has a unique document identification information. The document data in each data page can be displayed in a two-dimensional (rows and columns) grid. Column fields are displayed by name, and the number of fields displayed can be increased or decreased according to business needs. Row data is displayed in a paginated manner, with each row representing one document data. Each data page can display 10, 20, 50, or 100 rows of data, which can be flexibly switched as needed.
[0060] When the server queries document data in a preset data page of the database based on query filtering conditions, it asynchronously retrieves and caches document identification information corresponding to multiple data pages. This preset data page is the first data page the server queries in the database based on the query filtering conditions; that is, the first data page retrieved from this preset data page. While the server synchronously queries document data in the preset data page using the first component, it can use another component to asynchronously query document identification information corresponding to document data in multiple data pages of the database. Asynchronous queries avoid impacting the performance of synchronous queries. When asynchronously querying document identification information in the database, a preset number of data pages can be queried, such as querying pages 1 to 10, or pages 2 to 15; the specific query is not limited here.
[0061] 103. Obtain the target page number for data pagination.
[0062] This application's embodiments are primarily applied to pagination queries of document data in a database. This pagination query allows users to navigate to the next page or jump to the desired page number. The target page number can be obtained, for example, by the server receiving the target page number sent by the client. This target page number is typically the page number of the data page containing the document data selected by the user. For instance, if a user wants to retrieve the seventh page of the target document data, they can enter the page number seven on the client, causing the client to send page number seven to the server.
[0063] 104. Determine the target document identifier information corresponding to the target page number from the cached document identifier information.
[0064] After receiving the target page number, the server can determine the target document identifier information corresponding to the target page number from the cached document identifier information. That is, it searches for the data page corresponding to the target page number from the cached data pagination document identifier information and uses the document identifier information corresponding to that data page as the target document identifier information. It's understandable that the document identifier information is cached on the server, and querying using the cache is more efficient than querying from the database.
[0065] 105. Query document data in the database based on the target document identification information.
[0066] The server queries document data in the database based on the target document identifier information. For example, if the target document identifier information is the document primary key id (id1, id2, d3, ..., idn), the server queries the document data in the database using the primary key index. This eliminates the need to query the document data page by page from the database. The server only needs to retrieve the document data corresponding to the target document identifier information from the cache, without caching too much document data in the database, which can effectively improve query performance.
[0067] As can be seen, in this embodiment, a document query request is obtained, which includes query filtering conditions. When querying document data in the database's preset data pages based on the query filtering conditions, document identification information corresponding to multiple data pages is asynchronously obtained and cached. The target page number of the data page is obtained. The target document identification information corresponding to the target page number is determined from the cached document identification information. Document data in the database is then queried based on the target document identification information. By determining the target document identification information from the cached document identification information and querying the document data corresponding to the target document identification information from the database, without needing to query too much document data from the database, memory consumption during data querying is effectively reduced, and query performance is improved.
[0068] Furthermore, this document data query method can be applied to a server. In this embodiment, the server may include a synchronous server and an asynchronous server. Asynchronous querying of document identification information can be performed through an asynchronous server, such as... Figure 2 As shown, the specific steps include the following:
[0069] 201. The synchronization server receives document query requests sent by the client.
[0070] In this embodiment of the application, the synchronization server can receive document query requests sent by the client. It is understood that when the user triggers a document query request on the client, he / she will input multiple query conditions, such as the date being August and the amount being greater than a preset amount value. The synchronization server can receive multiple query conditions sent by the client based on the document query request, and then assemble the multiple query conditions into query filter conditions so that the queried document data simultaneously meets the multiple query conditions.
[0071] 202. The synchronization server performs the first pagination query of document data in the database based on the document query request.
[0072] After receiving a document query request from the client, the synchronization server can query document data in the database based on the initial page selection. Specifically, after receiving the query filter conditions, the synchronization server retrieves the document data from the first data page in the database, where the first data page is the first page queried in the database; this first data page can be a page with page number 1. The synchronization server uses the query filter conditions to query the document data in the first data page and returns the query results to the client. In other words, after querying a small amount of document data from the first data page, the synchronization server returns the document data that meets the query filter conditions to the client for display.
[0073] 203. The synchronous server sends asynchronous messages to the asynchronous server.
[0074] In this process, the synchronous server simultaneously queries the document data of the first data page and sends an asynchronous message to the asynchronous server. The sending and receiving of this asynchronous message are independent; the sender does not need to wait for a response from the receiver, thus achieving asynchronous decoupling. The synchronous server can send asynchronous messages to the asynchronous server through an asynchronous message middleware. This mechanism of sending asynchronous messages during synchronous data queries avoids impacting the efficiency of synchronous queries. This asynchronous message is used to indicate the document identification information for multiple data pages in the database.
[0075] 204. The asynchronous server queries and caches the document identification information for multiple data pages in the database.
[0076] The asynchronous server receives asynchronous messages sent by the synchronous server when querying document data for the first data page. Based on these messages, it queries and caches document identification information for multiple data pages in the database, and then sends the cached document identification information to the synchronous server. The number of data pages can be configured according to business needs; that is, the number of cached document identification information can be set to 100,000, 200,000, or 500,000, but is not limited here.
[0077] 205. The synchronous server receives and caches the document identification information for multiple data pages sent by the asynchronous server.
[0078] The synchronous server receives and caches document identification information for multiple data pages sent by the asynchronous server. Specifically, while synchronously querying the document data for the first data page, the synchronous server simultaneously sends asynchronous messages to query and cache document identification information for multiple data pages. The sorting rule for the document identification information can be determined based on the page number. Based on this sorting rule, the document identification information corresponding to multiple data pages is cached in the caching middleware. In other words, document identification information belonging to the same data page is cached in the same cache unit, and the cache is sorted according to the page number. This caching middleware is a high-performance cache middleware that supports high concurrency, such as Redis.
[0079] 207. The synchronization server receives the target page number sent by the client.
[0080] When performing a pagination query, users can input the target page number corresponding to the desired data page on the client side. The synchronization server can receive the target page number sent by the client. The synchronization server can retrieve the pagination display data corresponding to the data page in the database. This pagination display data is used to display the document data contained in each data page, and can be a document data display list. The server sends the pagination display data to the client and receives the target page number determined by the client based on the pagination display data. It can be understood that after receiving the pagination display data, the client can display the corresponding pagination display list, allowing users to intuitively see the document data contained in the data page in the database. The user determines the target page number of the data page containing the document data to be queried and sends the target page number to the synchronization server through the client. It can be understood that this target page number can be one or more; for example, a user can specify to query the fifth and seventh pages of the data page. Specific limitations are not specified here.
[0081] 208. The synchronization server determines the target document identifier information corresponding to the target page number from the cached document identifier information.
[0082] After receiving the target page number sent by the client, the synchronization server determines the target document identifier information corresponding to the target page number from the cached document identifier information. This document identifier information is cached in a cache middleware, meaning the target document identifier information corresponding to the target page number can be retrieved from the cache middleware. The synchronization server can also determine the target data pagination corresponding to the target page number and then determine the target document identifier information corresponding to the target data pagination from the cached document identifier information.
[0083] 209. The synchronization server queries the document data in the database based on the target document identification information.
[0084] After the synchronization server determines the target document identification information corresponding to the target page number, it can query the document data in the database based on the target document identification information. For example, it can query a specified number of actual document data from the database based on the document primary key ID, and use the primary key index to improve query performance.
[0085] 210. The synchronization server returns the retrieved document data to the client.
[0086] As can be seen, when a user flips through pages in the client, the system retrieves the target document identifier information for the specified target page number from the cache middleware, making full use of the cache to improve performance. Then, it retrieves the document data based on the target document identifier information, making full use of the primary key index to improve query performance.
[0087] This application also provides a document data query system, such as... Figure 3 As shown, it includes:
[0088] The first acquisition unit 301 is used to acquire a data query request, the data query request including query filtering conditions;
[0089] The caching unit 302 is used to obtain and cache document identification information corresponding to multiple data pages when querying document data in the preset data pages of the database based on the query filtering conditions; wherein, the database contains multiple data pages, and each data page contains multiple document data and document identification information corresponding to the document data;
[0090] The second acquisition unit 303 is used to acquire the target page number for data pagination;
[0091] The determining unit 304 is used to determine the target document identification information corresponding to the target page number from the cached document identification information;
[0092] The query unit 305 is used to query document data in the database based on the target document identification information.
[0093] Please see Figure 4The electronic device 400 in this application embodiment may include one or more central processing units (CPUs) 401 and a memory 402, wherein the memory 402 stores one or more applications or data.
[0094] The memory 402 can be volatile or persistent storage. The program stored in the memory 402 can include one or more modules, each module including a series of instruction operations on the electronic device. Furthermore, the central processing unit 401 can be configured to communicate with the memory 402 and execute the series of instruction operations stored in the memory 402 on the electronic device 400.
[0095] Electronic device 400 may also include one or more power supplies 405, one or more wired or wireless network interfaces 404, one or more input / output interfaces 403, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.
[0096] The central processing unit 401 can perform the operations performed by the first aspect or any specific method embodiment of the first aspect, which will not be described in detail here.
[0097] This application provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect or any specific implementation thereof.
[0098] This application provides a computer program product containing instructions or computer programs, which, when run on a computer, causes the computer to perform the method described in the first aspect or any specific implementation thereof.
[0099] It is understood that, in the various embodiments of this application, the sequence number of each step does not imply the order of execution. The execution order of each step should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0100] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0101] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system or apparatus, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product (computer program product) is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a business server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A method for querying document data, characterized in that, include: Obtain a document query request, wherein the document query request includes query filtering conditions; When querying document data in a preset data page of the database based on the query filtering conditions, the document identification information corresponding to multiple data pages in the database is asynchronously obtained and cached; wherein, the database contains multiple data pages, and each data page contains multiple document data and document identification information corresponding to the document data; The method is applied to a server, which includes a synchronous server and an asynchronous server. The step of querying document data in a preset data page of the database based on the query filter conditions includes: after receiving the query filter conditions, the synchronous server obtains the document data of the first data page in the database, wherein the first data page is the first data page queried in the database; the synchronous server uses the query filter conditions to query the document data of the first data page and returns the query results to the client. The asynchronous acquisition and caching of document identification information corresponding to multiple data pages includes: the asynchronous server receiving an asynchronous message sent by the synchronous server when querying document data of the first data page; the asynchronous server querying and caching document identification information of multiple data pages in the database based on the asynchronous message, and sending the cached document identification information to the synchronous server; Get the target page number for data pagination; Determine the target document identifier information corresponding to the target page number from the cached document identifier information; Based on the target document identification information, query the document data in the database.
2. The query method according to claim 1, characterized in that, The document retrieval request includes: Retrieve multiple query conditions; The multiple query conditions are assembled into the query filter conditions.
3. The query method according to claim 1, characterized in that, The document identification information corresponding to the cached multiple data pages includes: The sorting rules for document identification information are determined based on the page numbers of the data pagination. Based on the sorting rules, the document identification information corresponding to multiple data pages is cached in the cache middleware.
4. The query method according to claim 1, characterized in that, The target page number for data pagination includes: Retrieve the pagination display data corresponding to the data pagination in the database; The pagination display data is sent to the client, and the target page number for data pagination determined by the client based on the pagination display data is received.
5. The query method according to claim 1, characterized in that, The step of determining the target document identifier information corresponding to the target page number from the cached document identifier information includes: Determine the target data pagination corresponding to the target page number; The target document identifier information corresponding to the target data pagination is determined from the cached document identifier information.
6. A document data query system, characterized in that, include: The first acquisition unit is used to acquire a data query request, wherein the data query request includes query filtering conditions; A caching unit is used to retrieve and cache document identification information corresponding to multiple data pages in the database when querying document data in a preset data page of the database based on the query filtering conditions; wherein, the database contains multiple data pages, each data page contains multiple document data and document identification information corresponding to the document data; the preset data page is the data page that is first queried in the database based on the query filtering conditions; The document data query system is applied to a server, which includes a synchronous server and an asynchronous server. After receiving the query filter conditions, the synchronous server retrieves document data from the first data page in the database, where the first data page is the first data page queried in the database. The synchronous server uses the query filter conditions to query the document data from the first data page and sends the query results back to the client. The asynchronous server receives an asynchronous message sent by the synchronous server when querying the document data from the first data page. Based on the asynchronous message, the asynchronous server queries and caches document identification information from multiple data pages in the database and sends the cached document identification information to the synchronous server. The second acquisition unit is used to acquire the target page number for data pagination; The determining unit is used to determine the target document identification information corresponding to the target page number from the cached document identification information; The query unit is used to query document data in the database based on the target document identification information.
7. An electronic device, characterized in that, include: Central processing unit, memory, and input / output interfaces; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 5.