Index-based data query method, system and equipment
By creating index paths and index data files in the data storage file and cacheing these index data on the data nodes, the problems of low data query efficiency and reduced performance in the prior art are solved, and efficient data query in high concurrency and large data volume scenarios are achieved.
Patent Information
- Application Number
- CN202411776894.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-04-01
AI Technical Summary
In the prior art, data query methods based on traditional or distributed databases have problems such as centralization of index data structures, low query efficiency, and performance reduction caused by the growth of data volume, resulting in excessive server resource usage and extended response time, which affects the user experience of end users.
An index-based data query method is adopted to create index paths and index data files when writing data in data storage files, and cache these index data files into the index cache of each data node. When the user issues a query request, the management node forwards the request to multiple data nodes in parallel. The data node retrieves the index data from the index cache in parallel and reads the stored data from the data storage file, and finally returns the result through the management node.
This method can maintain the balance of search efficiency under high concurrency, avoid the reduction of query efficiency caused by the increase in data volume, ensure server performance and data error tolerance, and avoid overall search function failure when index files are corrupted, improving system stability and query efficiency.
Smart Images
Figure CN119739743A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of data query technology, and in particular, relates to an index-based data query method, system and device. Background Art
[0002] In the era of rapid development of aviation business, the volume of business requests is also gradually increasing. In the case of large-scale requests, historical request information needs to be retrieved and queried, requiring the server to search massive data, resulting in a gradual increase in the demand for server disk retrieval. Long-term disk retrieval and patrol inspections cause great damage to the hard disk, and there is a risk of disk damage and data loss. In order to ensure that the server can operate stably and can carry a large number of requests, the service needs to be guaranteed, and the retrieval information of different businesses is classified to customize the optimal retrieval route, avoid server unavailability, and ensure continuous response services to end users. Currently, there are query services for multiple businesses in the ticket query system. Airlines will use different query methods in different access businesses to ensure query restrictions and response efficiency, while also ensuring the service capabilities of the overall cluster.
[0003] The query in the prior art is based on the third-party storage service, which mainly relies on the storage server to allocate index retrieval. One is based on the traditional database query. Figure 1 This is an example diagram of querying based on a traditional database in the prior art; the other is querying based on a distributed database. Figure 2 This is an example diagram of querying based on a distributed database in the prior art.
[0004] The query based on the traditional database adopts the single-machine storage method. The retrieval index is processed on a single machine and stores the index data similar to the Btree data structure. Each request is searched in memory in the data structure. After finding the matching information, the data is searched according to the data location provided in the data structure. The query based on the distributed database also stores the index file data on a single node, which cannot achieve the parallel computing capability.
[0005] In actual applications, the above query method has the following shortcomings: 1. The index data structure is relatively centralized, and each request requires a full index search of the server. If the index file is damaged, the server will perform a full table query, resulting in reduced overall retrieval performance, resulting in excessive server resource usage, extended response time, and excessive disk and CPU computing waits, which reduces the overall server performance and query efficiency; 2. The query efficiency is proportional to the amount of data, that is, the more data, the lower the query efficiency and the more resources are occupied. As the order of magnitude increases, the query efficiency seriously affects the end-user experience.
[0006] Therefore, how to provide a more efficient and accurate index query method has become a technical problem that needs to be solved urgently. Summary of the invention
[0007] In response to the above problems, the present application provides an index-based data query method, system and device.
[0008] According to one aspect of the present application, a method for querying data based on an index is provided, the method comprising:
[0009] When writing storage data into the data storage file, an index path is created according to the written storage data, and a corresponding index data file is created according to the created index path;
[0010] Writing index data into the index data file according to the write information of the storage data, and caching the index data file in which the index data is written into the index cache of each data node;
[0011] When a user issues a query request, the management node collects the user's query request and forwards the collected query request to multiple data nodes in parallel;
[0012] Multiple data nodes retrieve index data from corresponding index caches in parallel according to received query requests, read storage data from data storage files according to the retrieved index data, and return the read storage data to the user through the management node.
[0013] Optionally, in the index-based data query method of the present application, the storage data written into the data storage file includes time information, primary key information and data content information.
[0014] Optionally, in the index-based data query method of the present application, an index path is created based on the written storage data, and a corresponding index data file is created based on the created index path, including: creating an index path based on the time information of the storage data, and creating a corresponding time folder and time file based on the created index path.
[0015] Optionally, in the index-based data query method of the present application, the time folder includes a year folder, a month folder and a day folder, the year folder, the month folder and the day folder are used to distinguish the location of the index data, and the time file is used to store the index data.
[0016] Optionally, in the index-based data query method of the present application, index data is written into an index data file according to the write information of the storage data, including: after writing the storage data into the data storage file, obtaining the position offset and data length of the storage data write, and writing the time information, position offset, data length and primary key information of the storage data as index data in the form of row data into the time file in the index data file.
[0017] Optionally, in the index-based data query method of the present application, when a user issues a query request, the user's query request is collected through the management node, and the collected query requests are forwarded in parallel to multiple data nodes, including: when the management node is unavailable, selecting a data node as a proxy node to collect the user's query request, and forwarding the collected query requests in parallel to the remaining data nodes.
[0018] Optionally, in the index-based data query method of the present application, multiple data nodes retrieve index data from the corresponding index cache in parallel according to the received query request, including: the data node parses the received query request, obtains the time information and primary key information of the query request, searches for the corresponding time folder and time file from the index data file in the corresponding index cache according to the time information of the query request, compares the primary key information of the query request with the primary key information of the storage data written by the index data in the time file, and when the primary key information of the query request is consistent with the primary key information of the storage data written by the index data in the time file, obtains the position offset and data length written in the index data of that row.
[0019] Optionally, in the index-based data query method of the present application, storage data is read from a data storage file according to the retrieved index data, including: intercepting file data in the data storage file according to the acquired position offset and data length, and returning the intercepted file data to the user through the management node.
[0020] According to a second aspect of the present application, there is provided an index-based data query system, the system comprising a data query server, the data query server comprising:
[0021] An index data file creation module is used to create an index path according to the written storage data when writing storage data into the data storage file, and to create a corresponding index data file according to the created index path;
[0022] An index data writing module is used to write index data into an index data file according to the writing information of the storage data, and cache the index data file into which the index data is written into the index cache of each data node;
[0023] The request collection and response module is used to collect the user's query request through the management node when the user issues a query request, forward the collected query request to multiple data nodes in parallel, and return the read storage data to the user through the management node;
[0024] The data query module includes multiple data nodes and multiple corresponding index caches. The data nodes are used to retrieve index data from the corresponding index caches in parallel according to the received query requests, and read storage data from the data storage file according to the retrieved index data. The index cache is used to store the index data file in which the index data is written.
[0025] According to a third aspect of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect of the present application when executing the program.
[0026] The index-based data query method and system of the present application have the following beneficial technical effects:
[0027] 1. It can maintain the balance of retrieval efficiency. In the case of high concurrency, it can avoid reducing the query efficiency of the server due to the increase of data volume. It can ensure the fault tolerance of data while ensuring that the performance of the server is not affected. The damage of the index file will not affect the overall retrieval function.
[0028] 2. By reducing the performance loss of queries, optimizing performance, avoiding burden on services, avoiding normal requests that affect traffic, ensuring the minimum consumption time, ensuring the normal TPS (Transactions Per Second) provision capacity of services, and avoiding the reduction of service TPS due to consumption of flow limiting;
[0029] 3. Reduce the construction dependency of the query system, avoid the avalanche effect caused by traffic cache server failure, improve the stability of the server, and enhance the independent operation capability of the server;
[0030] 4. Improve the flow calculation efficiency of the file query system, shorten the flow calculation time, improve the efficiency and accuracy of the query, reduce the resource waste caused by CPU consumption, and ensure the concurrency capability of the server.
[0031] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures indicated in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0033] Figure 1 This is an example diagram of querying based on a traditional database in the prior art;
[0034] Figure 2 This is an example diagram of querying based on a distributed database in the prior art;
[0035] Figure 3 This is an example diagram of the architecture of an index-based data query system according to an embodiment of the present application;
[0036] Figure 4 This is an example diagram of the architecture of a data query server of an index-based data query system according to an embodiment of the present application;
[0037] Figure 5 A flowchart of a method for index-based data query according to an embodiment of the present application;
[0038] Figure 6 This is a structural example diagram of an index data file in an index-based data query method according to an embodiment of the present application;
[0039] Figure 7 An example diagram of performing data query according to the index-based data query method according to an embodiment of the present application;
[0040] Figure 8 A schematic diagram of the structure of the device provided in this application. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0042] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.
[0043] Figure 3 FIG. 1 is an example diagram of an architecture of an index-based data query system according to an embodiment of the present application. Figure 3 As shown, the system may include a data query server 101, a communication network 102 and / or one or more data query clients 103. Figure 3 The example shown is a plurality of data query clients 103 .
[0044] The data query server 101 can be any appropriate server for storing information, data, programs and / or any other suitable type of content. In some embodiments, the data query server 101 can perform appropriate functions. For example, in some embodiments, the data query server 101 can be used for index-based data query. As an optional example, in some embodiments, the data query server 101 can be used to: when writing storage data to a data storage file, create an index path according to the written storage data, and create a corresponding index data file according to the created index path; write index data in the index data file according to the write information of the storage data, and cache the index data file with the written index data into the index cache of each data node; when a user issues a query request, collect the user's query request through the management node, and forward the collected query request in parallel to multiple data nodes; multiple data nodes retrieve index data from the corresponding index cache in parallel according to the received query request, read storage data from the data storage file according to the retrieved index data, and return the read storage data to the user through the management node.
[0045] As another example, in some embodiments, the data query server 101 may send an index-based data query method to the data query client 103 for the user to use according to a request of the data query client 103 .
[0046] Figure 4 FIG. 1 is an example diagram of the architecture of a data query server of an index-based data query system according to an embodiment of the present application, such as Figure 4As shown, the data query server of this embodiment includes:
[0047] The index data file creation module is used to create an index path according to the written storage data when writing storage data into the data storage file, and to create a corresponding index data file according to the created index path; the index data file of this embodiment stores all data, and in actual applications, when the storage data is written, the corresponding index data is generated, and the generated index data is written into the index data file, and the index cache is refreshed. This is convenient for refreshing the file location of the data loading to the index cache when the server is restarted.
[0048] An index data writing module is used to write index data into an index data file according to the writing information of the storage data, and cache the index data file into which the index data is written into the index cache of each data node;
[0049] The request collection and response module is used to collect the user's query request through the management node when the user issues a query request, forward the collected query request to multiple data nodes in parallel, and return the read storage data to the user through the management node;
[0050] The data query module includes multiple data nodes and multiple corresponding index caches. The data nodes are used to retrieve index data from the corresponding index caches in parallel according to the received query requests, and read the storage data from the data storage file according to the retrieved index data. The index cache is used to store the index data file written with the index data. For example, in actual applications, the index cache of this embodiment can cache and retain the index data file. When the server starts, the index data is obtained from the index data file and loaded into the memory. When processing the query request, the matching index data is retrieved from the cache, and the write location information of all data is obtained at the same time.
[0051] As an optional example, in some embodiments, the data query client 103 is used to provide a visual data query interface, which is used to receive a user's selection input operation for index-based data query, and, in response to the selection input operation, obtain a data query interface corresponding to the option selected by the selection input operation from the data query server 101 and display the data query interface, wherein the data query interface at least displays information on index-based data query and operation options for the information on index-based data query.
[0052] In some embodiments, the communication network 102 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 102 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The data query client 103 can be connected to the communication network 102 via one or more communication links (e.g., communication link 104), and the communication network 102 can be linked to the data query service end 101 via one or more communication links (e.g., communication link 105). The communication link can be any communication link suitable for transmitting data between the data query client 103 and the data query service end 101, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link, or any suitable combination of such links.
[0053] The data query client 103 may include any one or more clients that present an interface related to index-based data query in an appropriate form for use and operation by a user. In some embodiments, the data query client 103 may include any suitable type of device. For example, in some embodiments, the data query client 103 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, and / or any other suitable type of client device.
[0054] Although the data query server 101 is illustrated as one device, in some embodiments, any suitable number of devices may be used to perform the functions performed by the data query server 101. For example, in some embodiments, multiple devices may be used to implement the functions performed by the data query server 101. Alternatively, a cloud service may be used to implement the functions of the data query server 101.
[0055] Based on the above system, an embodiment of the present application provides an index-based data query method, which is described below through the following embodiments.
[0056] Figure 5 FIG. 1 is a flowchart of a method for querying data based on an index according to an embodiment of the present application. The method for querying data based on an index in this embodiment can be executed on a data query server, such as Figure 5 As shown, the index-based data query method includes the following steps:
[0057] Step S201: when writing storage data into a data storage file, an index path is created according to the written storage data, and a corresponding index data file is created according to the created index path.
[0058] As an optional example, in this embodiment, the storage data written into the data storage file includes time information, primary key information and data content information. This embodiment creates an index path according to the time information of the storage data, and creates a corresponding time folder and time file according to the created index path. Figure 6 FIG. 1 is a structural example diagram of an index data file in an index-based data query method according to an embodiment of the present application, such as Figure 6 As shown, the time folder in this embodiment includes a year folder, a month folder and a day folder, which are used to distinguish the location of index data to facilitate finding the location of index data, and the time file is used to store index data.
[0059] Step S202: writing index data into an index data file according to the writing information of the storage data, and caching the index data file into which the index data is written into the index cache of each data node.
[0060] As an optional example, after writing the storage data into the data storage file, this embodiment obtains the position offset and data length of the storage data, and writes the time information, position offset, data length and primary key information of the storage data into the time file of the index data file in the form of row data as index data. For example, the index data of this embodiment includes a primary key and a file address, wherein the primary key is the unique primary key information of the storage data, and the file address indicates the storage location of the storage data, including the position offset and data length.
[0061] Step S203: When a user issues a query request, the user's query request is collected through the management node, and the collected query requests are forwarded to multiple data nodes in parallel.
[0062] Figure 7 An example diagram of performing data query based on the index-based data query method according to an embodiment of the present application is shown in FIG. Figure 7As shown, in order to solve the stress resistance of the service and ensure the performance of data query, this embodiment allows multiple storage data nodes to participate in the calculation at the same time through the planning of storage data, and parallelizes the query request processing to reduce the computing pressure of the management node or proxy node, so that the management node or proxy node only makes simple forwarding requests and data responses, and the data node actually processes the real query operation. The data node knows the storage situation of the data best, and the query is more direct and simplified. The pressure is all handed over to the real data storage node, so that the cluster can process large quantities of requests more evenly, ensuring that the entire cluster can provide stable service capabilities and business management and control capabilities. It should be noted that when the management node is unavailable, this embodiment selects a data node as a proxy node to collect the user's query request, and forwards the collected query request to the remaining data nodes in parallel. In actual applications, this embodiment eliminates the computational consumption of each request of the data management node, uses the data node address stored in the memory, and directly forwards the request to the data node. The data node replaces the management node's calculation, avoiding the resource consumption problem caused by the excessive number of requests of the management node. At the same time, the establishment of the index is also executed and managed by the data node, without the participation of the management node. It only needs to load the data obtained from the data node address into the memory, thereby significantly improving the query performance.
[0063] In actual applications, the query request flow is huge, and the request must be calculated multiple times in the machine, which will only lock the computing resources in the same machine. Each time a request is made, the CPU needs to access the physical memory, go through the bus, bus competition, lock competition, and load the cache. Different systems and different hardware environments will have different consumption points. This embodiment uses parallel index query to add all data nodes to participate in calculation and query together. The data node is not only a storage node, but also a query node, and it is the query closest to the data storage location, making the query more refined.
[0064] Step S204: the multiple data nodes retrieve index data from the corresponding index cache in parallel according to the received query request, read the storage data from the data storage file according to the retrieved index data, and return the read storage data to the user through the management node.
[0065] As an optional example, the data node of this embodiment parses the received query request, obtains the time information and primary key information of the query request, searches for the corresponding time folder and time file from the index data file in the corresponding index cache according to the time information of the query request, compares the primary key information of the query request with the primary key information of the storage data written in the index data in the time file, and when the primary key information of the query request is consistent with the primary key information of the storage data written in the index data in the time file, obtains the position offset and data length written in the row of index data. File data is intercepted in the data storage file according to the obtained position offset and data length, and the intercepted file data is returned to the user through the management node.
[0066] The index-based data query method and system of this embodiment are described in detail below in a specific scenario.
[0067] When writing stored data, create an index file and write data:
[0068] {Date: 2024-10-01 22:30:30, Primary key: 3425436534, Content: xxxxxxxxxxxx}
[0069] At this time, three corresponding folder paths will be created: folder 2024, folder 10, folder 01; at the same time, a file will be created: file 22.txt.
[0070] After writing the specific data content, obtain the position offset and data length of the specific data written, and write the following sample in the index file 22.txt:
[0071] Mi n:2230,l ocat ion:0,l ength:54,id:3425436534
[0072] This is a row of data. When there is a second piece of data, write it in a new row to avoid data confusion. Min represents 22:30, location is the starting position of the file, length is the content length (54 in the same example), and id is the primary key information.
[0073] When initiating a query request, the requested data sample is:
[0074] {Start date: 2024-10-01 22:00:30, End date: 2024-10-01 23:40:30, Primary key: 3425436534}
[0075] Now search for folders by date content:
[0076] Folder 2024, Folder 10, Folder 01;
[0077] Search for hourly files. If multiple hourly files exist, the following are searched: file 22.txt. After checking, file 23.txt does not exist. Only file 22.txt is searched.
[0078] Use the primary key 3425436534 to compare the file line by line to determine which row of data it is. After a successful comparison, retrieve the index data of that row and return the location and length data information. Then, in the corresponding file storing the data, intercept the file content according to the retrieved location and length data information. The data obtained is the specific data content information found through the index.
[0079] The index-based data query method and system of this embodiment have the following beneficial technical effects:
[0080] 1. It can maintain the balance of retrieval efficiency. In the case of high concurrency, it can avoid reducing the query efficiency of the server due to the increase of data volume. It can ensure the fault tolerance of data while ensuring that the performance of the server is not affected. The damage of the index file will not affect the overall retrieval function.
[0081] 2. By reducing the performance loss of queries, optimizing performance, avoiding burden on services, avoiding normal requests that affect traffic, ensuring the minimum consumption time, ensuring the normal TPS (Transactions Per Second) provision capacity of services, and avoiding the reduction of service TPS due to consumption of flow limiting;
[0082] 3. Reduce the construction dependency of the query system, avoid the avalanche effect caused by traffic cache server failure, improve the stability of the server, and enhance the independent operation capability of the server;
[0083] 4. Improve the flow calculation efficiency of the file query system, shorten the flow calculation time, improve the efficiency and accuracy of the query, reduce the resource waste caused by CPU consumption, and ensure the concurrency capability of the server.
[0084] like Figure 8 As shown, the present application also provides a device, including a processor 310, a communication interface 320, a memory 330 for storing a processor executable computer program, and a communication bus 340. The processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 implements the above-mentioned index-based data query method by running the executable computer program.
[0085] Among them, the computer program in the memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0086] The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected based on actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.
[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.
[0088] Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A data query method based on index, characterized in that: The method comprises: When writing storage data into the data storage file, an index path is created according to the written storage data, and a corresponding index data file is created according to the created index path; Writing index data into the index data file according to the write information of the storage data, and caching the index data file in which the index data is written into the index cache of each data node; When a user issues a query request, the management node collects the user's query request and forwards the collected query request to multiple data nodes in parallel; Multiple data nodes retrieve index data from corresponding index caches in parallel according to received query requests, read storage data from data storage files according to the retrieved index data, and return the read storage data to the user through the management node.
2. The index-based data query method according to claim 1, characterized in that: The storage data written into the data storage file includes time information, primary key information and data content information.
3. The index-based data query method according to claim 2, characterized in that: Creating an index path according to the written storage data, and creating a corresponding index data file according to the created index path, including: creating an index path according to the time information of the storage data, and creating a corresponding time folder and time file according to the created index path.
4. The index-based data query method according to claim 3, characterized in that: The time folder includes a year folder, a month folder and a day folder, wherein the year folder, the month folder and the day folder are used to distinguish the location of the index data, and the time file is used to store the index data.
5. The index-based data query method according to claim 1, characterized in that: Writing index data into the index data file according to the writing information of the storage data includes: after writing the storage data into the data storage file, obtaining the position offset and data length of the storage data written, and writing the time information, position offset, data length and primary key information of the storage data as index data into the time file in the index data file in the form of row data.
6. The index-based data query method according to claim 1, characterized in that: When a user issues a query request, the user's query request is collected through the management node, and the collected query requests are forwarded in parallel to multiple data nodes, including: when the management node is unavailable, a data node is selected as a proxy node to collect the user's query request, and the collected query requests are forwarded in parallel to the remaining data nodes.
7. The index-based data query method according to claim 1, characterized in that: Multiple data nodes retrieve index data from the corresponding index cache in parallel according to the received query requests, including: the data node parses the received query request, obtains the time information and primary key information of the query request, searches for the corresponding time folder and time file from the index data file in the corresponding index cache according to the time information of the query request, compares the primary key information of the query request with the primary key information of the storage data written by the index data in the time file, and when the primary key information of the query request is consistent with the primary key information of the storage data written by the index data in the time file, obtains the position offset and data length written in the index data of that row.
8. The index-based data query method according to claim 7, characterized in that: Reading storage data from the data storage file according to the retrieved index data includes: intercepting file data in the data storage file according to the acquired position offset and data length, and returning the intercepted file data to the user through the management node.
9. An index-based data query system, characterized in that: The system includes a data query server, and the data query server includes: An index data file creation module is used to create an index path according to the written storage data when writing storage data into the data storage file, and to create a corresponding index data file according to the created index path; An index data writing module is used to write index data into an index data file according to the writing information of the storage data, and cache the index data file into which the index data is written into the index cache of each data node; The request collection and response module is used to collect the user's query request through the management node when the user issues a query request, forward the collected query request to multiple data nodes in parallel, and return the read storage data to the user through the management node; The data query module includes multiple data nodes and multiple corresponding index caches. The data nodes are used to retrieve index data from the corresponding index caches in parallel according to the received query requests, and read storage data from the data storage file according to the retrieved index data. The index cache is used to store the index data file in which the index data is written.
10. A computer device, characterized in that: The computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 8 when executing the program.