Data fast query method and device, electronic equipment and storage medium
By obtaining the write information of time-series data, predicting the backtracking interval and the number of queries, and optimizing the query process, the problems of low query efficiency and resource waste in time-series databases are solved, thereby improving query efficiency and reducing resource consumption.
Patent Information
- Application Number
- CN202310240060.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-06
- Publication Date
- 2026-02-27
- Estimated Expiration
- 2043-03-06
AI Technical Summary
Existing time-series databases suffer from low query efficiency and significant resource waste when querying the most recent N time-series data points up to a specified cutoff time, failing to effectively utilize optimizations based on backtracking intervals and query counts.
By obtaining the write information of time-series data, the backtracking interval and number of queries can be predicted, the query process can be optimized, the acquisition of useless data can be reduced, and the query efficiency can be improved.
This improves query efficiency and reduces waste of computing and memory resources when querying the N most recent time-series data at a specified time.
Smart Images

Figure CN116226234B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present disclosure relates to the technical field of data processing, and particularly relates to a data fast query method and device, electronic equipment and storage medium. BACKGROUND
[0002] In the industrial Internet of Things scene, various states of devices in the production process need to be monitored, such as temperature, torque, rotating speed and the like, and the time series data monitored at each moment needs to be recorded, which is referred to as time series data. Time series data is a series of data based on time, and multi-dimension reports can be made based on the historical data to reveal the trend, regularity and abnormality thereof; big data analysis, machine learning, prediction and early warning can also be implemented. At present, a time series database (OpenTSDB) is used as a database for storing time series data, which supports basic functions such as fast writing, persistence and multi-dimension aggregation query of time series data.
[0003] When querying the N latest time series data at a specified time, the upper application can only read the original time series data in batches according to a fixed time interval (such as 1 hour), and find the N data by backtracking in batches in a time reverse order, which is low in query efficiency and causes waste of computing resources and memory resources. SUMMARY
[0004] The present disclosure provides a data fast query method, device, electronic equipment and storage medium to at least solve the above technical problems in the prior art.
[0005] According to a first aspect of the present disclosure, a data fast query method is provided, comprising:
[0006] In response to writing first time series data into a database, write information corresponding to at least one time series data is acquired;
[0007] In response to receiving a query request, a prediction backtracking interval and a prediction query number are confirmed based on write information corresponding to the at least one time series data and query information corresponding to the query request;
[0008] The query request comprises acquiring a first data quantity of time series data in a first time interval.
[0009] According to a second aspect of the present disclosure, a data fast query device is provided, comprising:
[0010] An acquisition unit is configured to, in response to writing first time series data into a database, acquire write information corresponding to at least one time series data;
[0011] The confirmation unit is configured to, in response to receiving a query request, confirm a predicted backtracking interval and a predicted query number based on write information corresponding to the at least one piece of time-series data and query information corresponding to the query request.
[0012] The query request comprises time-series data of a first data quantity in a first time interval.
[0013] According to a third aspect of the present disclosure, an electronic device is provided, comprising:
[0014] at least one processor; and
[0015] a memory in communication with the at least one processor; wherein
[0016] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.
[0017] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, the computer instructions being used to cause the computer to perform the method of the present disclosure.
[0018] The data fast query method of the present disclosure, in response to writing first time-series data into a database, obtains write information corresponding to at least one piece of time-series data; in response to receiving a query request, confirms a predicted backtracking interval and a predicted query number based on write information corresponding to the at least one piece of time-series data and query information corresponding to the query request; wherein the query request comprises time-series data of a first data quantity in a first time interval; in this way, when querying the most recent N pieces of time-series data at a specified time, the predicted backtracking interval and the predicted query number can be confirmed based on the write information of the time-series data written into the database and the corresponding query information; and the time-series data at the specified time is backtracked based on the predicted backtracking interval and the predicted query number, and the most recent N pieces of time-series data are queried, thereby improving the query efficiency and reducing the waste of computing resources and memory resources.
[0019] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS
[0020] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:
[0021] The same or corresponding reference signs in the drawings indicate the same or corresponding parts.
[0022] Figure 1 An optional flowchart of a data fast query method provided by an embodiment of the present disclosure is shown.
[0023] Figure 2 Another optional flowchart of a data fast query method provided by an embodiment of the present disclosure is shown.
[0024] Figure 3 Still another optional flowchart of a data fast query method provided by an embodiment of the present disclosure is shown.
[0025] Figure 4 Still another optional flowchart of a data fast query method provided by an embodiment of the present disclosure is shown.
[0026] Figure 5 A data flow diagram of a data fast query method provided by an embodiment of the present disclosure is shown.
[0027] Figure 6 An optional structural diagram of a data fast query apparatus provided by an embodiment of the present disclosure is shown.
[0028] Figure 7 A structural diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0029] To make the objectives, features and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.
[0030] In related technologies, an application program interface (API) provided by a database can specify a time interval (start time and end time) to query historical time series data details of a certain point (a point on a device that generates time series data, referred to as a point). In actual data analysis business, there is a demand for an upper-layer application program to query the latest N pieces of time series data with a specified cutoff time (such as the latest N pieces of time series data up to XX year XX month XX day XX minute XX second). However, the mainstream time series data platform can only provide time series data corresponding to the time interval from the start time to the end time, and cannot support the query of N pieces of time series data with a specified cutoff time.
[0031] According to the current database capability, the upper application can only read the original time series data in batches according to a fixed time interval (for example, 1 hour), and find the N time series data in batches according to the time in reverse order, for example, to find the N time series data up to 18:00. The upper application can find the time series data from 17:00 to 18:00. If the number of time series data is more than N, the latest N time series data is sorted according to time; if the number of time series data is less than N, the upper application continues to find the time series data from 16:00 to 17:00, until the N time series data up to 18:00 is found, and the search is completed.
[0032] The interval of point data (time series data) generated by different devices will be very different according to the business, and the "backtracking interval" cannot be determined in advance. The backtracking interval and the number of queries (backtracking times) will affect the query efficiency. Table 1 shows the query performance and the proportion of useless time series data obtained in the query when searching for the latest 10 time series data under different backtracking intervals and query times.
[0033] Table 1
[0034] Backtracking interval Query frequency Query performance Proportion of acquired useless timing data 1 hour 240 times Low (about 250-300 ms) 0% 1 day 10 times Medium (about 19-20 ms) 0% 10 days 1 time High (about 8-10 ms) 0% 20 days 1 time Higher (about 9-13 ms) 50% 30 days 1 time Higher (about 15-18 ms) 66%
[0035] As shown in the first, second and third rows of Table 1, the overall time interval of the query is 10 days, but the different backtracking intervals affect the query performance. For example, if the backtracking interval is 1 hour, the query (backtracking) needs to be performed 240 times, but if the backtracking interval is 10 days, the query needs to be performed only once. In addition, the backtracking interval of 20 days or 30 days can obtain higher query performance, but the proportion of useless time series data obtained in the query is higher.
[0036] That is, if the backtracking interval is too small, the number of queries will increase, and the query efficiency will be low. (Referring to Table 1, the query performance under different "backtracking interval" of the same point has about 30 times difference); if the backtracking interval is too large, a large amount of invalid data will be processed, which will cause waste of computing and memory resources. (According to the above example, the resource consumption caused by useless data may be higher than the optimal 50%-66% difference).
[0037] Therefore, when querying the N time series data up to a specified time in the database, a reasonable backtracking interval and query times are particularly important.
[0038] Based on this, the present disclosure provides a data fast query method, which can solve part or all of the above technical problems.
[0039] Figure 1 An optional flowchart provided by an embodiment of the present disclosure is shown, which will be described according to each step.
[0040] In response to the first time series data being written into the database, write information corresponding to at least one time series data is obtained.
[0041] In some embodiments, the data fast query device (hereinafter referred to as the device) obtains write information corresponding to at least one time series data when first time series data is written into a database (such as a time series database).
[0042] In some embodiments, the at least one time series data is time series data written into the database before the first time series data is written into the database, that is, the timestamp of the at least one time series data written into the database is earlier than the timestamp of the first time series data written into the database.
[0043] In some embodiments, the write information corresponding to the at least one time series data can include the timestamp of the at least one time series data written into the database, and the average interval between the at least one time series data. The average interval refers to the average interval between the timestamps of the time series data written into the database. Optionally, the specific number of the at least one time series data can be set according to experimental results or actual needs, such as the number and frequency of time series data generated in different stages. If the stage of the device is not considered, the write information may not be reasonable.
[0044] In other embodiments, the write information corresponding to the at least one time series data can include the percentile write information of the write interval corresponding to the at least one time series data.
[0045] In response to receiving a query request, a predicted backtracking interval and a predicted query number are confirmed based on the write information corresponding to the at least one time series data and query information corresponding to the query request.
[0046] In some embodiments, the query request can be a time series data query request, and the query information can be a first time interval and a first data number corresponding to the query request. The first time interval can be a desired query time interval, and the first data number can be a total number of time series data desired to be queried; that is, the query request can represent which time interval and how many recent time series data are desired to be queried.
[0047] In some embodiments, after the device receives the query request, the predicted backtracking interval and the predicted query number can be confirmed based on the write information corresponding to the at least one time series data and the query information corresponding to the query request.
[0048] The predicted backtracking interval represents the backtracking interval when querying, and the predicted query number represents the number of queries when querying.
[0049] In some optional embodiments, the method can further include:
[0050] In step S103, the target time series data is obtained from the database based on the predicted backtracking interval and the predicted query number.
[0051] In some embodiments, the device obtains the time series data corresponding to the query request from the database based on the predicted backtracking interval and the predicted query number.
[0052] In this way, by using the data fast query method provided by the embodiments of the present disclosure, the write information of the time series data written into the database and the corresponding query information are used to determine the predicted backtracking interval and the predicted query number when querying the N latest time series data of a specified time, and the time series data of the specified time is backtracked based on the predicted backtracking interval and the predicted query number, so that the N latest time series data are queried, the query efficiency is improved, and the resource waste caused by useless data is reduced.
[0053] Figure 2 Another optional flowchart of the data fast query method provided by the embodiments of the present disclosure is shown, which will be described according to each step.
[0054] In step S201, in response to the first time series data being written into the database, the time stamp corresponding to at least one time series data before the first time series data is obtained, and the average interval between the at least one time series data is obtained.
[0055] In some embodiments, the average interval can be determined based on the difference between the time stamps of the latest X time series data of two adjacent points (X j+1 -X j ), a sampling interval Y (which can be set according to actual needs), and the total number of points K. j+1 -X j The sampling interval is used to sample the points in the database according to a certain sampling interval Y.
[0056] In step S202, the predicted backtracking interval is determined based on the upper limit of the query number included in the query information, the current remaining query data number, and the average interval between the at least one time series data.
[0057] In some embodiments, the upper limit of the query number is the maximum value of the number of time series data in a single query, which can be set according to actual needs; the current remaining query data number is determined based on the number of time series data that has been queried and the first data number included in the query information, and if no query is performed, the current remaining query data number is the first data number.
[0058] In some embodiments, the apparatus determines the predicted backtracking interval based on a product of a minimum value between the upper limit of the number of queries M and a current remaining number of query data Ni and an average interval I between the at least one piece of time series data, i.e., min(N i ,M)*I; wherein, N i represents a remaining number of query data after the ith query, i.e., a current remaining number of query data.
[0059] In step S203, the apparatus determines the predicted number of queries based on the first number of data, the upper limit of the number of queries, and a number of query coefficient.
[0060] In some embodiments, the first number of data, i.e., N0, and the number of query coefficient are determined based on a number of time series data written into the database and the first number of data.
[0061] In some embodiments, the apparatus determines the preset number of queries based on a product of an up-round value of a quotient of the first number of data and a minimum value between the upper limit of the number of queries and the first number of data and the number of query coefficient a, i.e.,
[0062] In step S204, the apparatus obtains target time series data from the database based on the predicted backtracking interval and the predicted number of queries.
[0063] In some embodiments, the apparatus obtains time series data corresponding to the query request from the database based on the predicted backtracking interval and the predicted number of queries.
[0064] In this way, the data fast query method provided by the embodiments of the present disclosure determines the predicted backtracking interval and the predicted number of queries based on the write information of the time series data written into the database and the corresponding query information when querying the N most recent pieces of time series data of a specified time, and backtracks the time series data of the specified time based on the predicted backtracking interval and the predicted number of queries to query the N most recent pieces of time series data, thereby improving the query efficiency and reducing resource waste caused by useless data.
[0065] Figure 3 Another optional flowchart of the data fast query method provided by the embodiments of the present disclosure is shown, which will be described according to each step.
[0066] In step S301, in response to the first time series data being written into the database, the apparatus determines percentile write information of a write interval corresponding to the at least one piece of time series data.
[0067] In some embodiments, the percentile write information refers to the position of a time series data in 100 time series data; for example, 100 time series data are sorted in ascending order, and the position of a time series data in the sorted queue is determined. If it is at the 50th position, it is the 50th percentile, and if it is at the 90th position, it is the 90th percentile. It can reflect the composition of the 100 time series data to a certain extent.
[0068] In step S302, the prediction backtracking interval is determined based on the query quantity upper limit included in the query information, the current remaining query data quantity, and the percentile write information.
[0069] In some embodiments, the device can determine the prediction backtracking interval based on the product of the minimum value between the query quantity upper limit and the current remaining query data quantity and the percentile write information P x , i.e., min(N i , M) * P x . The percentile write information includes the value corresponding to the percentile of the time series data in the write interval statistics (i.e., the position of the time series data in 100 time series data).
[0070] In step S303, the predicted query times are determined based on the first data quantity, the query quantity upper limit, and the query times coefficient.
[0071] In some embodiments, the first data quantity N0 and the query times coefficient are determined based on the number of time series data written into the database and the first data quantity.
[0072] In some embodiments, the device determines the preset query times based on the product of the first data quantity and the ceiling value of the quotient of the minimum value between the query quantity upper limit and the first data quantity; i.e.,
[0073] In step S304, the target time series data are obtained from the database based on the prediction backtracking interval and the prediction query times.
[0074] In some embodiments, the device obtains the time series data corresponding to the query request from the database based on the prediction backtracking interval and the prediction query times.
[0075] Thus, by the data fast query method provided by the embodiments of the present disclosure, when the write information of the time series data written into the database and the corresponding query information are used to confirm the query of the N latest time series data at a specified time, the prediction backtracking interval and the prediction query times are determined, the time series data at the specified time is backtracked based on the prediction backtracking interval and the prediction query times, and the N latest time series data are queried, so that the query efficiency is improved and the resource waste caused by useless data is reduced.
[0076] Figure 4 Another optional flowchart of the data fast query method provided by the embodiments of the present disclosure is shown, Figure 5 A data flow diagram of the data fast query method provided by the embodiments of the present disclosure is shown, which will be described according to each step.
[0077] In step S401, when the first time series data is written, the write information of the first time series data is confirmed.
[0078] In some embodiments, the device can sample 1 piece at a certain recording interval Y, record the time stamp and average interval I of the latest X (X>1) pieces of records of the point corresponding to the first time series data, or automatically perform percentile statistics (50th percentile, 90th percentile, 95th percentile, etc.) according to the reporting interval of the point corresponding to the first time series data. The first time series data is any time series data.
[0079] In step S402, when the query request is received, the prediction backtracking interval and the prediction query times are confirmed based on the write information.
[0080] In some embodiments, the device confirms the prediction backtracking interval based on the product of the minimum value between the query upper limit M and the current remaining query data quantity Ni and the average interval I between the at least one time series data, that is, min(N i ,M)*I; wherein, N i represents the remaining query data quantity after the i th query, that is, the current remaining query data quantity. Alternatively, the device can confirm the prediction backtracking interval based on the product of the minimum value between the query upper limit and the current remaining query data quantity and the percentile write information P x , that is, min(N i ,M)*P x . The percentile write information includes the value corresponding to the percentile of the time series data in the write interval statistics.
[0081] In some embodiments, the device confirms the preset query times based on the product of the first data quantity and the upward rounding value of the quotient of the minimum value between the query upper limit and the first data quantity, that is,
[0082] In some optional embodiments, the first time interval corresponding to the query request can be arbitrary (i.e., the time interval is not limited), and if the moment corresponding to the first time interval and the first data quantity are in the sampling interval, the query can be performed according to the hit interval.
[0083] In this way, by using the data fast query method provided in the embodiments of the present disclosure, the write information of the time series data written into the database and the corresponding query information are used to confirm the prediction backtracking interval and the prediction query number when querying the N latest time series data of a specified time, and the time series data of the specified time is backtracked based on the prediction backtracking interval and the prediction query number, so that the N latest time series data are queried, the query efficiency is improved, and the resource waste caused by useless data is reduced.
[0084] Figure 6 An optional structure diagram of the data fast query apparatus provided in the embodiments of the present disclosure is shown, which will be described according to each part.
[0085] In some embodiments, the data fast query apparatus 600 includes an acquisition unit 601 and a confirmation unit 602.
[0086] The acquisition unit 601 is configured to, in response to the first time series data being written into the database, acquire write information corresponding to at least one time series data.
[0087] The confirmation unit 602 is configured to, in response to receiving a query request, confirm a prediction backtracking interval and a prediction query number based on write information corresponding to the at least one time series data and query information corresponding to the query request.
[0088] The query request includes acquiring time series data of a first data quantity in a first time interval.
[0089] The acquisition unit 601 is further configured to acquire target time series data from the database based on the prediction backtracking interval and the prediction query number.
[0090] The acquisition unit 601 is specifically configured to acquire a timestamp corresponding to at least one time series data before a point corresponding to the first time series data and an average interval between the at least one time series data.
[0091] The confirmation unit 602 is specifically configured to confirm the prediction backtracking interval based on a query upper limit included in the query information, a current remaining query data quantity, and the average interval between the at least one time series data.
[0092] The prediction query number is confirmed based on the first data quantity, the query upper limit, and a query number coefficient.
[0093] The confirming unit 602 is specifically configured to confirm the predicted backtracking interval based on a product of a minimum value between the upper limit of the query quantity and a current remaining query data quantity and an average interval between the at least one piece of time series data.
[0094] The confirming unit 602 is specifically configured to confirm the preset query times based on a product of the first data quantity and an upward rounding value of a quotient of a minimum value between the upper limit of the query quantity and the first data quantity, and the query times coefficient.
[0095] The query times coefficient is determined based on a quantity of written time series data and the first data quantity.
[0096] The obtaining unit 601 is specifically configured to confirm percentile write information of a write interval corresponding to the at least one piece of time series data.
[0097] According to embodiments of the present disclosure, the present disclosure further provides an electronic device and a readable storage medium.
[0098] Figure 7 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.
[0099] As shown in Figure 7 The electronic device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for operation of the electronic device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0100] A plurality of components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.
[0101] The computing unit 801 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs various methods and processes described above, such as the fast data query method. For example, in some embodiments, the fast data query method can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded onto the RAM 803 and executed by the computing unit 801, one or more steps of the fast data query method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the fast data query method by any other appropriate means, such as by means of firmware.
[0102] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip system (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
[0103] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.
[0104] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0105] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
[0106] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.
[0107] The computer system can include clients and servers. The clients and the servers are generally remote from each other and typically interact through a communication network. The relationship of client and server is one of communication and distribution, with the server receiving requests from the client and transmitting data to the client. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0108] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without limitation, as long as the desired results of the technology disclosed in the present disclosure are achieved.
[0109] In addition, the terms "first", "second", etc., are used only for the purpose of description, and should not be understood as indicating or implying relative importance or implying the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.
[0110] The above description is merely that of a specific implementation of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present disclosure, which should be covered within the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be subject to the scope of protection of the claims.
Claims
1. A method for fast data retrieval, the method comprising: In response to the first time-series data being written to the database, the write information corresponding to at least one time-series data is obtained; The step of obtaining the write information corresponding to at least one time series data includes: obtaining the timestamp corresponding to at least one time series data before the point corresponding to the first time series data, and the average interval between the at least one time series data; or, confirming the percentile write information of the write interval corresponding to the at least one time series data. In response to receiving a query request, the prediction backtracking interval and the prediction query count are confirmed based on the write information corresponding to the at least one time-series data and the query information corresponding to the query request. The query request includes obtaining time-series data of a first number of data within a first time interval; The step of determining the prediction backtracking interval based on the write information corresponding to the at least one piece of time-series data and the query information corresponding to the query request includes: Based on the query information including the upper limit of the number of queries, the current number of remaining query data, and the average interval between the at least one time-series data, the prediction backtracking interval is determined; or, Based on the query information, including the upper limit of the number of queries, the current number of remaining query data, and the percentile writing information, the prediction backtracking interval is confirmed; wherein, the current number of remaining query data is determined based on the number of time-series data that has been queried and the first data quantity.
2. The method according to claim 1, further comprising: Target time-series data is obtained from the database based on the predicted backtracking interval and the predicted query count.
3. The method according to claim 1, wherein determining the predicted number of queries based on the write information corresponding to the at least one time-series data and the query information corresponding to the query request includes: The predicted number of queries is determined based on the first data quantity, the upper limit of the number of queries, and the query frequency coefficient.
4. The method according to claim 1, wherein determining the prediction backtracking interval based on the query information including the upper limit of the number of queries, the current number of remaining query data, and the average interval between the at least one time-series data includes: The prediction backtracking interval is determined by the product of the minimum between the upper limit of the number of queries and the number of currently remaining query data and the average interval between the at least one time-series data.
5. The method according to claim 3, wherein determining the predicted number of queries based on the first data quantity, the upper limit of the number of queries, and the query frequency coefficient includes: The predicted number of queries is determined by multiplying the first data quantity and the floor value of the quotient between the upper limit of the query quantity and the minimum value between the first data quantity and the query number coefficient. The query frequency coefficient is determined based on the number of time-series data written and the number of first data.
6. A data fast query device, the device comprising: The acquisition unit is used to acquire the write information corresponding to at least one piece of time-series data in response to the first time-series data being written to the database. The acquisition unit is specifically used to acquire the timestamp corresponding to at least one time series data point preceding the point of the first time series data, and the average interval between the at least one time series data point; Alternatively, confirm the percentile write information of the write interval corresponding to the at least one time-series data; The confirmation unit is configured to, in response to receiving a query request, confirm the predicted backtracking interval and the predicted number of queries based on the write information corresponding to the at least one piece of time-series data and the query information corresponding to the query request; wherein, the query request includes obtaining a first number of time-series data within a first time interval; The confirmation unit is specifically used to confirm the prediction backtracking interval based on the query information, including the upper limit of the number of queries, the current remaining number of query data, and the average interval between the at least one time-series data. The current remaining number of query data is determined based on the number of time-series data already queried and the first data quantity; or, Based on the query information, including the upper limit of the number of queries, the current number of remaining query data, and the percentile writing information, the prediction backtracking interval is determined.
7. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
8. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1-5.
Citation Information
Patent Citations
Data query method, system and device and electronic equipment
CN110096489A
Time-based data retrieval prediction
US20210319008A1