Data retrieval methods, devices, equipment and storage media
By storing candidate data in multiple storage units according to the collection time, and determining the candidate storage unit for data retrieval based on the retrieval time, the problem of long retrieval time and low efficiency in paginated retrieval is solved, and faster data retrieval is achieved.
Patent Information
- Application Number
- CN202311778596.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-21
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-12-21
AI Technical Summary
Existing technologies, when performing paginated searches, result in long search times and low search efficiency when dealing with large amounts of data.
The candidate data is stored in multiple storage units based on the collection time of the candidate data, and the candidate storage units are determined for data retrieval based on the retrieval time and collection time in the pagination retrieval request, thereby narrowing the retrieval scope.
It reduces the amount of data that needs to be retrieved, shortens the retrieval time, and improves retrieval efficiency.
Smart Images

Figure CN117520373B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a data retrieval method, apparatus, device, and storage medium. Background Technology
[0002] With the rapid development of information technology, software systems need to process massive amounts of data, for example, to present such data to system users. Because the display interface of a software system is limited, it will display the massive data in pages within that limited space. When displaying the current page, the software system first needs to retrieve the data to be displayed on that page, i.e., perform a paginated retrieval, also known as a paginated query.
[0003] Currently, pagination retrieval is often implemented based on the `limit` mechanism in the database. Specifically, the software system first uses `limit` to retrieve data from the entire database based on the current page number. Then, it sorts the retrieved data to determine the data to be displayed on the current page. For example, if the current page needs to display data from rows 10000 to 10030 in the database, the software system needs to retrieve the first 10030 rows from the entire database before displaying the current page. Then, it returns the last 30 rows from those first 10030 rows to the current page for display.
[0004] However, when the amount of data to be displayed is large, the number of pages displayed will be large, which will lead to a larger amount of data to be retrieved, resulting in a longer retrieval time and a decrease in retrieval efficiency. Summary of the Invention
[0005] This application provides a data retrieval method, apparatus, device, and storage medium to reduce data retrieval time and improve data retrieval efficiency.
[0006] In a first aspect, a data retrieval method is provided, comprising: storing multiple candidate data into multiple storage units based on the collection time of multiple candidate data; obtaining a pagination retrieval request, the pagination retrieval request including a target retrieval time, the pagination retrieval request being used to request the retrieval of data to be displayed; responding to the pagination retrieval request, determining at least one candidate storage unit among the multiple storage units based on the target retrieval time and multiple collection times; and performing data retrieval in the at least one candidate storage unit to obtain the data to be displayed.
[0007] Secondly, a data retrieval device is provided, comprising: a storage module for storing multiple candidate data into multiple storage units based on the acquisition time of multiple candidate data; an acquisition module for acquiring a pagination retrieval request, the pagination retrieval request including a target retrieval time, the pagination retrieval request being used to request the acquisition of data to be displayed; a determination module for determining at least one candidate storage unit among the multiple storage units in response to the pagination retrieval request, based on the target retrieval time and multiple acquisition times; and a retrieval module for performing data retrieval in the at least one candidate storage unit to obtain the data to be displayed.
[0008] Thirdly, an electronic device is provided, comprising: a processor and a memory for storing a computer program, the processor for calling and running the computer program stored in the memory, and performing the methods as described in the first aspect or its various implementations.
[0009] Fourthly, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the methods described in the first aspect or its various implementations.
[0010] Fifthly, a computer program product is provided, including computer program instructions that cause a computer to perform the methods as described in the first aspect or its various implementations.
[0011] Sixthly, a computer program is provided that causes a computer to perform the methods described in the first aspect or its various implementations.
[0012] The technical solution provided in this application allows an electronic device to store multiple candidate data in multiple storage units based on the acquisition time of these candidate data. The electronic device can then obtain a paginated retrieval request including a target retrieval time. This paginated retrieval request requests the data to be displayed. In response to the paginated retrieval request, the electronic device determines at least one candidate storage unit from the multiple storage units based on the target retrieval time and multiple acquisition times, and performs data retrieval within this candidate storage unit to obtain the data to be displayed. In this process, the electronic device can store candidate data in multiple storage units based on the acquisition time of the candidate data and select a subset of storage units (at least one candidate storage unit) from the multiple storage units based on the retrieval time in the paginated retrieval request. This allows for retrieval only within a subset of storage units, reducing the amount of data to be retrieved, thus narrowing the retrieval scope and reducing retrieval time while improving retrieval efficiency. Attached Figure Description
[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 An application scenario diagram provided for an embodiment of this application;
[0015] Figure 2 A flowchart illustrating a data retrieval method provided in this application embodiment;
[0016] Figure 3 A schematic diagram illustrating a data retrieval method provided in an embodiment of this application;
[0017] Figure 4 A schematic diagram illustrating another data retrieval method provided in an embodiment of this application;
[0018] Figure 5 A flowchart illustrating another data retrieval method provided in this application embodiment;
[0019] Figure 6 A schematic diagram illustrating another data retrieval method provided in an embodiment of this application;
[0020] Figure 7 A schematic diagram illustrating yet another data retrieval method provided in an embodiment of this application;
[0021] Figure 8 A flowchart illustrating another data retrieval method provided in this application embodiment;
[0022] Figure 9 A schematic diagram illustrating yet another data retrieval method provided in an embodiment of this application;
[0023] Figure 10 A flowchart illustrating yet another data retrieval method provided in this application embodiment;
[0024] Figure 11 A schematic diagram illustrating yet another data retrieval method provided in an embodiment of this application;
[0025] Figure 12 A schematic diagram illustrating yet another data retrieval method provided in an embodiment of this application;
[0026] Figure 13 A schematic diagram illustrating yet another data retrieval method provided in an embodiment of this application;
[0027] Figure 14 A schematic diagram of a data retrieval device 1400 provided in an embodiment of this application;
[0028] Figure 15 This is a schematic block diagram of an electronic device 1500 provided in an embodiment of this application. Detailed Implementation
[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0031] As mentioned above, in the existing technology, when the amount of data to be retrieved is large, there are problems such as long retrieval time and low retrieval efficiency.
[0032] To address the aforementioned technical problems, the inventive concept of this application is as follows: candidate data is stored in multiple storage units based on the collection time of the candidate data; candidate storage units are selected from the multiple storage units based on the retrieval time and collection time in the pagination retrieval request; data retrieval is performed in the candidate storage units, thereby reducing the amount of data that needs to be retrieved, i.e., narrowing the retrieval scope, thereby reducing retrieval time and improving retrieval efficiency.
[0033] It should be understood that the technical solution of this application can be applied to the following scenarios, but is not limited to:
[0034] In some possible implementations, the technical solution of this application can be applied to data retrieval scenarios, specifically, to paginated retrieval scenarios, but is not limited thereto.
[0035] In some possible ways, Figure 1 An application scenario diagram provided for an embodiment of this application, such as... Figure 1As shown, this application scenario may include electronic device 120 and terminal device 110. Electronic device 120 can establish a connection with terminal device 110 via a wired network or a wireless network.
[0036] For example, terminal device 110 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, in-vehicle device, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc., but is not limited thereto. Electronic device 120 may be a server, a server cluster composed of multiple servers, or a cloud platform control center; this application does not limit this.
[0037] In one embodiment of this application, the electronic device 120 can store multiple candidate data into multiple storage units according to the acquisition time of multiple candidate data. Then, the terminal device 110 can send a pagination retrieval request to the electronic device 120. The pagination retrieval request can be used to request the data to be displayed. The electronic device 120 can receive the pagination retrieval request and, in response to the pagination retrieval request, determine at least one candidate storage unit among the multiple storage units, and perform data retrieval in at least one candidate storage unit to obtain the data to be displayed. The data to be displayed is then sent to the terminal device 110, and the terminal device 110 can then display the data to be displayed.
[0038] also, Figure 1 An electronic device and a terminal device are given as examples, but in practice, other numbers of electronic devices and terminal devices may be included, and this application does not limit them.
[0039] In other possible implementations, the technical solution of this application may also be executed by the aforementioned electronic device 120, or by the aforementioned terminal device 110, and this application does not impose any restrictions on this.
[0040] After introducing the application scenarios of the embodiments of this application, the technical solution of this application will be described in detail below:
[0041] Figure 2 A flowchart illustrating a data retrieval method provided in this application embodiment, the method can be performed by, for example... Figure 1 The electronic device 120 shown performs, but is not limited to, its functions. For example... Figure 2 As shown, the method may include the following steps:
[0042] S210: Based on the acquisition time of multiple candidate data, store multiple candidate data into multiple storage units;
[0043] S220: Obtain a pagination search request. The pagination search request includes the target search time and is used to request the data to be displayed.
[0044] S230: In response to a pagination retrieval request, determine at least one candidate storage unit from a plurality of storage units based on the target retrieval time and multiple acquisition times;
[0045] S240: Perform data retrieval in at least one candidate storage unit to obtain the data to be displayed.
[0046] In some implementations, before executing S210, the electronic device can first determine the candidate data to be collected based on the content to be displayed in pagination, collect multiple candidate data, and determine the collection time for multiple candidate data. The collection time of the candidate data can be the system time of the electronic device when it collects the candidate data; for example, when the electronic device is a server, the collection time of the candidate data can be the server time when the server collects the candidate data.
[0047] For example, suppose that medical device data needs to be displayed in pages. The electronic device can determine that the candidate data to be collected is medical device data, collect multiple medical device data, record the system time when the multiple medical device data are collected, and obtain the collection time of multiple medical device data.
[0048] In some implementations, multiple storage units can be consecutive storage units within the same database. This allows electronic devices to perform continuous scanning of storage units when retrieving data, reducing the time spent determining and switching between storage units, thereby further improving data retrieval efficiency.
[0049] For example, multiple storage units can be multiple consecutive data tables in the same database.
[0050] In some possible implementations, storing multiple candidate data into multiple storage units based on the acquisition time of multiple candidate data includes: storing multiple candidate data consecutively into multiple storage units according to the chronological order of the multiple acquisition times.
[0051] The time order of the multiple acquisition times can be from earliest to latest or from latest to earliest; this application does not impose any limitation on this. Furthermore, this application will describe the technical solution of this application using the example of the multiple acquisition times being in an early-to-late order. When the time order of the multiple acquisition times is other than that described here, the corresponding content and effects are similar, and to avoid repetition, this application will not elaborate further.
[0052] For example, suppose there are multiple storage units: storage unit 1 and storage unit 2. Both storage unit 1 and storage unit 2 can store up to 3 candidate data items. The candidate data items are arranged in order of collection time from earliest to latest as follows: candidate data 1, candidate data 2, candidate data 3, candidate data 4, candidate data 5, and candidate data 6. Figure 3 As shown in (a), the electronic device can store the above 6 candidate data into the above 2 storage units in the order of collection time from earliest to latest: candidate data 1, candidate data 2, and candidate data 3 are stored in storage unit 1 in sequence, and then candidate data 4, candidate data 5, and candidate data 6 are stored in storage unit 2 in sequence.
[0053] Alternatively, the electronic device can also cyclically store multiple candidate data into multiple storage units according to the above time sequence, for example, in conjunction with the above embodiments, such as Figure 3 As shown in (b), the electronic device can store candidate data 1 and candidate data 2 in storage unit 1 and storage unit 2 in sequence, then store candidate data 3 and candidate data 4 in storage unit 1 and storage unit 2 in sequence, and finally store candidate data 5 and candidate data 6 in storage unit 1 and storage unit 2 in sequence.
[0054] It should be noted that this application does not restrict the storage method of storing multiple candidate data in multiple storage units in electronic devices. Specifically, storing candidate data continuously in storage units according to the chronological order of acquisition time ensures that when searching within the storage units based on the target retrieval time, the retrieved data meets the retrieval conditions related to that time. For example, it ensures that the retrieved data to be displayed is acquired earlier or later than the target retrieval time. This guarantees that the paginated content is displayed on the page in a continuous chronological order, ensuring the temporal sequence and non-repetitiveness of data display and improving the data display effect.
[0055] In some feasible implementations, before data retrieval, the electronic device can sort multiple candidate data according to the chronological order of multiple acquisition times, determining the sequence number of each candidate data, with no repetition of these sequence numbers. This allows the electronic device to determine a search condition based on the sequence number of the candidate data when performing a retrieval in the storage unit. This ensures that the retrieved data to be displayed is not only different from previously retrieved historical data (ensuring non-repetition of search results), but also that the acquisition time of the search results is earlier or later than the target search time, thus ensuring the temporal sequence of data display and improving the data display effect.
[0056] It should be noted that the specific content corresponding to determining a search condition based on the sequence number of the candidate data will be described in subsequent embodiments of this application, and will not be repeated here.
[0057] For example, such as Figure 4 As shown, assuming the electronic device is a server, the server can determine the collection time of the candidate data based on the server time when the candidate data is collected. Then, according to the order of collection times from earliest to latest, the collected candidate data is sorted using a distributed snowflake algorithm to determine a unique, unidirectionally increasing identifier (IDentity) for each candidate data point. These IDs are then used as the sequence numbers for each candidate data point. Afterward, the electronic device can process the candidate data based on its ID and collection time. For example, it can store the candidate data in a storage unit based on its ID and collection time, and then retrieve it.
[0058] Specifically, the electronic device can determine the acquisition period and store multiple candidate data into multiple storage units based on the acquisition period and acquisition time. Each storage unit stores candidate data for one acquisition period. The acquisition period can be hour, day, month, or year; this application does not impose any restrictions on this. For example, the electronic device can determine the acquisition period based on the total amount of candidate data. First, the electronic device can set multiple quantity ranges and multiple acquisition periods, with each quantity range corresponding to a specific acquisition period. The target acquisition period corresponding to the target quantity range including the total amount of data is determined as the acquisition period for that total amount of data. This ensures that the amount of candidate data stored in each storage unit is appropriate, thereby further improving data retrieval efficiency.
[0059] For example, assuming the candidate data collection period is one day, the electronic device can divide the database into multiple data partitions, each corresponding to a storage unit, and the multiple data partitions are used to store candidate data collected on the same day.
[0060] It should be noted that this application does not restrict the sorting method of the candidate data.
[0061] In some implementation methods, a user can trigger a display button displayed in a client installed on a terminal device. The terminal device can then receive and respond to this triggering operation, sending a pagination search request to the electronic device to retrieve the data to be displayed and displaying it on the current paginated page. Thus, the electronic device can receive the pagination search request.
[0062] The pagination search request may include a target search time, which may be a time randomly generated by the terminal device; it may also be the time when the user inputs or selects data before the user performs the above-mentioned triggering operation; or it may be the time when the last piece of data to be displayed on the previous pagination page of the terminal device was collected. This application does not limit this.
[0063] In some possible ways, Figure 5 A flowchart illustrating another data retrieval method provided in an embodiment of this application. Based on Figure 2 ,like Figure 5 As shown, the above S230 includes:
[0064] S510: Determine a first time interval consisting of the earliest and latest collection times among multiple collection times, and determine a second time interval consisting of the target retrieval time and either the latest or earliest collection time.
[0065] S520: Determine the time intersection between the first time interval and the second time interval;
[0066] S530: Among multiple storage units, the storage unit that stores candidate data whose acquisition time belongs to the time intersection is determined as at least one candidate storage unit.
[0067] For example, multiple storage units store candidate data for a collection period. The above-mentioned S530 may include: determining the period identifier of each of the multiple storage units according to the collection period and the collection time of the candidate data stored by each of the multiple storage units; determining the target period identifier of the time intersection according to the collection period and the time intersection; comparing the period identifier of each of the multiple storage units with the target period identifier; and determining the storage unit whose period identifier is the target period identifier as at least one candidate storage unit.
[0068] For example, for any one of multiple storage units, the electronic device can determine the period identifier of that storage unit by the number corresponding to the acquisition period in the acquisition time of the candidate data stored in that storage unit. For example, assuming the acquisition period is days, and storage unit 1 stores candidate data acquired on September 1, 2023, the electronic device can determine that the number corresponding to the acquisition period "days" is 20230901, 0901, or 01, and thus the electronic device can determine that the period identifier of storage unit 1 is: 20230901, 0901, or 01.
[0069] For example, the above method of determining the target period identifier of the time intersection based on the collection period and the time intersection can include: determining the number corresponding to the collection period in the time intersection as the target period identifier. For example, assuming the collection period is days and the time intersection is [September 1, 2023, September 3, 2023], the electronic device can determine the target period identifier as 20230901, 20230902, and 20230903.
[0070] For example, such as Figure 6 As shown, before comparing the period identifiers of multiple storage cells with the target period identifier, the electronic device can create non-empty data partition identifiers. These non-empty data partition identifiers can be stored using a Set data structure and can include multiple non-empty data partitions, each storing the period identifier of one storage cell. Then, the electronic device can compare the period identifiers of the multiple storage cells with the target period identifier based on the non-empty data partition identifiers to determine candidate storage cells.
[0071] In addition, the electronic device can also determine the earliest and latest acquisition times of the candidate data stored in each storage unit, and determine the time sharding coefficient (i.e., time division interval) corresponding to each storage unit. The earliest acquisition time, the latest acquisition time, and the time sharding coefficient are associated with the non-empty data partition in the non-empty data partition identifier for subsequent data retrieval. This will be described in subsequent embodiments of this application, and will not be elaborated here.
[0072] In some possible implementations, such as Figure 7As shown in the above embodiments, the electronic device can divide the same database into multiple data partitions based on the acquisition time and acquisition period of the candidate data. Each data partition corresponds to a data partition file, for example, a data storage table. Multiple data partition files are used to store candidate data for one acquisition period. Simultaneously, the electronic device can create an ID index file based on the sequence number of each candidate data, for example, the candidate data's ID, for subsequent data retrieval. When storing candidate data to disk in the database, the electronic device can determine the non-empty data partition identifier and correlate the earliest and latest acquisition times, time intervals, and non-empty data partitions in the acquisition time of the candidate data stored in each storage unit.
[0073] In some implementations, the pagination retrieval request also includes a target sequence number. In conjunction with the above embodiments, the above-mentioned data retrieval in at least one candidate storage unit to obtain the data to be displayed includes: retrieving candidate data in at least one candidate storage unit whose sequence number is less than or greater than the target sequence number to obtain the data to be displayed.
[0074] The target sequence number can be a sequence number randomly generated by the terminal device; it can also be a sequence number entered or selected by the user by the terminal device before the user performs the above-mentioned triggering operation; or it can be the sequence number of the last data to be displayed in the previous page of the terminal device. This application does not limit this.
[0075] For example, the electronic device can determine whether to retrieve candidate data with a sequence number less than the target sequence number in at least one candidate storage unit, or candidate data with a sequence number greater than the target sequence number, based on the display order of the data to be displayed on the paginated page and the time sequence of the acquisition time when the candidate data is stored in the storage unit. For instance, assuming that the data to be displayed on the paginated page is displayed in order of acquisition time from latest to earliest, and the electronic device stores the candidate data in the storage unit in order of acquisition time from earliest to latest, with the sequence number of the candidate data increasing unidirectionally with the acquisition time, then when the electronic device retrieves data in the candidate storage unit, it can retrieve candidate data with a sequence number less than the target sequence number to obtain the data to be displayed. This ensures that the data to be displayed on the paginated page is displayed in order of acquisition time from latest to earliest, and also ensures that the data to be displayed does not appear repeatedly on the paginated page.
[0076] In some possible ways, Figure 8 A flowchart illustrating yet another data retrieval method provided in an embodiment of this application. Based on Figure 2 ,like Figure 8As shown, the pagination retrieval request also includes the target sequence number, and the above S240 includes:
[0077] S810: Sort at least one candidate memory unit to obtain the sorting result;
[0078] S820: Perform a search in the first candidate storage unit of the sorting results;
[0079] S830: Determine whether the first candidate storage unit stores the target number of candidate data that meet the search conditions. If the first candidate storage unit stores the target number of candidate data that meet the search conditions, then execute S840; if the first candidate storage unit does not store the target number of candidate data that meet the search conditions, then execute S850.
[0080] S840: Select the target number of candidate data that meet the search criteria as the data to be displayed;
[0081] S850: Determine the existing number of candidate data that meet the search criteria stored in the first candidate storage unit, update the target number using the difference between the target number and the existing number, remove the first candidate storage unit from the sorting results to obtain the updated sorting results, and continue to execute S820 until the first candidate storage unit stores the target number of candidate data that meet the search criteria.
[0082] The target quantity refers to the amount of data to be displayed on the current pagination page. For example, the target quantity can be a quantity randomly generated by the terminal device; it can also be the quantity of user input or selection obtained by the terminal device before the user performs the above-mentioned triggering operation; or it can be the quantity of data to be displayed displayed on the previous pagination page of the terminal device. This application does not limit this.
[0083] For example, in conjunction with the above embodiments, such as Figure 9As shown, assuming the electronic device identifies 30 candidate storage units with period identifiers of 20230901, 20230902, ..., 20230930, these 30 candidate storage units can be consecutive storage units within the same database. The electronic device can sequentially search these 30 candidate storage units using a cursor to determine the target number of data items to be displayed. Specifically, the electronic device can first set the cursor index value to 0 to search the first candidate storage unit among the 30 candidate storage units, i.e., the candidate storage unit corresponding to period identifier 20230901. It then determines whether the target number of candidate data items matching the search criteria has been retrieved after the current candidate storage unit has been searched. If it determines that the target number of candidate data items matching the search criteria has been retrieved, i.e., the current candidate storage unit stores the target number of candidate data items matching the search criteria, it can directly determine these target number of candidate data items matching the search criteria as the data to be displayed; otherwise, if it determines that the target number of candidate data items matching the search criteria has not been retrieved... If the current candidate storage unit does not contain the target number of candidate data that meet the search criteria, then the target number is updated by the difference between the target number and the existing number of candidate data that meet the search criteria in the current candidate storage unit. The cursor index value is then incremented by 1, i.e., the cursor index value is set to 1. The cursor is then moved to search the candidate storage unit corresponding to period identifier 20230902, i.e., the first candidate storage unit after the update, and similar steps are performed until the electronic device determines that the target number of candidate data that meet the search criteria has been retrieved when the current candidate storage unit has been searched.
[0084] Specifically, if the electronic device determines that the collection time of the currently retrieved candidate data is consistent with the latest collection time corresponding to the current candidate storage unit, then the electronic device can determine that the current candidate storage unit has been completely retrieved. Alternatively, if the electronic device determines that the latest collection time corresponding to the last retrieved sub-storage unit in the current candidate storage unit is consistent with the latest collection time corresponding to the current candidate storage unit, then the electronic device can determine that the current candidate storage unit has been completely retrieved. The content corresponding to the sub-storage units will be described in subsequent embodiments, and will not be elaborated here.
[0085] The search criteria mentioned above can include a serial number less than the target serial number. For example, in conjunction with the above embodiments, the electronic device can first search for candidate data based on search criteria other than the serial number being less than the target serial number to determine initial candidate data. Then, the electronic device can determine candidate data with serial numbers less than the target serial number from the initial candidate data based on the ID index file, and identify them as data to be displayed. Alternatively, the electronic device can directly determine candidate data with serial numbers less than the target serial number from multiple candidate data based on the ID index file, and identify them as data to be displayed.
[0086] When searching the first candidate storage unit, the electronic device can sort the retrieved candidate data that meets the search criteria in reverse order and record the number of candidate data in the sorted sequence to obtain the existing quantity. Since the amount of data retrieved at this time is small, the sorting speed is relatively fast. If the electronic device determines that the existing quantity is equal to the target quantity before the current candidate storage unit is completely searched, it can be determined that the first candidate storage unit stores the target quantity of candidate data that meets the search criteria. If the existing quantity is less than the target quantity after the current candidate storage unit is completely searched, the electronic device can be determined that the current candidate storage unit does not store the target quantity of candidate data that meets the search criteria.
[0087] In some possible ways, Figure 10 A flowchart illustrating yet another data retrieval method provided in an embodiment of this application. Based on Figure 8 ,like Figure 10 As shown, the above-mentioned S820 includes:
[0088] S1010: Determine the time partitioning interval for the first candidate memory cell;
[0089] S1020: Based on the time division interval, the acquisition time of the candidate data in the first candidate storage unit is divided into multiple time groups;
[0090] S1030: Determine the sub-memory cells corresponding to each of the multiple time groups in the first candidate memory cell;
[0091] S1040: From multiple sub-storage units, determine the first sub-storage unit that stores candidate data whose acquisition time is closest to the target retrieval time;
[0092] S1050: Search in the first sub-storage unit and in the second sub-storage units of the plurality of sub-storage units whose corresponding time is later or earlier than that of the first sub-storage unit.
[0093] The time corresponding to the sub-storage unit can be either the earliest or the latest time among the acquisition times of the candidate data stored in the sub-storage unit.
[0094] For example, assuming the time interval is 1 hour, the first candidate storage unit stores candidate data 1, candidate data 2, and candidate data 3. The acquisition times of candidate data 1, candidate data 2, and candidate data 3 are 09:10, 09:20, and 10:30, respectively. The electronic device can divide these three acquisition times into two time groups: 09:10 and 09:20 as the first time group and 10:30 as the second time group. These two time groups correspond to sub-storage unit 1 and sub-storage unit 2, respectively. Sub-storage unit 1 stores candidate data 1 and candidate data 2, and sub-storage unit 2 stores candidate data 3.
[0095] Understandably, if a candidate storage unit contains a large amount of candidate data, the electronic device will need to retrieve a significant amount of data when searching within that unit. However, by dividing the candidate storage unit into two sub-storage units—one with a time closest to the target retrieval time and the other with a time later or earlier—the amount of data to be retrieved can be reduced, narrowing the search scope. This allows for faster identification of candidate data that meets the conditions related to the target retrieval time, thereby further improving data retrieval efficiency.
[0096] In some implementations, the time interval for determining the first candidate storage unit includes: determining the number of times the first candidate storage unit in the sorting results is removed; determining the acquisition rate for collecting multiple candidate data; and determining the time interval based on the number of removals and the acquisition rate.
[0097] For example, the electronic device can first calculate the basic time division interval based on the acquisition rate, and determine whether the number of removals is greater than a threshold. The number of removals is the number of times the electronic device removes the first candidate storage unit from the sorting results. If the number of removals is less than the threshold, the basic time division interval can be determined as the time division interval; if the number of removals is greater than or equal to the threshold, the update time division interval can be calculated based on the basic time division interval and the number of removals, and the update time division interval can be determined as the time division interval.
[0098] For example, the electronic device can calculate the hyperbolic cosine value of the acquisition rate and determine the basic time division interval based on the hyperbolic cosine value. Specifically, the electronic device can determine the minimum time division interval and use the maximum value between the minimum time division interval and the hyperbolic cosine value as the basic time division interval. The minimum time division interval ensures that the determined basic time division interval is not too large, thereby avoiding an increase in the retrieval range due to excessive data storage in each sub-storage unit, thus ensuring the rationality of the retrieval range and guaranteeing retrieval efficiency.
[0099] For example, assuming the acquisition rate is v, which represents the number of candidate data collected per second, the minimum time interval is 60 seconds, and the basic time interval is k, in milliseconds, the electronic device can determine the basic time interval according to formula (1).
[0100] Formula (1) is k = max(cothv, 60) * 1000
[0101] For example, an electronic device can determine a target polynomial, input the removal degree into the target polynomial to obtain a time gain, calculate the product of the time gain and the base time division interval to obtain the updated time division interval.
[0102] For example, assuming the number of removals is r and the update time interval is x, the electronic device can determine the update time interval according to formula (2).
[0103]
[0104] Understandably, when the acquisition rate is high, the electronic device will acquire more candidate data per unit time. Therefore, the electronic device can use a smaller time interval to divide the candidate storage units, ensuring that the number of candidate data stored in each sub-storage unit is not excessive. This ensures that the number of candidate data stored in the first and second sub-storage units is not excessive, avoiding low retrieval efficiency due to a large amount of data to be retrieved or a large retrieval range. Conversely, when the acquisition rate is low, the electronic device will acquire fewer candidate data per unit time. Therefore, the electronic device can use a larger time interval to divide the candidate storage units, ensuring that the number of candidate data stored in each sub-storage unit is not too small. This ensures that the number of candidate data stored in the first and second sub-storage units is not too small, avoiding the need for a longer time to determine the target number of candidate data that meet the retrieval conditions due to a small amount of data stored in the sub-storage units. In other words, through the above embodiments, the rationality of the search range can be ensured, guaranteeing retrieval efficiency.
[0105] Furthermore, if no candidate data matching the search criteria is found after searching multiple candidate storage units, it indicates that the candidate data stored in the candidate storage units is relatively sparse, meaning the amount of data stored in each unit is small. Continuing to divide subsequent candidate storage units using the basic time interval would result in more sub-storage units, increasing the number of sub-storage units to be searched and thus reducing search efficiency. Therefore, when the number of removal attempts is greater than or equal to a threshold, the electronic device can adjust the time interval. It can use a time gain determined by the number of removal attempts to increase the time interval, thereby reducing the number of subsequent sub-storage units and the number of loop searches, ultimately improving search efficiency.
[0106] For example, such as Figure 11 As shown, the electronic device can set a loop retrieval counter to record the number of removals. If the number of removals recorded in the loop retrieval counter is 0, the electronic device can determine that the retrieval of the current candidate storage unit (i.e., the first candidate storage unit) is the first retrieval. It can then use the basic time division interval to determine the time division range and divide the current candidate storage unit. If no candidate data matching the retrieval criteria is found in the current candidate storage unit, the electronic device can update the sorting results, determine a new first candidate storage unit as the current candidate storage unit, and simultaneously increment the loop retrieval counter by 1. It also checks whether the number of removals recorded in the loop retrieval counter triggers the threshold for determining the update time division interval, i.e., whether the number of removals is greater than the threshold. If the number of removals does not trigger the threshold for determining the update time division interval, i.e., the number of removals is less than or equal to the threshold, the electronic device can continue to use the basic time division interval to determine the time division range and divide the current candidate storage unit. If the number of removals triggers the threshold for determining the update time interval, i.e., the number of removals exceeds the threshold, the electronic device can determine the update time interval according to the method in the above embodiment, and use the update time interval to determine the time division range, divide the current candidate storage unit, and search in the current candidate storage unit until the target number of candidate data that meet the search conditions are determined.
[0107] The technical solution of this application will be further illustrated below with diagrams:
[0108] In some possible implementations, in conjunction with the above embodiments, such as Figure 12As shown, after receiving a pagination retrieval request from a terminal device, the electronic device can determine the time intersection based on the earliest and latest collection times of multiple storage units and the retrieval time in the pagination retrieval request. Based on this time intersection, it can then identify at least one candidate storage unit from the multiple storage units. Next, for the first candidate storage unit, the electronic device can divide it using a basic time partitioning interval to determine multiple sub-storage units. Based on the ID index file and the ID in the pagination retrieval request, it can identify data that meets the conditions and sort it to determine if its quantity (i.e., the existing quantity) reaches the target quantity in the pagination retrieval request. If the target quantity is reached, the electronic device can respond to the pagination retrieval request and return the data that meets the conditions to the terminal device. If the target number in the pagination retrieval request has not been reached, the electronic device can update the target number based on the existing data, record the number of polling searches (i.e., the number of removals), determine the time interval for the next candidate storage unit based on whether the number of removals has reached the threshold, determine the next candidate storage unit based on the cursor, and perform operations similar to those for the first candidate storage unit until the existing number reaches the target number. At this point, the electronic device can determine all the candidate data that has been retrieved and meets the search criteria as the data to be displayed and return it to the terminal device.
[0109] In some possible implementations, in conjunction with the above embodiments, such as Figure 13 As shown, the electronic device can generate unique, one-way auto-incrementing IDs for each candidate data point based on the server time when the candidate data is collected, using a distributed snowflake algorithm. It then creates an ID index file, timestamps the candidate data according to the server time, and determines multiple data partition files (storage units) in the database based on these timestamps. The candidate data is then stored in these corresponding storage units. When a pagination retrieval request is received from the terminal device, data retrieval can be performed based on the target ID, target quantity, and target retrieval time in the request, determining the data to be displayed. Specifically, multiple data partition files (storage units) can be located based on the target retrieval time to determine candidate storage units. The collection time corresponding to each candidate storage unit is then divided according to time segmentation to identify multiple sub-storage units within the candidate storage units. From these sub-storage units, the first and second sub-storage units are determined, thus reducing the retrieval scope. Finally, using the ID index file and the target ID, the target quantity of data matching the retrieval criteria is determined, sorted, and returned to the terminal device.
[0110] The technical solution provided in this application is illustrated below using experimental data:
[0111] In some feasible implementations, the electronic device can be tested under the test environment indicators shown in Table 1, based on three schemes, to evaluate the retrieval response time under different pagination depths and different amounts of candidate data. The pagination depth is the number of pages that need to be displayed. The three schemes are the prior art scheme based on limit pagination, the prior art scheme based on database indexing, and the technical solution of this application. The retrieval response time can refer to the time between the electronic device receiving the pagination retrieval request and the electronic device determining the data to be displayed.
[0112] Table 1
[0113] hardware index Number of servers 3 CPU 40-core Intel(R) Xeon(R) Silver4 114 CPU @ 2.20GHz Memory 128GB disk 12T HDD operating system CentOS Linux 7 (Core)
[0114] The test results for the response time of each scheme when the pagination depth is 1 are shown in Table 2.
[0115] Table 2
[0116]
[0117] The test results for the response time of each scheme when the pagination depth is 500 are shown in Table 3.
[0118] Table 3
[0119]
[0120]
[0121] Based on the above test results and embodiments, it can be seen that the technical solution of this application can intelligently and automatically determine the collection cycle and time division interval, thereby more reasonably determining the candidate storage unit and sub-storage unit, thus more reasonably reducing the amount of basic data to be retrieved, i.e., narrowing the retrieval range, achieving low resource consumption and millisecond-level retrieval response for massive data, which can significantly improve the response efficiency of pagination retrieval and ensure the stability of pagination retrieval.
[0122] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.
[0123] Figure 14 This application provides a schematic diagram of a data retrieval device 1400 according to an embodiment. (See attached diagram.) Figure 14 As shown, the device 1400 includes:
[0124] Storage module 1401 is used to store multiple candidate data into multiple storage units according to the collection time of multiple candidate data;
[0125] The acquisition module 1402 is used to acquire pagination search requests. The pagination search requests include the target search time and are used to request the data to be displayed.
[0126] The determination module 1403 is used to determine at least one candidate storage unit from a plurality of storage units in response to a pagination retrieval request, based on the target retrieval time and multiple collection times.
[0127] The retrieval module 1404 is used to retrieve data from at least one candidate storage unit to obtain the data to be displayed.
[0128] In some implementations, the pagination retrieval request also includes a target number; the retrieval module 1404 is specifically used for: sorting at least one candidate storage unit to obtain a sorting result; performing a retrieval in the first candidate storage unit of the sorting result; determining whether the first candidate storage unit stores a target number of candidate data that meet the retrieval conditions; if the first candidate storage unit stores a target number of candidate data that meet the retrieval conditions, then determining the target number of candidate data that meet the retrieval conditions as data to be displayed; if the first candidate storage unit does not store a target number of candidate data that meet the retrieval conditions, then determining the existing number of candidate data that meet the retrieval conditions stored in the first candidate storage unit, updating the target number using the difference between the target number and the existing number, removing the first candidate storage unit from the sorting result to obtain the updated sorting result, and continuing to perform a retrieval in the first candidate storage unit of the sorting result until the first candidate storage unit stores a target number of candidate data that meet the retrieval conditions.
[0129] In some implementations, the retrieval module 1404 is specifically used for: determining the time division interval of the first candidate storage unit; dividing the collection time of the candidate data in the first candidate storage unit into multiple time groups according to the time division interval; determining the sub-storage units corresponding to each of the multiple time groups in the first candidate storage unit; determining the first sub-storage unit that stores the candidate data whose collection time is closest to the target retrieval time from the multiple sub-storage units; and performing retrieval in the first sub-storage unit and in the second sub-storage units among the multiple sub-storage units whose corresponding time is later or earlier than the first sub-storage unit.
[0130] In some implementations, the retrieval module 1404 is specifically used to: determine the number of times the first candidate storage unit in the sorting results is removed; determine the collection rate for collecting multiple candidate data; and determine the time division interval based on the number of removals and the collection rate.
[0131] In some implementation methods, the retrieval module 1404 is specifically used to: calculate the basic time division interval based on the acquisition rate; determine whether the number of removals is greater than the number threshold; if the number of removals is less than the number threshold, then determine the basic time division interval as the time division interval; if the number of removals is greater than or equal to the number threshold, then calculate the update time division interval based on the basic time division interval and the number of removals, and determine the update time division interval as the time division interval.
[0132] In some implementations, the retrieval module 1404 is specifically used to: calculate the hyperbolic cosine value of the acquisition rate; and determine the basic time division interval based on the hyperbolic cosine value.
[0133] In some implementations, the retrieval module 1404 is specifically used to: determine the minimum time division interval; and determine the base time division interval by taking the maximum value between the minimum time division interval and the hyperbolic cosine value.
[0134] In some implementations, the retrieval module 1404 is specifically used to: determine the target polynomial; input the removal number into the target polynomial to obtain the time gain; and calculate the product of the time gain and the base time partition interval to obtain the update time partition interval.
[0135] In some implementations, the storage module 1401 is specifically used to: continuously store multiple candidate data into multiple storage units according to the time order of multiple acquisition times; the determination module 1403 is specifically used to: determine a first time interval composed of the earliest and latest acquisition times among the multiple acquisition times; determine a second time interval composed of the target retrieval time and either the latest or earliest acquisition time; determine the time intersection of the first time interval and the second time interval; and determine at least one candidate storage unit among the multiple storage units that stores candidate data whose acquisition times belong to the time intersection.
[0136] In some implementations, multiple storage units store candidate data for one acquisition cycle; the determining module 1403 is specifically used to: determine the cycle identifier of each of the multiple storage units according to the acquisition cycle and the acquisition time of the candidate data stored by each of the multiple storage units; determine the target cycle identifier of the time intersection according to the acquisition cycle and the time intersection; compare the cycle identifier of each of the multiple storage units with the target cycle identifier, and determine the storage unit whose cycle identifier is the target cycle identifier as at least one candidate storage unit.
[0137] In some implementations, the device 1400 further includes: a sorting module 1405, used to: sort multiple candidate data according to the time order of multiple acquisition times, determine the sequence number of each of the multiple candidate data, and the multiple sequence numbers are not repeated; and a retrieval module 1404, specifically used to retrieve candidate data in at least one candidate storage unit whose sequence number is less than or greater than the target sequence number, to obtain the data to be displayed.
[0138] In some implementations, multiple storage units are contiguous storage units within the same database.
[0139] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be found in the method embodiments. To avoid repetition, further details are omitted here. Specifically, Figure 14 The apparatus 1400 shown can perform the above-described method embodiments, and the aforementioned and other operations and / or functions of each module in the apparatus 1400 are respectively for implementing the corresponding processes in the above-described method. For the sake of brevity, they will not be described in detail here.
[0140] The apparatus 1400 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.
[0141] Figure 15 This is a schematic block diagram of an electronic device 1500 provided in an embodiment of this application.
[0142] like Figure 15 As shown, the electronic device 1500 may include:
[0143] The system includes a memory 1510 and a processor 1520. The memory 1510 stores computer programs and transfers the program code to the processor 1520. In other words, the processor 1520 can retrieve and run the computer program from the memory 1510 to implement the methods described in the embodiments of this application.
[0144] For example, the processor 1520 can be used to execute the above-described method embodiments according to instructions in the computer program.
[0145] In some embodiments of this application, the processor 1520 may include, but is not limited to:
[0146] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0147] In some embodiments of this application, the memory 1510 includes, but is not limited to:
[0148] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).
[0149] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 1510 and executed by the processor 1520 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.
[0150] like Figure 15 As shown, the electronic device may further include:
[0151] Transceiver 1530, which can be connected to processor 1520 or memory 1510.
[0152] The processor 1520 can control the transceiver 1530 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 1530 may include a transmitter and a receiver. The transceiver 1530 may further include antennas, and the number of antennas may be one or more.
[0153] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.
[0154] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.
[0155] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, the computer can perform all or part of the corresponding processes in the methods of the embodiments of this application, producing the functions achievable by the methods of the embodiments of this application. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., Digital Video Disc (DVD)), or a semiconductor medium (e.g., Solid State Disk (SSD)).
[0156] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0157] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.
[0158] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
[0159] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A data retrieval method, characterized in that, include: Based on the collection time of multiple candidate data, the multiple candidate data are stored in multiple storage units; Obtain a pagination search request, wherein the pagination search request includes the target search time, and the pagination search request is used to request the data to be displayed; In response to the pagination retrieval request, at least one candidate storage unit is determined from the plurality of storage units based on the target retrieval time and the plurality of collection times; Data retrieval is performed in at least one candidate storage unit to obtain the data to be displayed, including: The at least one candidate storage unit is sorted to obtain a sorting result; Retrieval is performed in the first candidate storage unit of the sorting results, including: Determine the time division interval of the first candidate memory unit; Based on the time division interval, the collection time of the candidate data in the first candidate storage unit is divided to obtain multiple time groups; Determine the sub-storage units corresponding to each of the plurality of time groups in the first candidate storage unit; From the plurality of said sub-storage units, a first sub-storage unit is determined to store candidate data whose acquisition time is closest to the target retrieval time; The search is performed in the first sub-storage unit and in the second sub-storage units of the plurality of sub-storage units, where the time is later or earlier than that of the first sub-storage unit; The step of determining the time interval for the first candidate memory cell includes: Determine the number of times to remove the first candidate storage unit in the sorting result; Determine the acquisition rate for collecting the multiple candidate data; The time division interval is determined based on the number of removals and the acquisition rate.
2. The method according to claim 1, characterized in that, The pagination retrieval request also includes a target quantity; the step of retrieving data from the at least one candidate storage unit to obtain the data to be displayed includes: Determine whether the first candidate storage unit contains the target number of candidate data that meet the search conditions; If the first candidate storage unit stores the target number of candidate data that meet the search conditions, then the target number of candidate data that meet the search conditions are determined as the data to be displayed; If the first candidate storage unit does not store the target number of candidate data that meet the search conditions, then the existing number of candidate data that meet the search conditions stored in the first candidate storage unit is determined, the target number is updated using the difference between the target number and the existing number, the first candidate storage unit is removed from the sorting result to obtain the updated sorting result, and the search is continued in the first candidate storage unit of the sorting result until the first candidate storage unit stores the target number of candidate data that meet the search conditions.
3. The method according to claim 1, characterized in that, Determining the time interval based on the number of removals and the acquisition rate includes: Calculate the base time interval based on the acquisition rate; Determine whether the number of removals exceeds a threshold. If the number of removals is less than the number threshold, then the basic time division interval is determined as the time division interval; If the number of removals is greater than or equal to the number threshold, then the update time interval is calculated based on the basic time interval and the number of removals, and the update time interval is determined as the time interval.
4. The method according to claim 3, characterized in that, The step of calculating the base time division interval based on the acquisition rate includes: Calculate the hyperbolic cosine value of the acquisition rate; Determine the minimum time interval; The maximum value between the minimum time interval and the hyperbolic cosine value is determined as the basic time interval.
5. The method according to claim 3, characterized in that, The step of calculating the update time interval based on the base time interval and the number of removals includes: Determine the objective polynomial; The removal count is input into the target polynomial to obtain the time gain; The product of the time gain and the base time division interval is calculated to obtain the update time division interval.
6. The method according to any one of claims 1-5, characterized in that, The step of storing the multiple candidate data into multiple storage units according to the collection time of the multiple candidate data includes: The candidate data are continuously stored in multiple storage units according to the time sequence of the multiple collection times; The step of determining at least one candidate storage unit from the plurality of storage units based on the target retrieval time and the plurality of acquisition times includes: A first time interval is determined, consisting of the earliest and latest collection times among a plurality of collection times; a second time interval is determined, consisting of the target retrieval time and either the latest or the earliest collection time. Determine the time intersection between the first time interval and the second time interval; Among the plurality of storage units, the storage unit that stores candidate data whose acquisition time belongs to the time intersection is determined as at least one candidate storage unit.
7. The method according to claim 6, characterized in that, The plurality of storage units each store candidate data for one acquisition cycle; determining the storage unit among the plurality of storage units that stores candidate data whose acquisition time belongs to the time intersection as the at least one candidate storage unit includes: Based on the acquisition period and the acquisition time of the candidate data stored in each of the plurality of storage units, the period identifier of each of the plurality of storage units is determined, and based on the acquisition period and the time intersection, the target period identifier of the time intersection is determined; The period identifier of each of the plurality of storage cells is compared with the target period identifier, and the storage cell whose period identifier is the target period identifier is determined as the at least one candidate storage cell.
8. A data retrieval device, characterized in that, include: A storage module is used to store the multiple candidate data into multiple storage units according to the collection time of the multiple candidate data; The acquisition module is used to acquire pagination search requests, wherein the pagination search requests include the target search time and are used to request the acquisition of data to be displayed. A determination module is configured to, in response to the pagination retrieval request, determine at least one candidate storage unit among the plurality of storage units based on the target retrieval time and a plurality of collection times; A retrieval module, configured to retrieve data from the at least one candidate storage unit to obtain the data to be displayed, includes: The at least one candidate storage unit is sorted to obtain a sorting result; Retrieval is performed in the first candidate storage unit of the sorting results, including: Determine the time division interval of the first candidate memory unit; Based on the time division interval, the collection time of the candidate data in the first candidate storage unit is divided to obtain multiple time groups; Determine the sub-storage units corresponding to each of the plurality of time groups in the first candidate storage unit; From the plurality of said sub-storage units, a first sub-storage unit is determined to store candidate data whose acquisition time is closest to the target retrieval time; The search is performed in the first sub-storage unit and in the second sub-storage units of the plurality of sub-storage units, where the time is later or earlier than that of the first sub-storage unit; The step of determining the time interval for the first candidate memory cell includes: Determine the number of times to remove the first candidate storage unit in the sorting result; Determine the acquisition rate for collecting the multiple candidate data; The time division interval is determined based on the number of removals and the acquisition rate.
9. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the processor being used to invoke and run the computer program stored in the memory to perform the method of any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1-7.
Citation Information
Patent Citations
Paging retrieval method and device
CN108228663A
Data storage method and equipment
CN114519055A
