Query offset reconstruction method and device, equipment and storage medium
By rebuilding the target unique key when the query offset expires, the problem of inefficient query is solved, and the effect of efficient querying data after the offset expires is achieved.
Patent Information
- Application Number
- CN202411439491.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-02-14
AI Technical Summary
In the prior art, the query offset expires and is deleted in the cache, resulting in inefficient query.
When the first query offset sends a query request to the data source in an abnormal situation, the standard unique key of the target query data is obtained, and by comparing the unique key in the target data set, the target unique key is reconstructed as a new query offset. Quantitatively store it in cache.
Even if the first query offset is deleted due to expire, it can still support querying subsequent data from its location, improving query efficiency.
Smart Images
Figure CN119441288A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment, storage medium and product for reconstructing a query offset. Background Art
[0002] With the development of computer technology, especially the advancement of database technology and network technology, data storage, processing and analysis have become increasingly important. In modern information systems, data is often scattered in different data sources. How to effectively integrate this data and provide users with accurate and timely information query services has become a key technical challenge.
[0003] In the related art, multiple data query services are usually called to filter data from different data sources, and then the data is merged and sorted according to the sorting rules to provide information query services for users. When the amount of data is large, the paging query method is used to query the data, and the paging display of the data is controlled by querying the offset. However, in this method, the query offset will generate a corresponding key-value pair stored in the cache, and the expired cache will be deleted regularly according to the cache strategy, making the deleted query offset unusable, resulting in low query efficiency.
[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention
[0005] The main purpose of this application is to provide a method for reconstructing a query offset, aiming to solve the technical problem of low query efficiency in related technologies.
[0006] To achieve the above object, the present application proposes a method for reconstructing a query offset, and the method for reconstructing a query offset includes:
[0007] In the case where sending a query request about the target query data to each data source based on the first query offset is abnormal, obtaining a standard unique key of the target query data;
[0008] Calling the data query service of each data source to obtain a target data set, and sequentially comparing the unique key of each data in the target data set with the standard unique key, until obtaining target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data;
[0009] The target unique key is stored in the cache as the reconstructed first query offset.
[0010] Optionally, the step of sequentially comparing the unique key of each data in the target data set with the standard unique key until obtaining the target positioning data having the same unique key as the standard unique key and the target unique key of the target positioning data comprises:
[0011] Generate a read pointer corresponding to the target data set, wherein the read pointer is used to point to the starting position of the data set;
[0012] Reading the pointing data of the target data set pointed to by the current read pointer, and comparing the unique key of the pointing data with the standard unique key;
[0013] If the unique key of the pointed data is not the same as the standard unique key, read the next pointed data of the target data set pointed to by the read pointer, return to the step of reading the pointed data of the target data set currently pointed to by the read pointer, and compare the unique key of the pointed data with the standard unique key until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained.
[0014] Optionally, the step of reading the pointing data of the target data set pointed to by the current read pointer and comparing the unique key of the pointing data with the standard unique key includes:
[0015] Get the sorting conditions;
[0016] Based on the sorting condition, sorting the target data set to obtain a sorted target data set;
[0017] Point the read pointer to the first piece of data in the sorted target data set, and use the first piece of data as current pointing data;
[0018] The unique key pointing to the data is compared with the standard unique key.
[0019] Optionally, when sending a query request about the target query data to each data source based on the first query offset is abnormal, the step of obtaining a standard unique key of the target query data includes:
[0020] In response to a query request for target query data, determining whether there is a deletion operation record for the target query data;
[0021] If there is no deletion operation record for the target query data, obtaining a first query offset of the target query data;
[0022] Based on the first query offset, the data query service of each data source is called to obtain abnormal result feedback. If the abnormal result feedback is that the query request is abnormal, the standard unique key of the target query data is obtained.
[0023] Optionally, the target data set includes multiple sub-target data sets, and the step of invoking the data query service of each data source to obtain the target data set, and sequentially comparing the target unique key of each data in the target data set with the standard unique key until the target data having the same target unique key as the standard unique key is obtained includes:
[0024] Get the initial query offset of each data source;
[0025] Based on the initial query offset, calling the data query service of each data source to obtain a first-order sub-target data set;
[0026] The unique key of each data in the sub-target data set is compared with the standard unique key in turn. If the unique key of each data in the sub-target data set is inconsistent with the standard unique key, the next sub-target data set is determined, and the step of comparing the unique key of each data in the sub-target data set with the standard unique key in turn is returned to until the target data whose target unique key is the same as the standard unique key is obtained.
[0027] Optionally, the step of sequentially comparing the unique key of each data in the sub-target data set with the standard unique key, and if the unique key of each data in the sub-target data set is inconsistent with the standard unique key, determining the sub-target data set of the next order, and returning to the step of sequentially comparing the unique key of each data in the sub-target data set with the standard unique key until the target data having the same target unique key as the standard unique key is obtained, comprises:
[0028] Generate a read pointer corresponding to the sub-target data set, wherein the read pointer is used to point to the starting position of the data set;
[0029] Reading the pointing data of the target data set pointed to by the current read pointer, and comparing the unique key of the pointing data with the standard unique key;
[0030] If the unique key of the pointed data is not the same as the standard unique key, then read the next pointed data of the target data set pointed by the read pointer, return to the step of reading the pointed data of the target data set pointed by the current read pointer, and compare the unique key of the pointed data with the standard unique key, until the unique key of each data in the sub-target data set is inconsistent with the standard unique key, then determine the sub-target data set of the next order;
[0031] The read pointer is re-pointed to the sub-target data set of the next order, and the step of comparing the unique key of the pointed data with the standard unique key is returned until the target positioning data having the same unique key as the standard unique key and the target unique key of the target positioning data are obtained.
[0032] In addition, to achieve the above purpose, the present application also proposes a query offset reconstruction device, the query offset reconstruction device comprising:
[0033] An acquisition module, configured to acquire a standard unique key of the target query data when sending a query request about the target query data to each data source based on the first query offset is abnormal;
[0034] A comparison module is used to call the data query service of each data source to obtain a target data set, and sequentially compare the unique key of each data in the target data set with the standard unique key until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained;
[0035] A storage module is used to store the target unique key as the reconstructed first query offset in a cache.
[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a query offset reconstruction device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the query offset reconstruction method as described above.
[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the query offset reconstruction method described above are implemented.
[0038] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the query offset reconstruction method described above are implemented.
[0039] One or more technical solutions proposed in this application have at least the following technical effects:
[0040] Compared with the related art, which usually calls multiple data query services to filter data from different data sources, and then merges and sorts these data according to sorting rules to provide information query services for users, and when the amount of data is large, while using paging query to query data, the paging display of data is also controlled by query offset. However, in this method, the query offset will generate a corresponding key-value pair stored in the cache, and the expired cache will be deleted regularly according to the cache strategy, so that the deleted query offset cannot be used, resulting in low query efficiency. In the present application, when a query request about target query data is sent to each data source based on the first query offset, the standard unique key of the target query data is obtained, the data query service of each data source is called, the target data set is obtained, and the unique key of each data in the target data set is compared with the standard unique key in turn, until the target positioning data with the same unique key as the standard unique key and the target unique key of the target positioning data are obtained, and the target unique key is stored in the cache as the reconstructed first query offset. It is understandable that after the first query offset of the target query data is deleted, the present application achieves reconstruction of the first query offset by comparing the unique key of each data in the target data set with the standard unique key, that is, in the scenario where the first query offset has expired and been deleted, it still supports the query requester to query subsequent data from the position of the first query offset, thereby improving query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0043] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the method for reconstructing a query offset of the present application;
[0044] Figure 2 A schematic diagram of an application scenario of the reconstruction method for querying offsets in this application;
[0045] Figure 3 A flowchart diagram of a second embodiment of a method for reconstructing a query offset in the present application;
[0046] Figure 4 A schematic diagram of the overall process of the reconstruction method for querying offsets in this application;
[0047] Figure 5 A schematic diagram of the module structure of a reconstruction device for querying offset according to an embodiment of the present application;
[0048] Figure 6 A schematic diagram of the device structure of the hardware operating environment involved in the offset query reconstruction method in the embodiment of the present application.
[0049] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0052] The main solution of the embodiment of the present application is: when a query request for target query data is sent to each data source based on a first query offset and is in an abnormal state, the standard unique key of the target query data is obtained; the data query service of each data source is called to obtain a target data set, and the unique key of each data in the target data set is compared with the standard unique key in turn until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained; the target unique key is stored in the cache as the reconstructed first query offset.
[0053] In this embodiment, a reconstruction device for querying offset is used as the execution subject. For the convenience of description, it is hereinafter referred to as "device" for explanation.
[0054] In the related art, multiple data query services are usually called to filter data from different data sources, and then the data is merged and sorted according to the sorting rules to provide information query services for users. When the amount of data is large, the paging query method is used to query the data, and the paging display of the data is controlled by the query offset. However, in this method, the query offset will generate a corresponding key-value pair stored in the cache. According to the cache strategy, the expired cache will be deleted regularly, making the deleted query offset unusable, resulting in low query efficiency.
[0055] The present application provides a solution to achieve the reconstruction of the query offset, thereby improving the query efficiency.
[0056] It can be seen from the above embodiments that, in the case where the query request for the target query data is sent to each data source based on the first query offset and is in an abnormal state, the present application obtains the standard unique key of the target query data, calls the data query service of each data source, obtains the target data set, and sequentially compares the unique key of each data in the target data set with the standard unique key until the target positioning data with the same unique key as the standard unique key and the target unique key of the target positioning data are obtained, and the target unique key is stored in the cache as the reconstructed first query offset. It is understandable that, after the first query offset of the target query data is deleted, the present application realizes the reconstruction of the first query offset by comparing the unique key of each data in the target data set with the standard unique key, that is, in the scenario where the first query offset has expired and been deleted, it still supports the query requester to query subsequent data from the position of the first query offset, thereby improving the query efficiency.
[0057] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a terminal system, etc. The following takes the reconstruction device for querying offset as an example to illustrate this embodiment and the following embodiments.
[0058] Based on this, the embodiment of the present application provides a method for reconstructing a query offset, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of a method for reconstructing an offset query in the present application.
[0059] In this embodiment, the query offset reconstruction method includes steps S100 to S300:
[0060] Step S100, when sending a query request about target query data to each data source based on the first query offset is abnormal, obtaining a standard unique key of the target query data;
[0061] In the specific implementation, refer to Figure 2, the application scenario of this application is specifically: the query requester queries data through external channels (such as bank apps, etc.). The query request is accepted by the data integration service, and multiple data query services are called respectively. The data query service filters data from multiple data sources according to the query conditions and returns it to the data integration service. The data integration service merges and sorts the data returned by multiple data query services according to the sorting rules, and returns it to the channel after completion. When the amount of data is large, a paging query method is adopted. When a page of data is fully searched, each data source has a corresponding query offset to identify how much data in the data source is included in the result of this query. The data integration service combines the query offsets of multiple data sources as a value, and then generates a random string as a key (hereinafter referred to as key), combines the key and value into a key-value pair and saves it in the cache. The data integration service returns this key and the queried data to the channel as a response to this query. When the query requester queries the next page of data, the channel integrates the query conditions and the key returned by the previous query as a request, and calls the data integration service again. The data integration service uses the key query cache to obtain the current query offset of each data source, and passes the query offset of the data source when calling the data query service, so that the data query service can continue to query backward from the offset position. The subsequent processing flow is as described above, until the query requester has queried all the data. The cache used in the query process is not infinite, and there must be a strategy for cache expiration replacement. If the cache used in a query has been replaced, the query offset of each current data source cannot be determined, resulting in the inability to return the next page of data. At this time, the query can only be prompted to fail, and the query requester is asked to start the query from the beginning. Therefore, the present application provides a solution to achieve the reconstruction of the query offset, thereby improving query efficiency.
[0062] It should be noted that the query offset (Offset) is usually used in paging queries, indicating how many records need to be skipped to start returning results when querying in a database or other data storage system. It is an important concept in the paging mechanism and is used to control the paging display of data. Specifically, the first query offset refers to the query offset used to query the target query data, and the target query data refers to the target data queried by the user. The device retrieves data from multiple data sources, each of which stores a part of the data set, or stores different types of data. The device sends query requests to these data sources to obtain the required target data.
[0063] In a specific implementation, the device sends a query request about the target query data to each data source based on the first query offset. Specifically, when an abnormal situation occurs, specifically when the query offset required to be used in the query of the target query data has been replaced, and the query offset of each current data source cannot be determined, resulting in the inability to return the data of the next page, the device obtains the standard unique key of the target query data, wherein the standard unique key refers to a database constraint on the target query data, which is used to ensure that each value in the target query data is unique.
[0064] In a specific implementation, when the device sends a query request about the target query data to each data source based on the first query offset and the query request is abnormal, the step of obtaining the standard unique key of the target query data includes:
[0065] In response to a query request for target query data, determine whether there is a deletion operation record for the target query data; if there is no deletion operation record for the target query data, obtain a first query offset for the target query data; based on the first query offset, call a data query service of each data source to obtain abnormal result feedback; if the abnormal result feedback is that the query request is abnormal, obtain a standard unique key for the target query data.
[0066] In a specific implementation, responding to a query request for target query data means that the device receives a request that is intended to query specific data (target query data), and then the device determines whether there is a deletion operation record for the target query data. That is, before performing the query, the device will check whether there is any deletion operation record for these target query data, specifically to ensure that the target query data has not been deleted. If the target query data has been deleted, the query offset cannot be reconstructed.
[0067] Furthermore, if there is no deletion operation record for the target query data, the device obtains the first query offset of the target query data, that is, if the check result shows that there is no deletion operation, the device will obtain the first query offset of the target query data, and based on the first query offset, call the data query service of each data source to obtain abnormal result feedback. If the abnormal result feedback is that the query request is abnormal, the standard unique key of the target query data is obtained, that is, when the target query data is not deleted but the request result cannot be received correctly, the standard unique key of the target query data is obtained.
[0068] In the specific implementation, this application requires that the queried data cannot be deleted. If the queried data is allowed to be deleted during the query process, it is possible that the data corresponding to the unique key sent from the external channel has been deleted. At this time, in the cache reconstruction process, it is impossible to determine the query offset of each data source by comparing the unique key, resulting in cache reconstruction failure.
[0069] Step S200, calling the data query service of each data source to obtain a target data set, and sequentially comparing the unique key of each data in the target data set with the standard unique key, until obtaining target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data;
[0070] In a specific implementation, the step of calling the data query service of each data source to obtain the target data set means that the device will send a query request to each connected data source in order to obtain the required target data. These data sources may include databases, file systems, cloud storage services, etc. Each data source executes the query and returns the target data set stored therein. Secondly, the device compares the unique key of each data in the target data set with the standard unique key in turn, that is, once the device obtains the target data set from each data source, it will start a comparison process. In this process, the device will check the unique key of each data in the target data set one by one. The "unique key" here usually refers to a field or field combination that can uniquely identify a data record, such as the primary key of a database table. Until the target positioning data with the same unique key as the standard unique key is obtained, specifically, the comparison process will continue until the device determines a piece of data whose unique key matches the pre-defined "standard unique key".
[0071] Furthermore, the target unique key of the target positioning data means that once the device finds a matching standard unique key, it will identify the corresponding target data (target positioning data) and obtain the "target unique key" of this data. This target unique key is used to identify the specific data record found.
[0072] In the specific implementation, the above process describes how to retrieve data from multiple data sources and locate specific data records by comparing the unique key of each data record. Specifically, the data query service corresponding to each data source is called to obtain the total data set S = {L1, L2, ..., Ln}, where Lj is the data obtained by querying the jth data source.
[0073] In a specific implementation, the device sequentially compares the unique key of each data in the target data set with the standard unique key until obtaining the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data, including:
[0074] Generate a read pointer corresponding to the target data set, wherein the read pointer is used to point to the starting position of the data set; read the pointing data of the target data set pointed to by the current read pointer, and compare the unique key of the pointing data with the standard unique key; if the unique key of the pointing data is different from the standard unique key, read the next pointing data of the target data set pointed to by the read pointer, return to the step of reading the pointing data of the target data set pointed to by the current read pointer, and compare the unique key of the pointing data with the standard unique key, until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained.
[0075] In a specific implementation, the device first generates a read pointer corresponding to the target data set. Specifically, the device first creates a read pointer, which is used to navigate in the target data set, that is, the read pointer is used to point to the starting position of the data set. Secondly, the device reads the pointing data of the target data set currently pointed to by the read pointer, that is, the device obtains the data record currently pointed to by the read pointer.
[0076] Furthermore, the device compares the unique key of the pointed data with the standard unique key. If the unique key of the pointed data is different from the standard unique key, that is, if the unique key of the currently pointed data record does not match the standard unique key, it means that this is not the data record the device is looking for. Then, the device reads the next pointed data of the target data set pointed by the read pointer. Specifically, the device will update the read pointer so that it points to the next data record in the data set.
[0077] Finally, the device returns to the step of reading the pointed data of the target data set pointed to by the current read pointer, and comparing the unique key of the pointed data with the standard unique key, that is, the device repeats the above steps, continues to read the data pointed to by the new pointer, and compares it until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained, that is, this process will continue until a data record is found whose unique key matches the standard unique key, that is, the device has located the target data (target positioning data) and obtained the target unique key of this data.
[0078] It is understandable that the above process is a cyclic search mechanism, which continuously moves the read pointer and compares the unique key until a data record matching the standard unique key is found.
[0079] In a specific implementation, the device reads the pointing data of the target data set pointed to by the current read pointer, and compares the unique key of the pointing data with the standard unique key, including:
[0080] The device obtains a sorting condition; based on the sorting condition, sorts the target data set to obtain a sorted target data set; points the read pointer to the first data of the sorted target data set, and uses the first data as the current pointing data; compares the unique key of the pointing data with the standard unique key.
[0081] In a specific implementation, the device obtains a sorting condition, wherein the sorting condition defines how the data should be sorted, such as by date, value, alphabetical order, etc. The purpose of sorting is mainly in the context of transaction record query, and the returned data needs to be sorted in reverse chronological order. Further, the device uses the obtained sorting condition to sort all the data in the target data set. This process will reorganize the data set so that the data is arranged in the specified order of conditions to obtain a sorted target data set, that is, after the sorting operation is completed, the device will obtain a new data set, which is arranged according to the specified sorting condition.
[0082] In a specific implementation, the device points the read pointer to the first data in the sorted target data set, that is, the device updates a read pointer to point to the first data in the sorted data set, and the device sets the data record currently pointed to by the read pointer as the current pointed data. Finally, the device compares the unique key of the pointed data with the standard unique key.
[0083] Step S300: storing the target unique key as the reconstructed first query offset in the cache.
[0084] In a specific implementation, the device stores the target unique key as the first query offset after reconstruction in the cache, that is, the target unique key that is the same as the standard unique key is the query offset determined by the requirement, and stores it as the first query offset after reconstruction in the cache, so that in the scenario where the cache has expired, the query requester is still supported to continue to turn pages from the queried page number to query subsequent data, thereby improving the query experience, and solving the problem of query failure caused by cache expiration in the multi-data source paging query scenario. The transformation has little impact on the original system, providing a better query experience for the query requester.
[0085] Compared with the related art, which usually calls multiple data query services to filter data from different data sources, and then merges and sorts these data according to sorting rules to provide information query services for users, and when the amount of data is large, while using paging query to query data, the paging display of data is also controlled by query offset. However, in this method, the query offset will generate a corresponding key-value pair stored in the cache, and the expired cache will be deleted regularly according to the cache strategy, so that the deleted query offset cannot be used, resulting in low query efficiency. In the present application, when a query request about target query data is sent to each data source based on the first query offset, the standard unique key of the target query data is obtained, the data query service of each data source is called, the target data set is obtained, and the unique key of each data in the target data set is compared with the standard unique key in turn, until the target positioning data with the same unique key as the standard unique key and the target unique key of the target positioning data are obtained, and the target unique key is stored in the cache as the reconstructed first query offset. It is understandable that after the first query offset of the target query data is deleted, the present application achieves reconstruction of the first query offset by comparing the unique key of each data in the target data set with the standard unique key, that is, in the scenario where the first query offset has expired and been deleted, it still supports the query requester to query subsequent data from the position of the first query offset, thereby improving query efficiency.
[0086] Based on the above first embodiment, the present application also proposes another embodiment, referring to Figure 3 , the query offset reconstruction method includes:
[0087] In a specific implementation, the target data set includes multiple sub-target data sets, the device calls the data query service of each data source to obtain the target data set, and sequentially compares the target unique key of each data in the target data set with the standard unique key until the target data whose target unique key is the same as the standard unique key is obtained, including:
[0088] Step A100, obtaining the initial query offset of each data source;
[0089] It should be noted that the initial query offset refers to the starting offset of each data source when a new paging query cycle starts. For the first query, the initial offset is usually 0, which means that the search starts from the first record of each data source.
[0090] In a specific implementation, the device obtains the initial query offset of each data source, that is, before preparing to query, the device needs to determine the initial offset of each data source. This can be achieved in the following ways:
[0091] Static setting: The initial offset of each data source is pre-set before the system is initialized or the query starts.
[0092] Dynamic calculation: The initial offset of each data source is dynamically calculated based on historical query data or user input.
[0093] Step A200, based on the initial query offset, calling the data query service of each data source to obtain a first-order sub-target data set;
[0094] In a specific implementation, the device calls the data query service of each data source. Specifically, the device sends a query request to each data source, wherein the query request contains an initial query offset so that the data source can determine from which position to start returning data, wherein the data query service is an interface or function provided by the data source for performing query operations and returning data. Further, the device is based on the initial query offset, that is, the query request uses the initial offset as a parameter, so that each data source can determine which data to return based on this offset. Finally, the device obtains a first-order sub-target data set, wherein the data set returned by each data source is called a sub-target data set, and the first order refers to the first batch of data in paging order, and each sub-target data set contains data from a specific data source that is filtered out according to query conditions and sorting rules.
[0095] Step A300, compare the unique key of each data in the sub-target data set with the standard unique key in turn. If the unique key of each data in the sub-target data set is inconsistent with the standard unique key, determine the next sub-target data set in sequence, and return to the step of comparing the unique key of each data in the sub-target data set with the standard unique key in turn, until the target data with the target unique key being the same as the standard unique key is obtained.
[0096] In a specific implementation, the device sequentially compares the unique key of each data in the sub-target data set with the standard unique key. Specifically, the device traverses each data record in each sub-target data set and compares its unique key with a predefined standard unique key. If the unique key of each data in the sub-target data set is inconsistent with the standard unique key, that is, if in the current sub-target data set, the unique key of all data records does not match the standard unique key, it means that there is no data to be found in the current set, and the next sub-target data set is determined. That is, in this case, the device will prepare to obtain the next sub-target data set, and return to the step of sequentially comparing the unique key of each data in the sub-target data set with the standard unique key, that is, the device will repeat the comparison process, but now it is operating on the next sub-target data set until the target data with the same target unique key as the standard unique key is obtained.
[0097] In a specific implementation, the above process is a cyclic search mechanism, which continuously compares the unique keys of data records in each sub-target data set until a data record matching the standard unique key is found.
[0098] In a specific implementation, the device sequentially compares the unique key of each data in the sub-target data set with the standard unique key. If the unique key of each data in the sub-target data set is inconsistent with the standard unique key, the next sub-target data set is determined, and the step of sequentially comparing the unique key of each data in the sub-target data set with the standard unique key is returned until the target data whose target unique key is the same as the standard unique key is obtained. The step includes:
[0099] Generate a read pointer corresponding to the sub-target data set, wherein the read pointer is used to point to the starting position of the data set; read the pointing data of the target data set pointed to by the current read pointer, and compare the unique key of the pointing data with the standard unique key; if the unique key of the pointing data is not the same as the standard unique key, read the next pointing data of the target data set pointed to by the read pointer, return to the step of reading the pointing data of the target data set pointed to by the current read pointer, and compare the unique key of the pointing data with the standard unique key, until the unique key of each data in the sub-target data set is inconsistent with the standard unique key, then determine the sub-target data set of the next order; re-point the read pointer to the sub-target data set of the next order, and return to the step of comparing the unique key of the pointing data with the standard unique key, until the target positioning data with the same unique key as the standard unique key and the target unique key of the target positioning data are obtained.
[0100] In the specific implementation, refer to Figure 4 , the specific process is as follows:
[0101] (1) Use the passed key to query the cache. If the query is successful, it is a normal query process and will not be described here. If the cache is not found and the query request contains the unique key K returned by the previous query, the following step (2) is executed.
[0102] (2) Set the query offset of each data source to the initial offset, and call the data query service corresponding to each data source in combination with other query conditions to obtain a total data set S = {L1, L2, ..., Ln}, where Lj is the data obtained by querying the jth data source;
[0103] (3) setting read pointers {P1, P2, ..., Pn} for the data sets {L1, L2, ..., Ln}, respectively pointing to the starting positions of the data sets;
[0104] (4) Read the data {D1, D2, ..., Dn} pointed to by the pointer {P1, P2, ..., Pn}, sort {D1, D2, ..., Dn} according to the sorting condition, obtain the data Di ranked first, and move the pointer Pi corresponding to the data Di backward;
[0105] (5) Let the data set where Di is located be Li. If Pi has exceeded the range of Li, the data integration service will call the data query service corresponding to Li again to obtain the next data set Li'. If Li' is not an empty set, Pi will be redirected to the starting position of Li' and Li in S will be replaced by Li'. Otherwise, Li will be directly deleted from S. If S is empty after deletion, it means that cache reconstruction has failed.
[0106] (6) Compare the unique key of data Di with K to see if they are consistent. If they are consistent, the data pointed to by the current pointer {P1, P2, ..., Pn} is the query offset of each data source. Store them in the cache and the cache is rebuilt successfully. Otherwise, return to (4).
[0107] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method for reconstructing the query offset of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.
[0108] This application also provides a query offset reconstruction device, please refer to Figure 5 , the query offset reconstruction device includes:
[0109] An acquisition module 10 is used to acquire a standard unique key of the target query data when sending a query request about the target query data to each data source based on the first query offset is abnormal;
[0110] The comparison module 20 is used to call the data query service of each data source to obtain a target data set, and sequentially compare the unique key of each data in the target data set with the standard unique key until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained;
[0111] The storage module 30 is used to store the target unique key as the reconstructed first query offset in the cache.
[0112] Optionally, the comparison module 20 includes:
[0113] A generating module, used for generating a read pointer corresponding to the target data set, wherein the read pointer is used for pointing to the starting position of the data set;
[0114] A reading module, used for reading the pointing data of the target data set pointed to by the current reading pointer, and comparing the unique key of the pointing data with the standard unique key;
[0115] A judgment module is used to read the next pointing data of the target data set pointed to by the read pointer if the unique key of the pointing data is different from the standard unique key, return to read the pointing data of the target data set pointed to by the current read pointer, and compare the unique key of the pointing data with the standard unique key, until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained.
[0116] Optionally, the reading module includes:
[0117] A sorting condition acquisition module is used to obtain sorting conditions;
[0118] A sorting module, used for sorting the target data set based on the sorting condition to obtain a sorted target data set;
[0119] A pointing module, used to point the read pointer to the first data of the sorted target data set, and use the first data as current pointing data;
[0120] A key comparison module is used to compare the unique key pointing to the data with the standard unique key.
[0121] Optionally, the acquisition module 10 includes:
[0122] A record determination module, configured to determine whether there is a deletion operation record for the target query data in response to a query request for the target query data;
[0123] A query offset acquisition module, configured to acquire a first query offset of the target query data if there is no deletion operation record for the target query data;
[0124] The calling module is used to call the data query service of each data source based on the first query offset to obtain abnormal result feedback. If the abnormal result feedback is that the query request is abnormal, the standard unique key of the target query data is obtained.
[0125] Optionally, the comparison module 20 further includes:
[0126] An initial query offset acquisition module is used to obtain the initial query offset of each data source;
[0127] A query service calling module, used to call the data query service of each data source based on the initial query offset to obtain a first-order sub-target data set;
[0128] A loop judgment module is used to compare the unique key of each data in the sub-target data set with the standard unique key in turn. If the unique key of each data in the sub-target data set is inconsistent with the standard unique key, the next sub-target data set is determined, and the step of comparing the unique key of each data in the sub-target data set with the standard unique key in turn is returned until the target data whose target unique key is the same as the standard unique key is obtained.
[0129] Optionally, the cycle determination module includes:
[0130] A read pointer generation module, used to generate a read pointer corresponding to the sub-target data set, wherein the read pointer is used to point to the starting position of the data set;
[0131] A pointing data reading module, used for reading the pointing data of the target data set pointed to by the current reading pointer, and comparing the unique key of the pointing data with the standard unique key;
[0132] A circular reading module, for reading the next pointed data of the target data set pointed by the read pointer if the unique key of the pointed data is not the same as the standard unique key, returning to the step of reading the pointed data of the target data set pointed by the current read pointer, and comparing the unique key of the pointed data with the standard unique key, until the unique key of each data in the sub-target data set is inconsistent with the standard unique key, then determining the sub-target data set of the next order;
[0133] The unique key comparison module is used to re-point the read pointer to the sub-target data set of the next order, and return to the step of comparing the unique key of the pointed data with the standard unique key until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained.
[0134] The query offset reconstruction device provided by the present application adopts the query offset reconstruction method in the above embodiment, which can solve the technical problem of query offset reconstruction. Compared with the prior art, the beneficial effect of the query offset reconstruction device provided by the present application is the same as the beneficial effect of the query offset reconstruction method provided by the above embodiment, and other technical features in the query offset reconstruction device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0135] The present application provides a query offset reconstruction device, the query offset reconstruction device comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the query offset reconstruction method in the above-mentioned embodiment 1.
[0136] Reference below Figure 6 , which shows a schematic diagram of a structure of a query offset reconstruction device suitable for implementing an embodiment of the present application. The query offset reconstruction device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The reconstruction device for querying the offset shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0137] like Figure 6As shown, the reconstruction device for querying the offset may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the reconstruction device for querying the offset are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the reconstruction device for querying offset to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows the reconstruction device for querying offset with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.
[0138] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0139] The query offset reconstruction device provided by the present application adopts the query offset reconstruction method in the above embodiment, which can solve the technical problem of query offset reconstruction. Compared with the prior art, the beneficial effect of the query offset reconstruction device provided by the present application is the same as the beneficial effect of the query offset reconstruction method provided by the above embodiment, and other technical features in the query offset reconstruction device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0140] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0141] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0142] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the query offset reconstruction method in the above-mentioned embodiment.
[0143] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0144] The computer-readable storage medium may be included in the reconstruction device for querying the offset; or may exist independently without being assembled into the reconstruction device for querying the offset.
[0145] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the reconstruction device for querying the offset, the reconstruction device for querying the offset can: reconstruct the query offset.
[0146] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0148] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0149] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned query offset reconstruction method, and can solve the technical problem of query offset reconstruction. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the query offset reconstruction method provided in the above-mentioned embodiment, and will not be repeated here.
[0150] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned query offset reconstruction method when executed by a processor.
[0151] The computer program product provided by the present application can solve the technical problem of reconstructing the query offset. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the query offset reconstruction method provided by the above embodiment, which will not be repeated here.
[0152] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A method for reconstructing a query offset, characterized in that: The method for reconstructing the query offset includes: In the case where sending a query request about the target query data to each data source based on the first query offset is abnormal, obtaining a standard unique key of the target query data; Calling the data query service of each data source to obtain a target data set, and sequentially comparing the unique key of each data in the target data set with the standard unique key, until obtaining target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data; The target unique key is stored in the cache as the reconstructed first query offset.
2. The method for reconstructing the query offset according to claim 1, characterized in that: The step of sequentially comparing the unique key of each data in the target data set with the standard unique key until obtaining the target positioning data having the same unique key as the standard unique key and the target unique key of the target positioning data comprises: Generate a read pointer corresponding to the target data set, wherein the read pointer is used to point to the starting position of the data set; Reading the pointing data of the target data set pointed to by the current read pointer, and comparing the unique key of the pointing data with the standard unique key; If the unique key of the pointed data is not the same as the standard unique key, read the next pointed data of the target data set pointed to by the read pointer, return to the step of reading the pointed data of the target data set currently pointed to by the read pointer, and compare the unique key of the pointed data with the standard unique key until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained.
3. The method for reconstructing the query offset according to claim 2, characterized in that: The step of reading the pointing data of the target data set pointed to by the current read pointer and comparing the unique key of the pointing data with the standard unique key includes: Get sorting conditions; Based on the sorting condition, sorting the target data set to obtain a sorted target data set; Point the read pointer to the first piece of data in the sorted target data set, and use the first piece of data as current pointing data; The unique key pointing to the data is compared with the standard unique key.
4. The method for reconstructing a query offset as claimed in claim 1, characterized in that: The step of acquiring the standard unique key of the target query data when sending a query request about the target query data to each data source based on the first query offset is abnormal includes: In response to a query request for target query data, determining whether there is a deletion operation record for the target query data; If there is no deletion operation record for the target query data, obtaining a first query offset of the target query data; Based on the first query offset, the data query service of each data source is called to obtain abnormal result feedback. If the abnormal result feedback is that the query request is abnormal, the standard unique key of the target query data is obtained.
5. The method for reconstructing a query offset as claimed in claim 1, characterized in that: The target data set includes a plurality of sub-target data sets, and the step of calling the data query service of each data source to obtain the target data set, and sequentially comparing the target unique key of each data in the target data set with the standard unique key until the target data whose target unique key is the same as the standard unique key is obtained includes: Get the initial query offset of each data source; Based on the initial query offset, calling the data query service of each data source to obtain a first-order sub-target data set; The unique key of each data in the sub-target data set is compared with the standard unique key in turn. If the unique key of each data in the sub-target data set is inconsistent with the standard unique key, the next sub-target data set is determined, and the step of comparing the unique key of each data in the sub-target data set with the standard unique key in turn is returned to until the target data whose target unique key is the same as the standard unique key is obtained.
6. The method for reconstructing the query offset according to claim 5, characterized in that: The step of sequentially comparing the unique key of each data in the sub-target data set with the standard unique key, and if the unique key of each data in the sub-target data set is inconsistent with the standard unique key, determining the sub-target data set of the next order, and returning to the step of sequentially comparing the unique key of each data in the sub-target data set with the standard unique key until the target data having the same target unique key as the standard unique key is obtained, comprises: Generate a read pointer corresponding to the sub-target data set, wherein the read pointer is used to point to the starting position of the data set; Reading the pointing data of the target data set pointed to by the current read pointer, and comparing the unique key of the pointing data with the standard unique key; If the unique key of the pointed data is not the same as the standard unique key, then read the next pointed data of the target data set pointed by the read pointer, return to the step of reading the pointed data of the target data set currently pointed by the read pointer, and compare the unique key of the pointed data with the standard unique key, until the unique key of each data in the sub-target data set is inconsistent with the standard unique key, then determine the sub-target data set of the next order; The read pointer is re-pointed to the sub-target data set of the next order, and the step of comparing the unique key of the pointed data with the standard unique key is returned until the target positioning data having the same unique key as the standard unique key and the target unique key of the target positioning data are obtained.
7. A device for reconstructing a query offset, characterized in that: The device comprises: An acquisition module, configured to acquire a standard unique key of the target query data when sending a query request about the target query data to each data source based on the first query offset is abnormal; A comparison module is used to call the data query service of each data source to obtain a target data set, and sequentially compare the unique key of each data in the target data set with the standard unique key until the target positioning data whose unique key is the same as the standard unique key and the target unique key of the target positioning data are obtained; A storage module is used to store the target unique key as the reconstructed first query offset in a cache.
8. A reconstruction device for querying offset, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the query offset reconstruction method according to any one of claims 1 to 6.
9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the query offset reconstruction method according to any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method for reconstructing the query offset according to any one of claims 1 to 6 are implemented.