Method and apparatus for accessing data

Abstract

The application provides a data access method and device, wherein the data access method comprises the following steps: analyzing a data access request to obtain a data target interface; determining a priority corresponding to the data target interface based on a stored data interface priority sequence; accessing data of the data target interface in a descending order of the priority; and updating the priority of the data interface in the data interface priority sequence according to incremental data corresponding to the accessed data target interface and a data interface update strategy. The timeliness of hotspot data access can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a method and apparatus for accessing data. Background Technology

[0002] Enterprises generate massive amounts of data during their production and operations, such as WeChat conversations, email communications, and machine operation data. In the era of big data, setting up an enterprise data platform and using interfaces to collect this data significantly impacts digital management and decision-making. Currently, data access to the data platform typically involves iterating through interfaces to access incremental or full data. However, for business scenarios with high real-time requirements, accessing hot data corresponding to interfaces through this method can be time-consuming, failing to meet the time-sensitive access needs of such business scenarios. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method and apparatus for accessing data, so as to improve the timeliness of accessing hotspot data.

[0004] In a first aspect, embodiments of the present invention provide a method for accessing data, including:

[0005] Parse the acquired data access request to obtain the target interface for the requested data access;

[0006] The priority of the target data interface is determined based on the stored data interface priority sequence.

[0007] Access the data from the corresponding target interface in descending order of priority.

[0008] Based on the incremental data corresponding to each data target interface that has been accessed and the update strategy of the data interface that has not requested data access, the priority of the data interface in the data interface priority sequence is updated.

[0009] In conjunction with the first aspect, embodiments of the present invention provide a first possible implementation of the first aspect, wherein determining the priority corresponding to the data target interface based on the stored data interface priority sequence includes:

[0010] For each data target interface, if the data target interface is included in the stored data interface priority sequence, the priority of the data target interface is obtained from the data interface priority sequence; if the data target interface is not included in the stored data interface priority sequence, the priority of the data target interface is set to the preset initial priority.

[0011] In conjunction with the first aspect, this embodiment of the invention provides a second possible implementation of the first aspect, wherein the step of accessing the data of the corresponding data target interface in descending order of acquisition priority includes:

[0012] Each thread in the concurrent thread set is assigned to a data target interface. Data target interfaces with higher priority receive more computing resources for the corresponding threads.

[0013] In conjunction with the first aspect, the first possible implementation of the first aspect, or the second possible implementation of the first aspect, this embodiment of the invention provides a third possible implementation of the first aspect, wherein updating the priority of the data interfaces in the data interface priority sequence based on the incremental data corresponding to each accessed data target interface and the update strategy of the data interfaces that have not requested data access includes:

[0014] Determine whether each data target interface has received incremental data. If so, increase the priority of the first data interface with incremental data in the data interface priority sequence. If not, decrease the priority of the second data interface without incremental data according to the first threshold.

[0015] From the data interface priority sequence, the priority of data interfaces other than the first data interface and the second data interface is reduced according to a second threshold, wherein the first threshold is less than the second threshold.

[0016] In conjunction with the third possible implementation of the first aspect, this embodiment of the invention provides a fourth possible implementation of the first aspect, wherein determining whether each data target interface has received incremental data includes:

[0017] Obtain the incremental data identifier of the target data interface. If the incremental data identifier is an identifier that supports incremental data retrieval, and if data is received from the target data interface, it is determined that the target data interface has received incremental data.

[0018] If the incremental data identifier is one that does not support incremental data retrieval, the data from the target interface is received, a hash operation is performed on the received data to obtain the current hash value, and the historical hash value of the target interface is queried. If the current hash value is inconsistent with the historical hash value, it is determined that the target interface has received incremental data, and the historical hash value is updated using the current hash value.

[0019] In conjunction with the first aspect, the first possible implementation of the first aspect, or the second possible implementation of the first aspect, this embodiment of the invention provides a fifth possible implementation of the first aspect, wherein, after updating the priority of the data interfaces in the data interface priority sequence, the method further includes:

[0020] Get the data interfaces in the data interface priority sequence that are not the lowest priority. If the data interface has a timestamp, delete the timestamp of the data interface.

[0021] Obtain the data interface with the lowest priority in the data interface priority sequence. If the data interface does not have a timestamp, timestamp the data interface. If the data interface has a timestamp, calculate the time difference between the timestamp and the current timestamp. If the time difference is greater than a preset time difference threshold, delete the data interface from the data interface priority sequence.

[0022] In conjunction with the first aspect, the first possible implementation of the first aspect, or the second possible implementation of the first aspect, this embodiment of the invention provides a sixth possible implementation of the first aspect, wherein the method further includes:

[0023] For data interfaces with a higher priority than the preferred priority in the data interface priority sequence, a corresponding active fetch frequency is set.

[0024] Secondly, embodiments of the present invention also provide a data access device, comprising:

[0025] The parsing module is used to parse the acquired data access request and obtain the target data interface for the requested data access;

[0026] The query module is used to determine the priority of the target data interface based on the stored data interface priority sequence;

[0027] The data access module is used to access the data from the corresponding data target interface in descending order of acquisition priority.

[0028] The update module is used to update the priority of the data interfaces in the data interface priority sequence based on the incremental data corresponding to each data target interface and the update strategy of the data interfaces that have not requested data access.

[0029] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0030] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described method.

[0031] The data access method and apparatus provided in this invention parse the acquired data access request to obtain the data target interface requesting data access; determine the priority of the data target interface based on a stored data interface priority sequence; access the data of the corresponding data target interface in descending order of the acquired priority; and update the priority of the data interfaces in the data interface priority sequence according to the incremental data corresponding to each accessed data target interface and the update strategy of data interfaces that have not requested data access. In this way, by updating the priority of the data target interface based on the presence or absence of incremental data returned by the data target interface, the data interfaces for hot data can be prioritized, thereby ensuring the timeliness of the acquired hot data.

[0032] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This diagram illustrates a method flow chart for accessing data provided in an embodiment of the present invention.

[0035] Figure 2 This diagram illustrates the priority queue update of the cascading interface provided in an embodiment of the present invention.

[0036] Figure 3 A schematic diagram of the device structure for accessing data provided in an embodiment of the present invention is shown;

[0037] Figure 4 This is a schematic diagram of the structure of a computer device 400 provided in an embodiment of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0039] This invention provides a method and apparatus for accessing data, which will be described below through embodiments.

[0040] Figure 1 A schematic flowchart of the data access method provided in an embodiment of the present invention is shown. Figure 1 As shown, the method includes:

[0041] Step 101: Parse the acquired data access request to obtain the target data interface for the requested data access;

[0042] In one embodiment of the present invention, as an optional implementation, the interface caller, the data platform, pulls data access requests from the main data interface requiring data access according to a set data access cycle, parses the pulled data access requests, and obtains the target data interface requiring data access. Alternatively, in another optional implementation, the main data interface requiring data access can send a data access request to the interface caller when it has a data access need; the interface caller then parses the data access request to obtain the target data interface requiring data access.

[0043] In this embodiment of the invention, a data interface with data access requirements refers to a data interface that needs to transfer the received data to a server for data storage, analysis and processing. The server can be the aforementioned interface caller, such as a data platform with a remote dictionary server (Redis).

[0044] In this embodiment of the invention, taking the email interface as an example, the main data interface requiring data access can be called the primary interface, and the target data interface (user email interface) can be called the cascading interface. The primary interface includes multiple cascading interfaces. The email interface can be a Microsoft Graph interface. Through this Microsoft Graph interface, the email interfaces of various users in Outlook, i.e., the users interface, can be obtained to retrieve the corresponding user's emails, calendar events, contacts, and other information. In this embodiment of the invention, the user ID can be obtained through the users interface, and then the user's emails can be retrieved according to the users / {user ID} / message interface.

[0045] Step 102: Determine the priority of the target data interface based on the stored data interface priority sequence;

[0046] In this embodiment of the invention, as an optional implementation, the data interface priority sequence is stored in Redis of the data platform, including multiple data interfaces and the priority corresponding to each data interface. It should be noted that other storage structures can also be used to store the data interface priority sequence, and this embodiment of the invention does not limit this.

[0047] In this embodiment of the invention, as an optional embodiment, determining the priority corresponding to the data target interface based on the stored data interface priority sequence includes:

[0048] For each data target interface, if the data target interface is included in the stored data interface priority sequence, the priority of the data target interface is obtained from the data interface priority sequence; if the data target interface is not included in the stored data interface priority sequence, the priority of the data target interface is set to the preset initial priority.

[0049] Step 103: Access the data from the corresponding data target interface in descending order of priority.

[0050] In this embodiment of the invention, for cases with the same priority, as an optional embodiment, data from data interfaces with the same priority are accessed in a random order.

[0051] In this embodiment of the invention, the interface caller pulls data from the first-level interface, parses it to obtain the latest cascading interfaces, and stores it in Redis. After obtaining the sorted cascading interfaces, incremental data for each cascading interface is requested using a distributed multi-node or multi-threaded concurrent approach. For example, in a data platform where data access can be performed concurrently by threads, as an optional embodiment, data from the corresponding target data interface is accessed in descending order of priority, including:

[0052] Each thread in the concurrent thread set is assigned to a data target interface. Data target interfaces with higher priority receive more computing resources for the corresponding threads.

[0053] In this embodiment of the invention, as an optional embodiment, if the number of threads in the concurrent thread set is less than the number of data target interfaces, threads are configured according to the priority order of the data target interfaces, and the higher the priority of the data target interface, the more computing resources are allocated to the configured threads. If the number of threads in the concurrent thread set is greater than or equal to the number of data target interfaces, one thread is configured for each data target interface, and the higher the priority of the data target interface, the more computing resources are allocated to the configured threads.

[0054] Step 104: Update the priority of the data interfaces in the data interface priority sequence according to the incremental data corresponding to each data target interface and the update strategy of the data interface that has not requested data access.

[0055] In this embodiment of the invention, as an optional implementation, the priority of the data interfaces in the data interface priority sequence is updated based on the incremental data corresponding to each accessed data target interface and the update strategy of the data interfaces that have not requested data access, including:

[0056] Determine whether each data target interface has received incremental data. If so, increase the priority of the first data interface with incremental data in the data interface priority sequence. If not, decrease the priority of the second data interface without incremental data according to the first threshold.

[0057] From the data interface priority sequence, the priority of data interfaces other than the first data interface and the second data interface is reduced according to a second threshold, wherein the first threshold is less than the second threshold.

[0058] In this embodiment of the invention, if the first data interface is not stored in the data interface priority sequence, the first data interface and its corresponding priority are written into the data interface priority sequence.

[0059] In one embodiment of the present invention, as an optional implementation, the lowest priority of a data interface is its initial priority, for example, zero. When the priority of a data interface is reduced to zero, its priority will not be reduced further. Alternatively, for data target interfaces without incremental data access, their priority may not be reduced.

[0060] In this embodiment of the invention, as an optional embodiment, determining whether each data target interface has received incremental data includes:

[0061] Obtain the incremental data identifier of the target data interface. If the incremental data identifier is an identifier that supports incremental data retrieval, and if data is received from the target data interface, it is determined that the target data interface has received incremental data.

[0062] If the incremental data identifier is one that does not support incremental data retrieval, the data from the target interface is received, a hash operation is performed on the received data to obtain the current hash value, and the historical hash value of the target interface is queried. If the current hash value is inconsistent with the historical hash value, it is determined that the target interface has received incremental data, and the historical hash value is updated using the current hash value.

[0063] In this embodiment of the invention, for cases where the cascading interface only supports full data retrieval, the data of the cascading interface is obtained, and a hash operation is performed on the obtained data (current hash value). This hash value is then compared with the hash value of the data previously obtained by the cascading interface (historical hash value). If the hash values ​​are the same, it is determined that the cascading interface has no hot data and has not connected any incremental data, and a penalty is imposed. If the hash values ​​are different, a reward is given, thereby updating the priority of the cascading interface.

[0064] In this embodiment of the invention, by distinguishing whether the data target interface (cascaded interface) supports incremental data retrieval, if incremental data retrieval is supported, the accessed incremental data is determined to be hot data, and the retrieval priority of the cascaded interface is updated according to the size of the incremental data; if incremental data retrieval is not supported, a hash operation is performed on the data before and after accessing the cascaded interface to determine whether the currently accessed data is hot data, and the retrieval priority of the cascaded interface is updated according to the number of times the hash values ​​are inconsistent.

[0065] In this embodiment of the invention, the priority of a cascading interface is set based on the amount of incremental data returned by the cascading interface or the presence or absence of incremental data. When incremental data is present, the cascading interface is rewarded, and its priority is increased. When no incremental data is present, the cascading interface is penalized, and its priority is decreased until it reaches 0. In this way, by dynamically updating the fetch priority through multiple data accesses (cold starts), cascading interfaces for hotspots can be prioritized, thereby ensuring the timeliness of the acquired hotspot data.

[0066] In this embodiment of the invention, as another optional embodiment, the priority of the data interfaces in the data interface priority sequence is updated based on the incremental data corresponding to each accessed data target interface and the update strategy of the data interfaces that have not requested data access, including:

[0067] For each data target interface, obtain the incremental data volume connected to the data target interface, query the mapping relationship between the incremental data volume and the priority upgrade number, and upgrade the priority of the data target interface according to the priority upgrade number obtained from the query.

[0068] From the data interface priority sequence, reduce the priority of data interfaces other than the data target interface.

[0069] In this embodiment of the invention, the lowest priority of a data interface is zero. When the priority of a data interface is reduced to zero, the priority of that data interface will not be reduced further.

[0070] In this embodiment of the invention, to avoid the increasing number of data interfaces stored in the data interface priority sequence, which would prolong the time required for querying, as an optional embodiment, after updating the priority of the data interfaces in the data interface priority sequence, the method further includes:

[0071] Get the data interfaces in the data interface priority sequence that are not the lowest priority. If the data interface has a timestamp, delete the timestamp of the data interface.

[0072] Obtain the data interface with the lowest priority in the data interface priority sequence. If the data interface does not have a timestamp, timestamp the data interface. If the data interface has a timestamp, calculate the time difference between the timestamp and the current timestamp. If the time difference is greater than a preset time difference threshold, delete the data interface from the data interface priority sequence.

[0073] In this embodiment of the invention, the priority of the cascading interface is calculated by counting the number of times the cascading interface appears within a preset time-to-live threshold (time difference threshold). This avoids cascading interfaces that are hotspots remaining at the top of the list even after they have cooled down.

[0074] In this embodiment of the invention, during data access, by probing the hot interfaces that generate hot data, setting request priorities or increasing the request frequency for the hot interfaces, the cascading interfaces that generate hot data are prioritized at the front of the queue and thus requested first, thereby improving the timeliness of hot data access.

[0075] Figure 2 This diagram illustrates a cascading interface priority queue update provided in an embodiment of the present invention. Figure 2As shown, assume the initial cascading interface priority queue contains five cascading interfaces: A, B, C, D, and E. Assume that each round iterates through all cascading interfaces, and the total time required for each round is the same. Taking data access through cascading interface C as an example, in the first round of data access, when pulling data from the cascading interface for the first time, data from cascading interface A is accessed first, followed by data from cascading interface C, which is the third to last. Assume that after two rounds, the priority of cascading interface D is updated to 10, the priority of cascading interface C to 8, the priority of cascading interface A to 5, the priority of cascading interface E to 2, and the priority of cascading interface B to 0. Therefore, in the next round of data access, data from cascading interface D is accessed first, followed by data from cascading interface C. Data from cascade interface C is accessed first, making cascade interface C a high-priority cascade interface. This increases its priority in the cascade interface priority queue, resulting in a shorter data access interval in this round compared to the first round. Assuming that after several more rounds, the priority of cascade interface C is updated to 25, cascade interface E to 18, cascade interface D to 15, cascade interface B to 6, and cascade interface A to 1, then in the next round of data access, data from cascade interface C will be accessed first.

[0076] In this embodiment of the invention, as an optional embodiment, the method may further include:

[0077] For data interfaces with a higher priority than the preferred priority in the data interface priority sequence, a corresponding active fetch frequency is set.

[0078] In this embodiment of the invention, by setting a higher active fetch frequency for data interfaces with higher priority than preferred data, the timeliness of hot data can be effectively improved for cascaded interfaces that frequently generate hot data.

[0079] In this embodiment of the invention, the data platform needs to store the data it receives. As an optional embodiment, the method further includes:

[0080] Data pulled from data interfaces with higher priority than the preferred priority is stored in the first storage area, and data pulled from data interfaces with lower priority than the preferred priority is stored in the second storage area. The data read / write speed of the first storage area is set to be greater than that of the second storage area.

[0081] Based on the priority of the updated data interface, update the data stored in the first storage area and the second storage area.

[0082] Figure 3A schematic diagram of the device structure for accessing data provided in an embodiment of the present invention is shown. Figure 3 As shown, the device includes:

[0083] Parsing module 301 is used to parse the acquired data access request and obtain the data target interface for the requested data access;

[0084] The query module 302 is used to determine the priority of the target data interface based on the stored data interface priority sequence;

[0085] In this embodiment of the invention, as an optional embodiment, the query module 302 is specifically used for:

[0086] For each data target interface, if the data target interface is included in the stored data interface priority sequence, the priority of the data target interface is obtained from the data interface priority sequence; if the data target interface is not included in the stored data interface priority sequence, the priority of the data target interface is set to the preset initial priority.

[0087] The data access module 303 is used to access the data of the corresponding data target interface in descending order of acquisition priority.

[0088] In this embodiment of the invention, as an optional embodiment, the data access module 303 is specifically used for:

[0089] Each thread in the concurrent thread set is assigned to a data target interface. Data target interfaces with higher priority receive more computing resources for the corresponding threads.

[0090] The update module 304 is used to update the priority of the data interfaces in the data interface priority sequence based on the incremental data corresponding to each data target interface and the update strategy of the data interface that has not requested data access.

[0091] In this embodiment of the invention, as an optional embodiment, the update module 304 is specifically used for:

[0092] Determine whether each data target interface has received incremental data. If so, increase the priority of the first data interface with incremental data in the data interface priority sequence. If not, decrease the priority of the second data interface without incremental data according to the first threshold.

[0093] From the data interface priority sequence, the priority of data interfaces other than the first data interface and the second data interface is reduced according to a second threshold, wherein the first threshold is less than the second threshold.

[0094] In this embodiment of the invention, as an optional embodiment, determining whether each data target interface has received incremental data includes:

[0095] Obtain the incremental data identifier of the target data interface. If the incremental data identifier is an identifier that supports incremental data retrieval, and if data is received from the target data interface, it is determined that the target data interface has received incremental data.

[0096] If the incremental data identifier is one that does not support incremental data retrieval, the data from the target interface is received, a hash operation is performed on the received data to obtain the current hash value, and the historical hash value of the target interface is queried. If the current hash value is inconsistent with the historical hash value, it is determined that the target interface has received incremental data, and the historical hash value is updated using the current hash value.

[0097] In this embodiment of the invention, as another optional embodiment, the update module 304 is specifically used for:

[0098] For each data target interface, obtain the incremental data volume connected to the data target interface, query the mapping relationship between the incremental data volume and the priority upgrade number, and upgrade the priority of the data target interface according to the priority upgrade number obtained from the query.

[0099] From the data interface priority sequence, reduce the priority of data interfaces other than the data target interface.

[0100] In an optional embodiment of the present invention, the device further includes:

[0101] The marking module (not shown in the figure) is used to obtain the data interfaces in the data interface priority sequence that are not the lowest priority. If the data interface has a timestamp mark, the timestamp mark of the data interface is deleted.

[0102] Obtain the data interface with the lowest priority in the data interface priority sequence. If the data interface does not have a timestamp, timestamp the data interface. If the data interface has a timestamp, calculate the time difference between the timestamp and the current timestamp. If the time difference is greater than a preset time difference threshold, delete the data interface from the data interface priority sequence.

[0103] In another optional embodiment of the present invention, the device further includes:

[0104] The fetch frequency update module is used to set the corresponding active fetch frequency for data interfaces with a higher priority than the preferred priority in the data interface priority sequence.

[0105] like Figure 4As shown, one embodiment of this application provides a computer device 400 for performing... Figure 1 The device includes a memory 401, a processor 402 connected to the memory 401 via a bus, and a computer program stored on the memory 401 and executable on the processor 402, wherein the processor 402 executes the computer program to implement the steps of the above-mentioned data access method.

[0106] Specifically, the memory 401 and processor 402 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 402 runs the computer program stored in the memory 401, it can execute the above-mentioned method for accessing data.

[0107] Corresponding to Figure 1 In addition to the method for accessing data, this application also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the steps of the above-described method for accessing data.

[0108] Specifically, the storage medium can be a general-purpose storage medium, such as a removable disk or hard disk, and when the computer program on the storage medium is run, it can execute the above-mentioned method for accessing data.

[0109] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0112] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0114] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for accessing data, characterized in that, include: Parse the acquired data access request to obtain the target interface for the requested data access; The priority of the target data interface is determined based on the stored data interface priority sequence. Access the data from the corresponding target interface in descending order of priority. Based on the incremental data corresponding to each data target interface that has been accessed and the update strategy of the data interface that has not requested data access, the priority of the data interface in the data interface priority sequence is updated. The step of updating the priority of data interfaces in the data interface priority sequence based on the incremental data corresponding to each accessed data target interface and the update strategy of data interfaces that have not requested data access includes: Determine whether each data target interface has received incremental data. If so, increase the priority of the first data interface with incremental data in the data interface priority sequence. If not, decrease the priority of the second data interface without incremental data according to the first threshold. From the data interface priority sequence, the priority of data interfaces other than the first data interface and the second data interface is reduced according to a second threshold, wherein the first threshold is less than the second threshold; After updating the priority of the data interfaces in the data interface priority sequence, the method further includes: Get the data interfaces in the data interface priority sequence that are not the lowest priority. If the data interface has a timestamp, delete the timestamp of the data interface. Obtain the data interface with the lowest priority in the data interface priority sequence. If the data interface does not have a timestamp, timestamp the data interface. If the data interface has a timestamp, calculate the time difference between the timestamp and the current timestamp. If the time difference is greater than a preset time difference threshold, delete the data interface from the data interface priority sequence.

2. The method according to claim 1, characterized in that, The storage-based data interface priority sequence determines the priority of the target data interface, including: For each data target interface, if the data target interface is included in the stored data interface priority sequence, the priority of the data target interface is obtained from the data interface priority sequence; if the data target interface is not included in the stored data interface priority sequence, the priority of the data target interface is set to the preset initial priority.

3. The method according to claim 1, characterized in that, The process of accessing data to the corresponding data target interface in descending order of priority includes: Each thread in the concurrent thread set is assigned to a data target interface. Data target interfaces with higher priority receive more computing resources for the corresponding threads.

4. The method according to claim 1, characterized in that, The determination of whether each data target interface has received incremental data includes: Obtain the incremental data identifier of the target data interface. If the incremental data identifier is an identifier that supports incremental data retrieval, and if data is received from the target data interface, it is determined that the target data interface has received incremental data. If the incremental data identifier is one that does not support incremental data retrieval, the data from the target interface is received, a hash operation is performed on the received data to obtain the current hash value, and the historical hash value of the target interface is queried. If the current hash value is inconsistent with the historical hash value, it is determined that the target interface has received incremental data, and the historical hash value is updated using the current hash value.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: For data interfaces with a higher priority than the preferred priority in the data interface priority sequence, a corresponding active fetch frequency is set.

6. A device for accessing data, characterized in that, include: The parsing module is used to parse the acquired data access request and obtain the target data interface for the requested data access; The query module is used to determine the priority of the target data interface based on the stored data interface priority sequence; The data access module is used to access the data from the corresponding data target interface in descending order of acquisition priority. The update module is used to update the priority of the data interfaces in the data interface priority sequence based on the incremental data corresponding to each data target interface and the update strategy of the data interfaces that have not requested data access. The update module is specifically used for: Determine whether each data target interface has received incremental data. If so, increase the priority of the first data interface with incremental data in the data interface priority sequence. If not, decrease the priority of the second data interface without incremental data according to the first threshold. From the data interface priority sequence, the priority of data interfaces other than the first data interface and the second data interface is reduced according to a second threshold, wherein the first threshold is less than the second threshold; The device also includes a marking module, which is used for: Get the data interfaces in the data interface priority sequence that are not the lowest priority. If the data interface has a timestamp, delete the timestamp of the data interface. Obtain the data interface with the lowest priority in the data interface priority sequence. If the data interface does not have a timestamp, timestamp the data interface. If the data interface has a timestamp, calculate the time difference between the timestamp and the current timestamp. If the time difference is greater than a preset time difference threshold, delete the data interface from the data interface priority sequence.

7. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the data access method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method for accessing data as described in any one of claims 1 to 5.

Images