Data interaction transfer method and data transfer system

Abstract

The invention provides a data interaction transfer method and a data transfer system, and relates to the field of data storage. According to the invention, the interaction data of each to-be-interacted system is stored in a centralized manner through the data transfer system, so that a data request from the to-be-interacted system or a client for each to-be-interacted system can be quickly responded to the outside; the situation that the system performance is reduced due to the fact that the API response time is prolonged due to the fact that interactive data between the systems is increased sharply and the number of interactive systems is increased is avoided, and user experience is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of data storage, and in particular to a data interactive transfer method and a data transfer system. Background Art

[0002] In the current technological environment, with the continuous expansion and diversification of corporate business, data resource sharing and interaction between different systems or applications are becoming increasingly frequent.

[0003] At present, the most common interaction method is to directly call the third-party API (Application Programming Interface) interface through HTTP (Hyper Text Transfer Protocol) request to obtain the required data, and then perform corresponding processing and format conversion to meet the business display requirements of its own system. For example, when system A needs to use the data of system B, system A will directly call the third-party API interface provided by system B to obtain the data, and then system A will make necessary format adjustments to the obtained data based on its own business logic to ensure that the data can meet the requirements of its business display.

[0004] When there is less data interaction between systems, directly calling the third-party API interface of the other system is an effective and feasible method. However, when the interactive data between systems increases dramatically and the number of interactive systems increases, direct calls between systems may cause the API response time to increase, affecting system performance and user experience. Summary of the invention

[0005] The purpose of the present invention is to provide a data interactive transfer method and a data transfer system to improve the problems existing in the prior art.

[0006] The embodiments of the present invention can be implemented as follows:

[0007] In a first aspect, the present invention provides a data interactive transfer method, which is applied to a data transfer system, wherein the data transfer system includes a data acquisition component, a consumption service, a response component, a database, and a cache library; the consumption service maintains a message queue; the method includes:

[0008] The data acquisition component periodically calls the third-party API interface of each system to be interacted with to obtain interaction data, and stores the interaction data in the message queue;

[0009] The consumer service performs version processing on the data in the message queue and writes the data into the database and the cache library;

[0010] When receiving a data request from any of the systems or clients to be interacted with, the response component searches out response data from the cache or database based on the data request and returns the response data to the system or client to be interacted with that issued the data request.

[0011] In an optional implementation, the data acquisition component maintains a thread pool, the thread pool includes multiple threads; the step of the data acquisition component periodically calling a third-party API interface of each system to be interacted with to obtain interaction data, and storing the interaction data in the message queue includes:

[0012] For each of the systems to be interacted with, the data acquisition component periodically determines the number of available threads and the single-thread acquisition amount based on a preset total acquisition amount and the real-time idle thread amount of the thread pool;

[0013] The data acquisition component randomly selects the number of available threads from all idle threads in the thread pool, and uses each selected thread as a calling thread of the system to be interacted with;

[0014] Each calling thread generates a data acquisition request based on the ranking of its own thread program number among all calling threads, the third-party API interface of the system to be interacted with, and the single-thread acquisition amount, and sends the data acquisition request to the system to be interacted with;

[0015] When receiving the interaction data returned by the to-be-interacted system, each calling thread stores the received interaction data in the message queue.

[0016] In an optional implementation, the interaction data includes the interface number of the system to be interacted with; the interaction data stored in the database and the cache storehouse both have associated version information; the version information includes the interface number and version number of the system to be interacted with to which the associated interaction data belongs;

[0017] The step of the consumer service performing version processing on the data in the message queue and then writing the data into the database and the cache library comprises:

[0018] The consumer service reads a copy of the interaction data to be stored from the message queue;

[0019] The consumer service searches the cache for current version information of the interaction data to be stored based on the interface number carried by the interaction data to be stored;

[0020] The consumption service determines new version information based on the current version information;

[0021] The consumption service associates the to-be-stored interaction data with the new version information and stores them in the database and the cache library.

[0022] In an optional implementation manner, the step of determining the new version information based on the current version information by the consumption service includes:

[0023] The consumption service adds 1 to the version number in the current version information to obtain the new version information.

[0024] In an optional implementation manner, the interactive data stored in the database and the cache storehouse are associated with version information, and the version information includes an interface number and a version number of the system to be interacted with to which the associated interactive data belongs;

[0025] When the response component receives a data request from any of the systems or clients to be interacted with, the response component searches for response data from the cache or database based on the data request and returns the response data to the system or client to be interacted with that issued the data request, including:

[0026] When receiving a data request from any of the to-be-interacted systems or the client, the response component obtains the interface number of the target interactive system carried in the data request;

[0027] The response component searches for at least one matching pending interaction data from the cache based on the interface number of the target interaction system, and searches for the at least one matching pending interaction data from the database if the matching pending interaction data does not exist in the cache;

[0028] The response component determines the target interaction data with the largest version number from the at least one piece of pending interaction data based on the version information corresponding to each piece of pending interaction data;

[0029] The response component returns the target interaction data and its version information as the response data to the system to be interacted with or the client that issued the data request.

[0030] In a second aspect, the present invention provides a data transfer system, the data transfer system comprising a data acquisition component, a consumption service, a response component, a database and a cache library; the consumption service maintains a message queue;

[0031] The data acquisition component is used to periodically call the third-party API interface of each system to be interacted with to obtain interaction data, and store the interaction data in the message queue;

[0032] The consumption service is used to write the data in the message queue into the database and the cache after version processing;

[0033] The response component is used for, upon receiving a data request from any of the systems or clients to be interacted with, searching for response data from the cache library or database based on the data request and returning the response data to the system or client to be interacted with that issued the data request.

[0034] In an optional implementation, the data acquisition component maintains a thread pool, and the thread pool includes multiple threads; for each of the systems to be interacted with:

[0035] The data acquisition component is used to determine the number of available threads and the single-thread acquisition amount based on a preset total acquisition amount and the real-time idle thread amount of the thread pool at a fixed time;

[0036] The data acquisition component is also used to randomly select the number of available threads from all idle threads in the thread pool, and use each selected thread as a calling thread of the system to be interacted with;

[0037] Each calling thread is used to generate a data acquisition request based on the ranking of its own thread program number among all calling threads, the third-party API interface of the system to be interacted with, and the single-thread acquisition amount, and send the data acquisition request to the system to be interacted with;

[0038] Each calling thread is used for storing the received interaction data into the message queue when receiving the interaction data returned by the to-be-interacted system.

[0039] In an optional implementation, the interaction data includes the interface number of the system to be interacted with; the interaction data stored in the database and the cache storehouse both have associated version information; the version information includes the interface number and version number of the system to be interacted with to which the associated interaction data belongs;

[0040] When the consumption service is used to write the data in the message queue into the database and the cache after version processing, it is specifically used to:

[0041] Reading a copy of the interaction data to be stored from the message queue;

[0042] Based on the interface number carried by the to-be-stored interaction data, searching the cache library for current version information of the to-be-stored interaction data;

[0043] Based on the current version information, determine new version information;

[0044] The to-be-stored interaction data is associated with the new version information and then stored in the database and the cache library.

[0045] In an optional implementation manner, when the consumption service is used to determine the new version information based on the current version information, it is specifically used to: add 1 to the version number in the current version information to obtain the new version information.

[0046] In an optional implementation manner, the interactive data stored in the database and the cache storehouse are associated with version information, and the version information includes an interface number and a version number of the system to be interacted with to which the associated interactive data belongs;

[0047] The response component is used to, when receiving a data request from any of the systems or clients to be interacted with, search for response data from the cache library or database based on the data request and return it to the system or client to be interacted with that issued the data request, specifically to:

[0048] Upon receiving a data request from any of the systems to be interacted with or the client, obtaining an interface number of the system to be accessed and interacted with in the data request;

[0049] Based on the interface number of the system to be accessed and interacted, searching the cache library for at least one matching piece of pending interaction data, and if the cache library does not contain the matching piece of pending interaction data, searching the database for the matching piece of the at least one matching piece of pending interaction data;

[0050] Based on the version information of each of the pending interaction data, determining the response interaction data with the largest version number from the at least one pending interaction data;

[0051] The response interaction data and its version information are returned as the response data to the system to be interacted with or the client that issued the data request.

[0052] Compared with the prior art, the embodiment of the present invention provides a data interaction transfer method and a data transfer system. The method is applied to the data transfer system, and the data transfer system includes a data acquisition component, a consumer service maintaining a message queue, a response component, a database and a cache library. The data acquisition component periodically calls the third-party API interface of each system to be interacted with to obtain the interaction data, and stores the interaction data in the message queue; the consumer service writes the data in the message queue into the database and the cache library after version processing; when the response component receives a data request from any system to be interacted with or a client, it searches the cache library or the database for the response data based on the data request and returns it to the system to be interacted with or the client that issued the data request. The present invention centrally stores the interaction data of each system to be interacted with through the data transfer system, and can quickly respond to the data requests from the system to be interacted with or the client for each system to be interacted with, thereby avoiding the increase of the interaction data between the systems and the increase of the number of interacting systems, which leads to the increase of the API response time and the degradation of the system performance, and ensures the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention.

[0055] Figure 2 One of the flow charts of a data interaction transfer method provided in an embodiment of the present invention.

[0056] Figure 3 The second flowchart of a data interaction transfer method provided by an embodiment of the present invention.

[0057] Figure 4 The third flowchart of a data interaction transfer method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0059] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0061] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0062] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0063] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0064] Please refer to Figure 1 , Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention. Figure 1 ,The data transfer system includes data acquisition components, consumption services, response components, database and cache library. Among them, the consumption service maintains a message queue.

[0065] The data transfer system may be deployed on one server, or distributed on multiple servers, which is not limited in the embodiment of the present invention.

[0066] Please refer to Figure 2 , Figure 2 A flow chart of a data exchange transfer method provided by an embodiment of the present invention, the execution subject of the method is as follows Figure 1 The data transfer system shown, the data interactive transfer method includes the following steps S101 to S103.

[0067] S101. The data acquisition component periodically calls the third-party API interface of each system to be interacted with to obtain interaction data, and stores the interaction data in a message queue.

[0068] In this embodiment, each system to be interacted with and the data transfer system may be each subsystem of an Internet platform. For example, each subsystem of a shopping platform may include a product management system, an order management system, a payment system, a logistics management system, a merchant management system, a user management system, a search recommendation system, etc. Alternatively, each system to be interacted with and the data transfer system may also belong to different systems. For the data transfer system, each system to be interacted with belongs to an external system. Alternatively, a part of the systems to be interacted with and the data transfer system may belong to the same Internet platform, and another part of the systems to be interacted with may be external systems that do not belong to the Internet platform.

[0069] Optionally, the interaction data obtained in different systems to be interacted with are different. Taking the system to be interacted with as a merchant management system as an example, the interaction data may be merchant information; taking the system to be interacted with as a charging station management system as an example, the interaction data may be charging pile data. This example is only an example and is not limited here.

[0070] S102: The consumer service performs version processing on the data in the message queue and writes the data into the database and cache.

[0071] In this embodiment, the consumer service can subscribe to the message queue. Message Queue NG is used to decouple the communication between services and provide asynchronous processing capabilities. Kafka or RabbitMQ can be used. The database can be used but not limited to MySQL database, Prometheus database, ElasticSearch database, etc., and the cache library can be Redis, Memcached, etc.

[0072] S103: When receiving a data request from any system or client to be interacted with, the response component searches for response data from a cache or database based on the data request and returns the response data to the system or client to be interacted with that issued the data request.

[0073] In this embodiment, in order to allow external systems or services to interact with the data transfer system and implement data query, transmission, update and other operations, the data transfer system exposes an API query interface to the outside world. When the response component perceives the call of the API query interface, it is responsible for finding the corresponding data from the cache or database and returning it to the system or client to be interacted with that initiated the data request.

[0074] The data interaction transfer method provided by the embodiment of the present invention is applied to a data transfer system, and the data transfer system includes a data acquisition component, a consumption service maintaining a message queue, a response component, a database, and a cache library. The data acquisition component periodically calls the third-party API interface of each system to be interacted with to obtain interaction data, and stores the interaction data in the message queue; the consumption service writes the data in the message queue into the database and cache library after version processing; when the response component receives a data request from any system to be interacted with or a client, it searches the cache library or database for response data based on the data request and returns it to the system to be interacted with or the client that issued the data request. In this way, the interaction data of each system to be interacted with is centrally stored through the data transfer system, and the data request from the system to be interacted with or the client for each system to be interacted with can be quickly responded to externally, avoiding the sharp increase in interaction data between systems and the increase in the number of interacting systems. The API response time is prolonged, resulting in a decrease in system performance, and the user experience is guaranteed.

[0075] In an optional implementation, multi-threading technology may be used to obtain data from each system to be interacted. Therefore, the data acquisition component may maintain a thread pool, which includes multiple threads. Figure 2 Based on Figure 3 The sub-steps of the above step S101 may include S1011 to S1014.

[0076] S1011. For each system to be interacted with, the data acquisition component periodically determines the number of available threads and the single-thread acquisition amount based on a preset total acquisition amount and the number of idle threads in the thread pool.

[0077] In this embodiment, for different systems to be interacted, the preset total amount and frequency of obtaining data may be different or the same. For example, system A to be interacted may obtain data every 10 minutes, with a preset total amount of 1000 data; while system B to be interacted may obtain data every 20 minutes, with a preset total amount of 5000 data. This example is only an example and is not intended to be limiting.

[0078] Optionally, based on the number of real-time idle threads in the thread pool, a preset number of threads can be reserved for use when acquiring other interactive system data, and the preset number can be 2, 3, 5, etc., which is not limited here. After determining the number of available threads, the ratio of the preset total acquisition amount to the number of available threads is calculated as the single-thread acquisition amount.

[0079] S1012: The data acquisition component randomly selects available threads from all idle threads in the thread pool, and uses each selected thread as a calling thread of the system to be interacted with.

[0080] In this embodiment, assuming that the number of available threads is X, the data acquisition component can randomly select X threads from all currently idle threads in the thread pool, all of which serve as calling threads of the system to be interacted with.

[0081] S1013. Each calling thread generates a data acquisition request based on the ranking of its own thread program number among all calling threads, the third-party API interface of the system to be interacted with, and the single-thread acquisition amount, and sends the data acquisition request to the system to be interacted with.

[0082] It can be understood that each thread will have a default name when it is created, such as "pool-1-thread-1", "pool-1-thread-2", etc., where "pool-1" is the thread pool number and "thread-1" is the thread number in the thread pool.

[0083] In this embodiment, the data acquisition component can sort the calling threads in ascending or descending order based on the thread program number to determine the ranking of each calling thread. Then each calling thread determines its own data acquisition interval based on its thread program number ranking among all calling threads and the single-thread acquisition amount, and then uses the request data name and data acquisition interval as request parameters, and collects the third-party API interface of the system to be interacted with to generate a data acquisition request, and sends the data acquisition request to the system to be interacted with.

[0084] Assuming that a calling thread is ranked x (x=1, 2, 3...X), and the single-thread acquisition amount is m, then the data acquisition interval of the calling thread can be the latest first (x-1)*m items to the latest first x*m items. For example, when the single-thread acquisition amount is 100, if the calling thread is ranked 1, then the data acquisition interval of the calling thread is the latest first 100 items; if the calling thread is ranked 2, then the data acquisition interval of the calling thread is the latest first 100 items to the first 200 items. This example is only an example and is not limited here.

[0085] S1014. When each calling thread receives the interaction data returned by the interaction system, it stores the received interaction data in the message queue.

[0086] In this embodiment, when each calling thread receives the interaction data (including multiple pieces of data) returned by the interactive system, it directly stores the received interaction data into the message queue.

[0087] For each system to be interacted with, a complete data acquisition can be performed according to the above steps S1011 to S1014.

[0088] In an optional implementation, the interaction data includes the interface number of the system to be interacted with; the interaction data stored in the database and the cache library have associated version information, and the version information includes the interface number and version number of the system to be interacted with the corresponding associated interaction data. Figure 2 Based on Figure 4 , the sub-steps of the above step S102 may include S1021 to S1024;

[0089] S1021. The consumer service reads a copy of the interaction data to be stored from the message queue;

[0090] S1022. The consumer service searches the cache for current version information of the interaction data to be stored based on the interface number carried by the interaction data to be stored.

[0091] S1023. The consumer service determines new version information based on the current version information;

[0092] S1024: The consumer service associates the interaction data to be stored with the new version information and stores them in the database and cache.

[0093] In this embodiment, the consumption service may determine the new version information based on the current version information by adding 1 to the version number in the current version information to obtain the new version information.

[0094] In an optional implementation, the interface number can identify the system to be interacted. Therefore, the sub-steps of the above step S103 may include S1031 to S1034:

[0095] S1031. When receiving a data request from any system to be interacted with or a client, the response component obtains the interface number of the target interaction system carried in the data request;

[0096] S1032. The response component searches for at least one matching pending interaction data from the cache based on the interface number of the target interaction system. If the matching pending interaction data does not exist in the cache, the response component searches for at least one matching pending interaction data from the database.

[0097] S1033. The response component determines the target interaction data with the largest version number from at least one piece of pending interaction data based on the version information corresponding to each piece of pending interaction data;

[0098] S1034. The response component returns the target interaction data and its version information as response data to the system to be interacted with or the client that issued the data request.

[0099] In this embodiment, the cache is preset with a life span. Once the storage time of the interaction data in the cache reaches the life span, it is deemed invalid and automatically cleared. Therefore, when the response component fails to find matching pending interaction data in the cache, it will turn to the database for search. The response component needs to find the target interaction data with the largest version information and its version information as response data to return to the system or client to be interacted that issued the data request.

[0100] It should be noted that the execution order of each step in the above method embodiment is not limited to that shown in the drawings, and the execution order of each step shall be based on the actual application situation.

[0101] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0102] The present invention stores the interaction data of each system to be interacted in a centralized manner through a data transfer system, and can quickly respond to data requests from the system to be interacted or from the client to each system to be interacted, thereby avoiding a sharp increase in interaction data between systems and an increase in the number of interacting systems, which results in a prolonged API response time and a decrease in system performance, and ensures user experience;

[0103] The present invention uses multi-threading technology to obtain data from each system to be interacted. During data acquisition, for each system to be interacted, the number of available threads and the single-thread acquisition amount are determined based on the preset total acquisition amount and the number of idle threads in the thread pool, and then each calling thread is found, so that each calling thread acquires data based on the calculated data acquisition interval, thereby ensuring data acquisition efficiency.

[0104] The embodiment of the present invention also provides a data transfer system, which includes a data acquisition component, a consumption service, a response component, a database and a cache library; the consumption service maintains a message queue. Among them:

[0105] The data acquisition component is used to periodically call the third-party API interface of each system to be interacted with to obtain interaction data and store the interaction data in the message queue;

[0106] The consumer service is used to process the data in the message queue and write it into the database and cache library after version processing;

[0107] The response component is used to find response data from the cache or database based on the data request when receiving a data request from any system or client to be interacted with, and return it to the system or client to be interacted with that issued the data request.

[0108] Optionally, the data acquisition component maintains a thread pool, which includes multiple threads; for each system to be interacted with: the data acquisition component is used to determine the number of available threads and the single-thread acquisition amount based on a preset total acquisition amount and the real-time idle thread amount of the thread pool; the data acquisition component is also used to randomly select the number of available threads from all idle threads in the thread pool, and use each selected thread as a calling thread of the system to be interacted with; each calling thread is used to generate a data acquisition request based on the ranking of its own thread program number among all calling threads, the third-party API interface of the system to be interacted with, and the single-thread acquisition amount, and send the data acquisition request to the system to be interacted with; each calling thread is used to store the received interaction data in a message queue when receiving the interaction data returned by the system to be interacted with.

[0109] Optionally, the interaction data includes the interface number of the system to be interacted with; the interaction data stored in the database and cache are associated with version information; the version information includes the interface number and version number of the system to be interacted with to which the associated interaction data belongs. When the consumer service is used to write the data in the message queue to the database and cache after version processing, it can be used to: read a copy of the interaction data to be stored from the message queue; find the current version information of the interaction data to be stored from the cache based on the interface number carried by the interaction data to be stored; determine the new version information based on the current version information; associate the interaction data to be stored with the new version information and store them in the database and cache.

[0110] Optionally, when the consumption service is used to determine the new version information based on the current version information, it is specifically used to: add 1 to the version number in the current version information to obtain the new version information.

[0111] Optionally, the interactive data stored in the database and the cache library are associated with version information, and the version information includes the interface number and version number of the system to be interacted to which the corresponding associated interactive data belongs. The response component is used to, when receiving a data request from any system to be interacted or a client, find out the response data from the cache library or the database based on the data request and return it to the system to be interacted or the client that issued the data request. Specifically, it is used to: when receiving a data request from any system to be interacted or a client, obtain the interface number of the system to be visited in the data request; based on the interface number of the system to be visited, find at least one matching pending interactive data from the cache library, if there is no matching pending interactive data in the cache library, find at least one matching pending interactive data from the database; based on the version information of each pending interactive data, determine the response interactive data with the largest version number from at least one pending interactive data; return the response interactive data and its version information as response data to the system to be interacted or the client that issued the data request.

[0112] In summary, the embodiment of the present invention provides a data interaction transfer method and a data transfer system. The method is applied to the data transfer system, and the data transfer system includes a data acquisition component, a consumption service maintaining a message queue, a response component, a database and a cache library. The data acquisition component periodically calls the third-party API interface of each system to be interacted with to obtain the interaction data, and stores the interaction data in the message queue; the consumption service writes the data in the message queue to the database and the cache library after version processing; when the response component receives a data request from any system to be interacted with or a client, it searches the cache library or the database for the response data based on the data request and returns it to the system to be interacted with or the client that issued the data request. The present invention centrally stores the interaction data of each system to be interacted with through the data transfer system, and can quickly respond to the data requests from the system to be interacted with or the client for each system to be interacted with, thereby avoiding the increase of interaction data between systems and the increase of the number of interacting systems, which leads to the increase of API response time and the degradation of system performance, and ensures user experience.

[0113] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A data interaction transfer method, characterized in that: Applied to a data transfer system, the data transfer system includes a data acquisition component, a consumption service, a response component, a database and a cache library; the consumption service maintains a message queue; the method includes: The data acquisition component periodically calls the third-party API interface of each system to be interacted with to obtain interaction data, and stores the interaction data in the message queue; The consumer service performs version processing on the data in the message queue and writes the data into the database and the cache library; When receiving a data request from any of the systems or clients to be interacted with, the response component searches out response data from the cache or database based on the data request and returns the response data to the system or client to be interacted with that issued the data request.

2. The method according to claim 1, characterized in that The data acquisition component maintains a thread pool, which includes multiple threads; the data acquisition component periodically calls the third-party API interface of each system to be interacted with to obtain interaction data, and stores the interaction data in the message queue, including: For each of the systems to be interacted with, the data acquisition component periodically determines the number of available threads and the single-thread acquisition amount based on a preset total acquisition amount and the real-time idle thread amount of the thread pool; The data acquisition component randomly selects the number of available threads from all idle threads in the thread pool, and uses each selected thread as a calling thread of the system to be interacted with; Each calling thread generates a data acquisition request based on the ranking of its own thread program number among all calling threads, the third-party API interface of the system to be interacted with, and the single-thread acquisition amount, and sends the data acquisition request to the system to be interacted with; When receiving the interaction data returned by the to-be-interacted system, each calling thread stores the received interaction data in the message queue.

3. The method according to claim 1, characterized in that The interaction data includes the interface number of the system to be interacted with; the interaction data stored in the database and the cache storehouse both have associated version information; the version information includes the interface number and version number of the system to be interacted with to which the associated interaction data belongs; The step of the consumer service performing version processing on the data in the message queue and then writing the data into the database and the cache library comprises: The consumer service reads a copy of the interaction data to be stored from the message queue; The consumer service searches the cache for current version information of the interaction data to be stored based on the interface number carried by the interaction data to be stored; The consumption service determines new version information based on the current version information; The consumption service associates the to-be-stored interaction data with the new version information and stores them in the database and the cache library.

4. The method according to claim 3, characterized in that The step of determining the new version information based on the current version information by the consumption service includes: The consumption service adds 1 to the version number in the current version information to obtain the new version information.

5. The method according to claim 1, characterized in that The interactive data stored in the database and the cache storehouse all have associated version information, and the version information includes the interface number and version number of the system to be interacted with to which the associated interactive data belongs; When the response component receives a data request from any of the systems or clients to be interacted with, the response component searches for response data from the cache or database based on the data request and returns the response data to the system or client to be interacted with that issued the data request, including: When receiving a data request from any of the to-be-interacted systems or the client, the response component obtains the interface number of the target interactive system carried in the data request; The response component searches for at least one matching pending interaction data from the cache based on the interface number of the target interaction system, and searches for the at least one matching pending interaction data from the database if the matching pending interaction data does not exist in the cache; The response component determines the target interaction data with the largest version number from the at least one piece of pending interaction data based on the version information corresponding to each piece of pending interaction data; The response component returns the target interaction data and its version information as the response data to the system to be interacted with or the client that issued the data request.

6. A data transfer system, characterized in that: The data transfer system includes a data acquisition component, a consumption service, a response component, a database and a cache library; the consumption service maintains a message queue; The data acquisition component is used to periodically call the third-party API interface of each system to be interacted with to obtain interaction data, and store the interaction data in the message queue; The consumption service is used to write the data in the message queue into the database and the cache after version processing; The response component is used for, upon receiving a data request from any of the systems or clients to be interacted with, searching for response data from the cache library or database based on the data request and returning the response data to the system or client to be interacted with that issued the data request.

7. The data transfer system according to claim 6, characterized in that: The data acquisition component maintains a thread pool, which includes multiple threads; for each of the systems to be interacted with: The data acquisition component is used to determine the number of available threads and the single-thread acquisition amount based on a preset total acquisition amount and the real-time idle thread amount of the thread pool at a fixed time; The data acquisition component is also used to randomly select the number of available threads from all idle threads in the thread pool, and use each selected thread as a calling thread of the system to be interacted with; Each calling thread is used to generate a data acquisition request based on the ranking of its own thread program number among all calling threads, the third-party API interface of the system to be interacted with, and the single-thread acquisition amount, and send the data acquisition request to the system to be interacted with; Each calling thread is used for storing the received interaction data into the message queue when receiving the interaction data returned by the to-be-interacted system.

8. The data transfer system according to claim 6, characterized in that: The interaction data includes the interface number of the system to be interacted with; the interaction data stored in the database and the cache storehouse both have associated version information; the version information includes the interface number and version number of the system to be interacted with to which the associated interaction data belongs; When the consumption service is used to write the data in the message queue into the database and the cache after version processing, it is specifically used to: Reading a copy of the interaction data to be stored from the message queue; Based on the interface number carried by the to-be-stored interaction data, searching the cache library for current version information of the to-be-stored interaction data; Based on the current version information, determine new version information; The to-be-stored interaction data is associated with the new version information and then stored in the database and the cache library.

9. The data transfer system according to claim 8, characterized in that: When the consumption service is used to determine the new version information based on the current version information, it is specifically used to: add 1 to the version number in the current version information to obtain the new version information.

10. The data transfer system according to claim 6, characterized in that: The interactive data stored in the database and the cache storehouse all have associated version information, and the version information includes the interface number and version number of the system to be interacted with to which the associated interactive data belongs; The response component is used to, when receiving a data request from any of the systems or clients to be interacted with, search for response data from the cache library or database based on the data request and return it to the system or client to be interacted with that issued the data request, specifically to: Upon receiving a data request from any of the systems to be interacted with or the client, obtaining an interface number of the system to be accessed and interacted with in the data request; Based on the interface number of the system to be accessed and interacted, searching the cache library for at least one matching piece of pending interaction data, and if the cache library does not contain the matching piece of pending interaction data, searching the database for the matching piece of the at least one matching piece of pending interaction data; Based on the version information of each of the pending interaction data, determining the response interaction data with the largest version number from the at least one pending interaction data; The response interaction data and its version information are returned as the response data to the system to be interacted with or the client that issued the data request.

Images