A data exchange method and device in a multi-network environment

Abstract

The purpose of this application is to provide a data exchange method and device in a multi-network environment. The data exchange mobilization center abstracts the data exchange between different business systems into system-level topics and their corresponding business type tags; receives a data exchange task sent from a target business system through a data pipeline, where the data exchange task includes target data, the target topic to which the target data belongs, and the target business type tag; determines at least one consuming business system that subscribes to the target topic and the target business type tag and its task scheduling priority; and pushes the target data to the at least one consuming business system in descending order of the task scheduling priority and records their respective logs, achieving the purpose of liberating different business systems from complex interaction relationships and recording the entire exchange life cycle of data exchange through logging for subsequent problem tracking and positioning.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a data exchange method and device in a multi-network environment. Background Art

[0002] In a multi-network environment for data exchange, the traditional mode is to implement it by writing data interfaces. For example, suppose there are three business systems, A, B, and C. A function of the A business system is to synchronize processed data to the B business system and the C business system for subsequent data processing or business operations. Then, the B business system and the C business system must each provide a data receiving interface, which is called separately by the A business system. In actual situations, different business systems of a company entity rely on each other, call each other, and jointly use and process the same data objects to achieve specific functions. Therefore, in the system architecture, taking the above three business systems as an example, the A, B, and C business systems call each other. This mode will cause the following problems: the number of data exchange interfaces is increasing and redundant; the interface relationships are not clear, making maintenance inconvenient, it is difficult to evaluate the risks of interface modifications, and interface changes require multiple other business systems or even all business systems to provide change support; the operation of interface calls is one-time, and only single call success or failure records can be generated.

[0003] In the traditional data exchange method in a multi-network environment, each system needs to write a large number of data exchange interfaces for Cartesian product-style data exchange, and requires programmers of different systems to perform a large number of interface writing and mutual calling tasks with a high degree of repetition. Summary of the Invention

[0004] An object of this application is to provide a data exchange method and device in a multi-network environment. Through a data exchange scheduling center, data is distributed according to the index definitions of different business systems, achieving the purpose of liberating different business systems from complex interaction relationships. And by recording logs, the entire exchange life cycle of data exchange is recorded, solving the problem that it is difficult to locate and troubleshoot data exchange results, so as to facilitate subsequent problem tracking and positioning.

[0005] According to one aspect of this application, a data exchange method in a multi-network environment is provided, which is applied to a data exchange scheduling center. Among them, the method includes:

[0006] Abstract the data exchange between different business systems into system-level topics and their corresponding business type tags. If the business system is the data sender, define a production topic for the sender to produce data and one or more business type tags included therein. If the business system is the data receiver, define a subscription topic for the receiver to receive data and one or more business type tags included therein;

[0007] Receive data exchange tasks sent from a target business system through a data pipeline, where the data exchange tasks include target data and target topic and target business type tags to which the target data belongs;

[0008] Determine at least one consuming business system that subscribes to the target topic and the target business type tags and their task scheduling priorities;

[0009] Push the target data to the at least one consuming business system respectively in the order from high to low of the task scheduling priorities and record their respective logs, where each log record includes the exchanged data and the result of whether the exchange is successful or not.

[0010] Further, in the above method, the determining at least one consuming business system that subscribes to the target topic and the target business type tags and their task scheduling priorities includes:

[0011] Query and determine the at least one consuming business system that subscribes to the target topic and the target business type tags;

[0012] Detect the response duration of each consuming business system through heartbeats respectively;

[0013] Based on the response duration of each consuming business system, determine the task scheduling priority of each consuming business system.

[0014] Further, in the above method, the result of whether the exchange is successful or not in the log record includes data exchange failure and data exchange success.

[0015] Further, in the above method, if the result of whether the exchange is successful or not in the log record is data exchange failure, the method further includes:

[0016] Retry pushing the target data to the consuming business system corresponding to the data exchange failure until the number of retries is equal to the maximum number of retries or until the data exchange is successful.

[0017] Further, in the above method, the method further includes:

[0018] Set the maximum number of retries for data exchange retry after data exchange failure.

[0019] According to another aspect of the present application, there is also provided a non-volatile storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the processor is enabled to implement a data exchange method in a multi-network environment as described above.

[0020] According to another aspect of the present application, there is also provided a data exchange device in a multi-network environment. The device includes:

[0021] One or more processors;

[0022] A computer-readable medium for storing one or more computer-readable instructions.

[0023] When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement a data exchange method in a multi-network environment as described above.

[0024] Compared with the prior art, the present application introduces an intermediate medium, namely a data exchange scheduling center, between different business systems. In the data exchange scheduling center, the data exchange between different business systems is abstracted as system-level topics and their corresponding business type tags. If the business system is the data sender, a production topic for producing data and one or more business type tags included therein are defined for the sender. If the business system is the data receiver, a subscription topic for receiving data and one or more business type tags included therein are defined for the receiver. And a data exchange task sent from a target business system is received through a data pipeline. The data exchange task includes target data, the target topic to which the target data belongs, and target business type tags. At least one consuming business system that subscribes to the target topic and the target business type tags and its task scheduling priority are determined. The target data is pushed to the at least one consuming business system in descending order of the task scheduling priority, and respective logs are recorded. Each log record includes the exchanged data and the result of whether the exchange is successful or not. Thus, through the data exchange scheduling center, data distribution is performed according to the index definitions of different business systems, achieving the purpose of liberating different business systems from complex interaction relationships. And by recording logs, the entire exchange life cycle of data exchange is recorded, solving the problem that it is difficult to locate and troubleshoot data exchange results, so as to facilitate subsequent problem tracking and positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects, and advantages of the present application will become more apparent:

[0026] Figure 1 A flowchart showing a data exchange method in a multi-network environment according to one aspect of the present application;

[0027] Figure 2 An interaction diagram showing a data exchange method in a multi-network environment according to one aspect of the present application in an actual application scenario.

[0028] Like or similar reference numerals in the drawings represent like or similar components. Detailed implementation manners

[0029] The present application will be further described in detail below with reference to the drawings.

[0030] In a typical configuration of the present application, the terminal, the device of the service network, and the trusted party all include one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0031] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0032] The computer-readable medium includes permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of the computer storage medium include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. As defined herein, the computer-readable medium does not include transitory media such as modulated data signals and carrier waves.

[0033] As Figure 1 shown, an aspect of the present application provides a schematic flowchart of a data exchange method in a multi-network environment. The method is applied to a data exchange scheduling center. The method includes steps S11, S12, S13, and S14, and specifically includes the following steps:

[0034] Step S11 abstracts the data exchange between different business systems into system-level topics and their corresponding business type tags. If the business system is the data sender, define the production topic of the production data for the sender and one or more business type tags it contains. If the business system is the data receiver, define the subscription topic of the received data for the receiver and one or more business type tags it contains. This realizes abstracting the data exchange between different business systems into system-level topics and their corresponding business type tags through the data exchange scheduling center, reducing the requests and receptions of duplicate data exchange interfaces, isolating the data exchange interface from the business interface through the topic and its corresponding business type tags, that is, satisfying both processing different businesses and conducting data exchange through the data exchange scheduling center, avoiding the impact of changes in the data exchange interface on other business systems, and making the data exchange relationship between different business systems clear and easy to maintain.

[0035] Here, the business type tag corresponding to each topic can be one or multiple. For example, when the topic is an order, the corresponding business type tags can include tags such as order payment successful, order shipped, order cancelled, and order received.

[0036] In step S11, the data exchange scheduling center abstracts the data exchange between different business systems into system-level topics and specific corresponding business type tags, enabling the establishment of a data exchange pipeline between the two parties exchanging data, that is, each business system participating in the data exchange can be both the data sender and the data receiver. For example, the data exchange process can be as follows: when the business system is a system for producing data, the business system is the sender, and define the topic of the production data of the business system and one or more business type tags it contains in the data exchange scheduling center. After the data is processed, it is sent to the data pipeline for data exchange; when the business system is a system for consuming data, the business system is the receiver, and define the subscribed topic and one or more business type tags it contains in the data exchange scheduling center to retrieve the corresponding data from the data pipeline later.

[0037] After the target business system sends a data exchange task that requires data exchange to the data exchange scheduling center, step S12 receives the data exchange task sent from the target business system through the data pipeline. The data exchange task includes the target data and the target topic and target business type tags to which the target data belongs.

[0038] Since each business system's producible data topics, their corresponding business type tags, subscribed topics, and their corresponding business type tags have been defined in the data exchange scheduling center, in step S13, at least one consuming business system that has subscribed to the target topic and the target business type tag and its task scheduling priority can be determined from the data exchange scheduling center;

[0039] In step S14, the target data is pushed to the at least one consuming business system in descending order of the task scheduling priority and the respective logs are recorded, where each log record includes the exchanged data and the result of whether the exchange was successful or not, so that the data exchange result can be clearly perceived by recording the logs, facilitating problem tracking and positioning later.

[0040] It should be noted that the result of whether the exchange was successful or not in the log record includes data exchange failure and data exchange success, so as to record the result of data exchange and facilitate subsequent tracking and positioning.

[0041] Through the above steps S11 to S14, through the data exchange scheduling center, data is distributed according to the metric definitions of different business systems, achieving the purpose of liberating different business systems from complex interaction relationships, and by recording logs, the entire exchange life cycle of data exchange is recorded, solving the problem that it is difficult to locate and troubleshoot data exchange results, facilitating subsequent problem tracking and positioning.

[0042] In a practical application scenario of this application, as Figure 2 shown, in a multi-network environment, the servers of different business systems are physically isolated from each other and are connected by gateways. For example, the servers 1, 2, and 3 corresponding to business system 1, business system 2, and business system 3 can each be both the sender and the receiver of data exchange, that is, each server can be both the sender and the receiver, and the relationship between the sender and the receiver is defined by the abstracted topic topic and business type tag tag in the data exchange scheduling center. After a group of data exchanges is defined, such as the defined sender, the corresponding one or more receivers, and the topic abstracted from the data to be exchanged and its corresponding business type, the data exchange scheduling center delivers the data to be exchanged in the data pipeline according to the priority of the receivers, and each delivery is carried out by a lightweight coroutine, improving the parallelism and efficiency of the delivery. Each group of deliveries has its own log record, recording the delivered data and the result of whether the data exchange was successful or not.

[0043] In an embodiment of this actual application scenario, the method performs data exchange between different business systems. For example, the order center, merchant center, user center, product center, marketing center, etc. Although they are interrelated in business and require data sharing, they can be independent of each other. The data that needs to be shared is exchanged through the data exchange scheduling center. Each business system only needs to write unified data sending and receiving interfaces, and the call relationships are defined in the scheduling center. The sender and receiver can be adjusted at any time, and there are complete and clear record logs for tracking and positioning the calls between systems, thereby establishing an efficient and stable system architecture for data exchange in a multi-network environment.

[0044] Continuing with the above embodiment of the present application, step S13 of determining at least one consuming business system that subscribes to the target topic and the target business type tag and its task scheduling priority specifically includes:

[0045] Query and determine the at least one consuming business system that subscribes to the target topic and the target business type tag;

[0046] Detect the response duration of each consuming business system through heartbeats respectively;

[0047] Based on the response duration of each consuming business system, determine the task scheduling priority of each consuming business system. For example, the data exchange priority of a business system with a short response duration is higher than that of a business system with a long response duration, so as to determine the priority of different business systems for data exchange tasks through the response duration, thereby reducing the average response duration of data exchange.

[0048] For example, when the sender that needs to perform data exchange, that is, the target business system, sends a data exchange task to the data exchange scheduling center, it is necessary to query from the data exchange scheduling center which specific business systems have subscribed to the target topic and the target business type tag in the data exchange task, so as to determine one or more consuming business systems that have subscribed to the target topic and the target business type tag; in order to improve the efficiency of data exchange, the data exchange scheduling center detects the response duration of each consuming business system through heartbeats respectively, and dynamically allocates the task scheduling priority of each consuming business system through the response duration of each consuming business system, thereby improving the overall data exchange efficiency.

[0049] In this embodiment, a heartbeat detection mechanism exists between the data exchange scheduling center and each business system to detect the response duration of different business systems. When the response duration of the heartbeat detection is greater than the preset response duration threshold or the response fails, it is necessary to reduce the priority of data exchange execution for this business system; when the response duration of the heartbeat detection is much lower than the preset response duration or the response is rapid (such as immediate response), it is necessary to increase the priority of data exchange execution for the business system with a fast response speed or short response time, so as to achieve the purpose of improving the average response duration of data exchange and further improve the data exchange efficiency.

[0050] Continuing with the above embodiment of the present application, if the result of whether the exchange is successful or not in the recorded log is that the data exchange fails, the method further includes:

[0051] Retry pushing the target data to the consuming business system corresponding to when the data exchange fails until the number of retries is equal to the maximum number of retries or until the data exchange is successful.

[0052] For example, when the data exchange fails due to influencing factors such as network fluctuations, it can be learned from the recorded log that the data exchange fails. In order to improve the success rate of data exchange, after the data exchange fails, the data exchange scheduling center can retry the failed target data through a retry mechanism, that is, the data exchange scheduling center repeatedly retries pushing the failed target data to the consuming business system corresponding to when the consumption fails until the number of retries reaches the maximum number of retries (that is, the number of retries is equal to the maximum number of retries) and then stops retrying, or until the target data is successfully exchanged to the consuming business system and the data exchange is successful.

[0053] Continuing with the above embodiment of the present application, in order to avoid wasting resources caused by infinite loop retries of the failed exchanged data after the data exchange fails, a data exchange method in a multi-network environment provided by one aspect of the present application further includes:

[0054] Set the maximum number of retries for data exchange retry after the data exchange fails. Here, the maximum number of retries can be several times, or dozens of times or any number of retries, so as to control the number of retries after the data exchange fails, ensure that not only can the retry after the data exchange be satisfied, but also the situation of infinite retries can be avoided, thereby increasing the probability of successfully exchanging the data that fails in the data exchange.

[0055] In all embodiments of the application, the data exchange between different business systems is uniformly managed and maintained through a data exchange scheduling center. The data sending and receiving, exception retry, and logging are separated, which is clear and straightforward. By having an independent data exchange scheduling center, the writing of data exchange interfaces is reduced, and the problem of difficult maintenance of data exchange relationships is solved. The unified exception retry mechanism can re-perform data exchange when data exchange fails due to factors such as network fluctuations, improving the success rate of data exchange. The response time of different business systems is detected based on the heartbeat mechanism, and then the task priorities of the queues are dynamically allocated according to the response time of different business systems, improving the overall data exchange efficiency. The logs of the entire life cycle of data exchange are recorded for subsequent problem tracking and positioning.

[0056] According to another aspect of the present application, there is also provided a non-volatile storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the processor is caused to implement the data exchange method in the multi-network environment as described above.

[0057] According to another aspect of the present application, there is also provided a data exchange device in a multi-network environment. Among them, the device includes:

[0058] One or more processors;

[0059] A computer-readable medium for storing one or more computer-readable instructions,

[0060] When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors are caused to implement the data exchange method in the multi-network environment as described above.

[0061] Herein, for the detailed content of each embodiment in the data exchange device in the multi-network environment, reference may specifically be made to the corresponding part of the embodiment of the data exchange method in the multi-network environment as described above, and details are not repeated herein.

[0062] In summary, through the data exchange scheduling center of the present application, the data metrics of the exchange data required between different business systems can be defined, and the exchange relationships of data can be added, modified, or removed to achieve data sharing in a multi-network environment. The data exchange scheduling center has an execution queue for data exchange, maintaining a data pipeline for the sender and the receiver. The sender adds data to the data pipeline, and the receiver consumes data from the data pipeline. Detailed log records are made for the data exchange between different business systems, including records of successful and failed exchanges. For records of failed exchanges, there is a retry mechanism. According to the definition of data metrics, the retry is performed until the maximum number of retries or until the data exchange is successful. The data exchange scheduling center has a heartbeat detection mechanism with each business system. For a business system with a heartbeat detection response duration greater than the response duration threshold or a failed response, the priority of data exchange execution is reduced, and the average response duration of data exchange is increased.

[0063] It should be noted that the present application can be implemented in software and / or a combination of software and hardware. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of the present application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present application (including related data structures) can be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with the processor to execute each step or function.

[0064] In addition, a part of the present application can be applied as a computer program product. For example, computer program instructions, when executed by a computer, can call or provide the methods and / or technical solutions according to the present application through the operation of the computer. The program instructions for calling the methods of the present application may be stored in a fixed or removable recording medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium, and / or stored in the working memory of a computer device running according to the program instructions. Here, an embodiment according to the present application includes a device that includes a memory for storing computer program instructions and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the device is triggered to run based on the methods and / or technical solutions according to the foregoing multiple embodiments of the present application.

[0065] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is limited by the attached claims rather than the above description, so it is intended to include all changes that fall within the meaning and scope of the equivalent elements of the claims in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A data exchange method in a multi-network environment, applied to a data exchange scheduling center, wherein, The method includes: Abstracting the data exchange between different business systems into system-level topics and their corresponding business type tags. If the business system is the data sender, define a production topic for the sender to produce data and one or more business type tags included therein. If the business system is the data receiver, define a subscription topic for the receiver to receive data and one or more business type tags included therein; Receiving, through a data pipeline, a data exchange task sent from a target business system, where the data exchange task includes target data, the target topic to which the target data belongs, and target business type tags; Determining at least one consuming business system that subscribes to the target topic and the target business type tags and its task scheduling priority; Pushing the target data to the at least one consuming business system respectively in the order from high to low of the task scheduling priority and recording their respective logs, where each log record includes the exchanged data and the result of whether the data exchange is successful or not.

2. The method according to claim 1, wherein The determining at least one consuming business system that subscribes to the target topic and the target business type tags and its task scheduling priority includes: Querying and determining the at least one consuming business system that subscribes to the target topic and the target business type tags; Detecting the response duration of each consuming business system through heartbeats respectively; Based on the response duration of each consuming business system, determining the task scheduling priority of each consuming business system.

3. The method according to claim 1, wherein, The result of whether the data exchange is successful or not in the recorded log includes data exchange failure and data exchange success.

4. The method according to claim 3, wherein, If the result of whether the data exchange is successful or not in the recorded log is data exchange failure, the method further includes: Retrying to push the target data to the consuming business system corresponding to the data exchange failure until the retry count is equal to the maximum retry count or until the data exchange is successful.

5. The method according to claim 4, wherein, The method further includes: Setting a maximum retry count for data exchange retry after data exchange failure.

6. A non-volatile storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the processor implements the method according to any one of claims 1 to 5.

7. A data exchange device in a multi-network environment, wherein, The device includes: One or more processors; A computer-readable medium for storing one or more computer-readable instructions, When the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

Images