A gateway-based data delivery method, device and computer equipment

Abstract

The application relates to a gateway-based data issuing method, system and computer device, which persist data to be issued into a multi-tenant issuing state management table, and persist a flow text directional message data set of a corresponding tenant into a client landing library, so that the multi-tenant issuing state management table keeps the same data as the client landing library, and direct tracking and positioning of data issuing progress can be realized in the cloud. Meanwhile, a state synchronization data set of a landing state feedback of the data is listened to by a tenant issuing service module, and is updated to the multi-tenant issuing state management table, so that the multi-tenant issuing state management table, the flow text directional message protocol data set of the tenant, a client acceptance service module and the client landing library constitute a cycle with the tenant issuing service module as a medium, and thus the data issuing state can be monitored and tracked, high-time-efficiency data synchronization can be realized, and timely reissuing or supplementary issuing of data is supported.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer communication, in particular to a data delivery method and system based on a gateway and a computer device. BACKGROUND

[0002] Kafka is a high-throughput distributed publish / subscribe message system that can handle all action flow data of consumers in a website, but due to technical management specification requirements, it is not allowed to use Kafka as a data exchange channel in the industry. Therefore, data exchange between the industry and the external network needs to pass through a unified data gateway, and such a gateway generally only allows general http protocols and tcp protocols. The delivery of news information and other data has an important technical indicator - timeliness, but the client polling method for obtaining data greatly reduces the timeliness. SUMMARY

[0003] Therefore, it is necessary to provide a data delivery method and system based on a gateway and a computer device to solve the above technical problems.

[0004] A data delivery method based on a gateway, the method comprising:

[0005] synchronizing the data to be delivered in real time to a synchronization landing library;

[0006] pushing the data to be delivered in the synchronization database to a stream text directional message protocol data set that has corresponding data authority through a tenant delivery service module, and persisting the data to be delivered to a multi-tenant delivery state management table; each tenant corresponds to at least one stream text directional message protocol data set;

[0007] listening to the stream text directional message protocol data set of the corresponding tenant through a client receiving service module and persisting it to a client landing library, and sending a landing state to a state synchronization data set;

[0008] listening to the state synchronization data set of the landing state feedback sent by the client receiving service module through the tenant delivery service module, and updating the multi-tenant delivery state management table;

[0009] listening to the client landing library through an application service module, and synchronizing the delivery data in the client landing library to the database of the application service.

[0010] A data delivery method and system based on a gateway, the system comprising:

[0011] a cloud gateway, a gateway client, and an application system;

[0012] The cloud gateway comprises a synchronization landing library, a tenant delivery service module, a stream text directional message protocol data set, and a multi-tenant delivery state management table.

[0013] The gateway client comprises a client receiving service module and a client landing database;

[0014] The application system comprises an application service module and a database of the application service;

[0015] The synchronization landing database is used for synchronizing the data to be delivered in real time;

[0016] The tenant delivery service module is used for pushing the data to be delivered in the synchronization database to the stream text directional message protocol data set having the corresponding data authority of the tenant, and simultaneously persisting the data to be delivered to the multi-tenant delivery state management table; each tenant corresponds to at least one stream text directional message protocol data set;

[0017] The client receiving service module listens to the stream text directional message protocol data set of the corresponding tenant and persists to the client landing database, and simultaneously sends the landing state to the state synchronization data set;

[0018] The tenant delivery service module listens to the state synchronization data set of the landing state feedback sent by the client receiving service module, and updates to the multi-tenant delivery state management table;

[0019] The application service module listens to the client landing database, and synchronizes the delivery data in the client landing database to the database of the application service.

[0020] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps in the above method when executing the computer program.

[0021] The above gateway-based data delivery method, system and computer device persist the data to be delivered to the multi-tenant delivery state management table, and persist the stream text directional message data set of the corresponding tenant to the client landing database, so that the multi-tenant delivery state management table keeps the same data as the client landing database, in order to directly track the data delivery progress in the cloud. At the same time, the tenant delivery service module listens to the state synchronization data set of the landing state feedback sent by the client receiving service module, and updates to the multi-tenant delivery state management table, that is, the tenant delivery service module is used as a medium, and the multi-tenant delivery state management table, the stream text directional message protocol data set of the tenant, the client receiving service module and the client landing database form a cycle, so that the data delivery state can be monitored and tracked, high-time-efficiency data synchronization can be realized, and timely retransmission or supplementary delivery of data can be supported. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 It is a flowchart of a gateway-based data delivery method in one embodiment;

[0023] Figure 2 Fig. 1 is a schematic diagram of a gateway-based data delivery system according to an embodiment;

[0024] Figure 3 Fig. 2 is a schematic diagram of an internal structure of a computer device according to an embodiment. DETAILED DESCRIPTION

[0025] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0026] In one embodiment, as shown in Fig. 1, a gateway-based data delivery method is provided, comprising the following steps: Figure 1

[0027] Step 102, synchronizing the data to be delivered to a synchronization landing database in real time.

[0028] Step 104, pushing the data to be delivered in the synchronization database to a stream text directional message protocol data set having corresponding data authority through a tenant delivery service module, and persisting the data to be delivered to a multi-tenant delivery state management table.

[0029] Each tenant corresponds to at least one stream text directional message protocol data set. In order to avoid data channel congestion, it is not limited that each tenant has only one stream text directional message protocol data set, i.e. Topic, which can be configured through a management end. At the same time, a user management module can isolate user data by tenant ID in a multi-tenant form, use the same set of system and resources to provide services for multiple different users, and support time partition isolation of tenant historical data, which is convenient for operation and maintenance.

[0030] Step 106, listening to the stream text directional message protocol data set of the corresponding tenant through a client receiving service module and persisting to a client landing database, and sending a landing state to a state synchronization data set.

[0031] The landing state refers to that the client confirms to receive the required data and the data processing is normal. The landing state can be sent simultaneously with an ACK confirmation, which informs the cloud gateway that the data has been synchronized. The purpose of sending the landing state is to monitor whether the data delivery is normal at the cloud end, and it can also be used for data re-delivery. It is worth noting that the monitoring is to monitor whether there is an exception in the data delivery process; the listening is to capture data changes, i.e. whether there is new data or data change.

[0032] ​The to-be-delivered data is persisted in the multi-tenant delivery state management table, and the flow text targeted message data set of the corresponding tenant is persisted in the client landing library, so that the multi-tenant delivery state management table maintains the same data as the client landing library, in order to directly track the data delivery progress in the cloud, that is, to solve the problem of data synchronization, so that when the data synchronization problem occurs, the problem can be checked in the tenant persistent library in the cloud, that is, the multi-tenant delivery state management table, which is equivalent to checking in the client landing library. Specifically, it is determined which data needs to be retransmitted by querying the state. Timed tasks can be set to scan or manually selected according to the maintenance page to trigger.

[0033] In step 108, the tenant delivery service module listens to the state synchronization data set of the landing state feedback sent by the client receiving service module, and updates to the multi-tenant delivery state management table.

[0034] As can be seen, the multi-tenant delivery state management table, the tenant's flow text targeted message protocol data set, the client receiving service module, and the client landing library constitute a cycle through the tenant delivery service module as a medium, so that the data delivery state can be monitored and tracked, high-efficiency data synchronization can be achieved, and timely retransmission or retransmission of data is supported. 7*24 hours online service is supported.

[0035] In step 110, the application service module listens to the client landing library, and synchronizes the delivery data in the client landing library to the database of the application service.

[0036] Specifically, data synchronization is performed according to the respective business logic.

[0037] In the above-mentioned gateway-based data delivery method, the to-be-delivered data is persisted in the multi-tenant delivery state management table, and the flow text targeted message data set of the corresponding tenant is persisted in the client landing library, so that the multi-tenant delivery state management table maintains the same data as the client landing library, in order to directly track the data delivery progress in the cloud. At the same time, the tenant delivery service module listens to the state synchronization data set of the landing state feedback sent by the client receiving service module, and updates to the multi-tenant delivery state management table, that is, the multi-tenant delivery state management table, the tenant's flow text targeted message protocol data set, the client receiving service module, and the client landing library constitute a cycle through the tenant delivery service module as a medium, so that the data delivery state can be monitored and tracked, high-efficiency data synchronization can be achieved, and timely retransmission or retransmission of data is supported.

[0038] In summary, the method supports configuring data delivery rules at the tenant level, can provide high-reliable 7*24-hour online service, supports high-efficiency data synchronization and data retransmission.

[0039] In one embodiment, the data to be issued is synchronized in real time to the synchronization landing library, including:

[0040] The business data to be issued is carried by json.

[0041] When a new data source is added, the access of data is completed through a special flink job configuration.

[0042] The data to be issued is synchronized in real time to the synchronization landing library by using the data processing platform.

[0043] The data gateway does not care about the specific data format: the whole data message is saved as a json string on the data_payload field of the data table of the gateway, and it does not care about the format of the source data message. Therefore, when a new data source or data type is added, the cloud service does not need to be restarted, and 7*24 online service can be provided without downtime in the scenario of adding data sources / data types, and the data message can be arbitrarily extended.

[0044] In one embodiment, the tenant issuing service module pushes the data to be issued in the synchronization database to the stream text directional message protocol data set that has corresponding data rights, including:

[0045] After the tenant issuing service module receives the data to be issued in the synchronization database, it searches for tenants with corresponding data rights in the pre-constructed multi-tenant data rights library according to the data to be issued, and pushes the data to be issued to the stream text directional message protocol data set with corresponding data rights.

[0046] The multi-tenant data rights library is provided by the tenant management service module.

[0047] In one embodiment, after the tenant issuing service module listens to the state synchronization data set of the landing state feedback sent by the client receiving service module and updates to the multi-tenant issuing state management table, it further includes:

[0048] If the multi-tenant issuing state management table monitors data that is issued abnormally, the data to be issued in the state synchronization data set is retrieved to resend or supplement the data that needs to be resent or supplemented.

[0049] In one embodiment, the method further includes:

[0050] When the data to be issued is non-text data, the non-text data is issued to the cloud OSS, and the application service module determines whether to download and locally persist the non-text data.

[0051] Non-text data, i.e. audio, video or picture.

[0052] In one embodiment, the data resources in the cloud OSS are provided with protection measures such as access tokens and validity periods to avoid unlimited access to the data.

[0053] It should be understood that, although Figure 1 The steps in the flowcharts are shown in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the flowcharts can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be alternately executed with other steps or at least part of the sub-steps or stages of other steps.

[0054] In one embodiment, as shown in Figure 2 A gateway-based data delivery system is provided, comprising:

[0055] a cloud gateway, a gateway client, and an application system;

[0056] The cloud gateway includes a synchronization landing database, a tenant delivery service module, a flow text directional message protocol data set, and a multi-tenant delivery state management table.

[0057] The gateway client includes a client receiving service module and a client landing database.

[0058] The application system includes an application service module and a database of the application service.

[0059] The synchronization landing database is used for real-time synchronization of data to be delivered.

[0060] The tenant delivery service module is used for pushing the data to be delivered in the synchronization database to the flow text directional message protocol data set that has the corresponding data authority, and persisting the data to be delivered to the multi-tenant delivery state management table; each tenant corresponds to at least one flow text directional message protocol data set.

[0061] The client receiving service module listens to the flow text directional message protocol data set of the corresponding tenant and persists to the client landing database, and sends the landing state to the state synchronization data set.

[0062] The tenant delivery service module listens to the state synchronization data set of the landing state feedback sent by the client receiving service module, and updates to the multi-tenant delivery state management table.

[0063] The application service module listens to the client landing library, and synchronizes the data issued in the client landing library to the database of the application service.

[0064] In one embodiment, the system further comprises a tenant management service module;

[0065] The tenant management service module is configured to provide a multi-tenant data authority library.

[0066] After the tenant issuing service module receives the data to be issued in the synchronization database, the tenant management service module searches for a tenant having a corresponding data authority in the pre-constructed multi-tenant data authority library according to the data to be issued, and pushes the data to be issued to the flow text targeted message protocol data set having the corresponding data authority.

[0067] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 3 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data to be issued. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a gateway-based data issuing method.

[0068] Those skilled in the art can understand that Figure 3 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0069] In one embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method in the above-mentioned embodiments.

[0070] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the method in the above-mentioned embodiments.

[0071] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0072] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0073] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent protection of the present application should be subject to the appended claims.

Claims

1. A gateway-based data delivery method, characterized in that, The method comprises: synchronizing the data to be delivered to a synchronization landing library in real time; pushing the data to be delivered in the synchronization landing library to a stream text targeted message protocol data set with corresponding data authority through a tenant delivery service module, while persisting the data to be delivered to a multi-tenant delivery state management table; each tenant corresponds to at least one stream text targeted message protocol data set; listening to the stream text targeted message protocol data set of the corresponding tenant through a client receiving service module and persisting to a client landing library, while sending a landing state to a state synchronization data set; listening to the state synchronization data set of the landing state feedback sent by the client receiving service module through the tenant delivery service module, and updating to the multi-tenant delivery state management table; listening to the client landing library through an application service module, and synchronizing the delivery data in the client landing library to the database of the application service; the pushing of the data to be delivered in the synchronization landing library to the stream text targeted message protocol data set with corresponding data authority through the tenant delivery service module comprises: after the tenant delivery service module receives the data to be delivered in the synchronization landing library, searching for tenants with corresponding data authority in a pre-constructed multi-tenant data authority library according to the data to be delivered, and pushing the data to be delivered to the stream text targeted message protocol data set with corresponding data authority; the multi-tenant data authority library is provided by a tenant management service module; the synchronization landing library, the tenant delivery service module, the stream text targeted message protocol data set, the multi-tenant delivery state management table, the state synchronization data set, and the tenant management service module are arranged in a cloud gateway; the client receiving service module and the client landing library are arranged in a gateway client; the application service module and the database of the application service are arranged in an application system.

2. The method of claim 1, wherein, synchronizing the data to be delivered to a synchronization landing library in real time comprises: carrying the business data to be delivered by json; when a new data source is added, completing the access of data through a special flink job configuration; synchronizing the carried or accessed data to be delivered to the synchronization landing library in real time by using a data processing platform.

3. The method of claim 1, wherein, after listening to the state synchronization data set of the landing state feedback sent by the client receiving service module through the tenant delivery service module and updating to the multi-tenant delivery state management table, the method further comprises: if the data to be delivered persistently in the state synchronization data set is monitored to be abnormally delivered through the multi-tenant delivery state management table, resending or supplementing the data that needs to be resent or supplemented.

4. The method of claim 1, wherein, The method further comprises: when the data to be delivered is non-text data, delivering the non-text data to a cloud OSS, and determining whether to download and locally persist the non-text data by an application service module.

5. The method of claim 4, wherein, The data resources in the cloud OSS are provided with an access token and a validity period. 6.A gateway-based data delivery system, characterized in that, The system comprises: a cloud gateway, a gateway client, and an application system; The cloud gateway comprises a synchronization landing library, a tenant issuing service module, a stream text directional message protocol data set, a multi-tenant issuing state management table, a state synchronization data set and a tenant management service module; The gateway client comprises a client receiving service module and a client landing library; The application system comprises an application service module and a database of the application service; The synchronization landing library is used for real-time synchronization of data to be issued; The tenant issuing service module is used for pushing the data to be issued in the synchronization landing library to a stream text directional message protocol data set with corresponding data authority, and persisting the data to be issued to a multi-tenant issuing state management table; each tenant corresponds to at least one stream text directional message protocol data set; The client receiving service module listens to the stream text directional message protocol data set of the corresponding tenant and persists to the client landing library, and sends a landing state to the state synchronization data set; The tenant issuing service module listens to the state synchronization data set of the landing state feedback sent by the client receiving service module, and updates to the multi-tenant issuing state management table; The application service module listens to the client landing library, and synchronizes the issuing data in the client landing library to the database of the application service; The tenant management service module is used for providing a multi-tenant data authority library; After the tenant issuing service module receives the data to be issued in the synchronization landing library, the tenant with corresponding data authority is searched in the pre-constructed multi-tenant data authority library according to the data to be issued, and the data to be issued is pushed to the stream text directional message protocol data set with corresponding data authority. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The processor executes the computer program to realize the steps of the method in any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 5.

Images