A data transmission method and system based on a distributed server

Abstract

The application discloses a data transmission method and system based on a distributed server, when an application instance needs to transmit data, an application identifier is configured for the data and written directly into a local cache queue; a background process program of a server where the application instance is located scans the data in the local cache queue regularly, and sends the aggregated scanned data to a registration center; the registration center needs to unpack the data packet after receiving the data packet, and obtains the data transmitted by each application instance according to the application identifier. Compared with the method in the prior art that each application instance needs to create a connection with the registration center for interaction, the data transmission method of the application specially manages the connection to the same registration center in one server, and the application instance on one server only creates a connection with the same registration center, so that the number of connections of the registration center can be reduced, and the number of instances, the service performance and the service efficiency of the registration center service can be improved.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, and in particular to a data transmission method and system based on a distributed server. Background Technology

[0002] Currently, most server-side systems adopt a distributed deployment approach to meet requirements such as high concurrency and high availability. This distributed deployment architecture requires numerous application instances to be deployed across different machines. The popular microservice architecture also deploys many instances across multiple machines. Furthermore, in a microservice architecture, a registry center exists, and all microservice instances need to register with this center. After successful registration, each client establishes a connection with the registry center, transmitting heartbeats and monitoring information through this connection. The registry center then aggregates and performs subsequent analysis and processing.

[0003] As the number of service instances supported by microservices increases, each instance needs to establish a connection with the registry center to transmit heartbeats. After the number of connected microservices reaches a certain level, when network anomalies or other abnormal situations occur, the information communication between the registry center and each client will increase with the increase in the number of connections and heartbeat transmissions, which will place a huge burden on the registry center. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a data transmission method and system based on a distributed server, which can reduce the number of connections between instances on the server and the registry center, and improve the number of instances served by the registry center as well as service performance and efficiency.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A data transmission method based on a distributed server includes the following steps:

[0007] The application instance's registry client obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client.

[0008] The server where the application instance is located periodically scans the data in the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and then sends the first data packet to the registration center;

[0009] The registration center unpacks the first data packet and obtains the first data transmitted by each application instance according to the application identifier.

[0010] In order to solve the above technical problems, another technical solution adopted by the present application is:

[0011] A data transmission system based on a distributed server comprises an application instance, a server and a registration center.

[0012] The registration center client of the application instance acquires first data to be transmitted, configures a corresponding application identifier for the first data to obtain to-be-transmitted data, and writes the to-be-transmitted data into a local cache queue of the registration center client.

[0013] The server where the application instance is located scans the data of the local cache queue of the registration center client through a corresponding background process program in a timely manner, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center.

[0014] The registration center unpacks the first data packet and obtains the first data transmitted by each application instance according to the application identifier.

[0015] The present application has the beneficial effects that when the application instance needs to transmit data, an application identifier is configured for the data, and the data is directly written into the local cache queue; the background process program of the server where the application instance is located scans the data in the local cache queue in a timely manner, aggregates the scanned data and sends the aggregated data to the registration center; and the registration center unpacks the data packet after receiving the data packet and obtains the data transmitted by each application instance according to the application identifier. Compared with the method in the prior art in which each application instance needs to create a connection with the registration center for interaction, the data transmission method of the present application specially manages the connection to the same registration center in one server, and the application instance on one server only creates a connection with the same registration center, so that the number of connections of the registration center can be reduced, and the number of instances, service performance and efficiency of the registration center service can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of a data transmission method based on a distributed server according to an embodiment of the present application;

[0017] Figure 2 A schematic diagram of a data transmission system based on a distributed server according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] To explain the technical content, purposes and effects of the present application in detail, the following will be described in combination with the embodiments and the accompanying drawings.

[0019] Please refer to Figure 1 The embodiment of the present application provides a data transmission method based on a distributed server, which comprises the following steps:

[0020] The registration center client of the application instance acquires first data to be transmitted, configures a corresponding application identifier for the first data, obtains to-be-transmitted data, and writes the to-be-transmitted data into a local cache queue of the registration center client;

[0021] The server where the application instance is located scans data of the local cache queue of the registration center client through a corresponding background process program in a time manner, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center.

[0022] The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

[0023] From the above description, it can be known that the application has the beneficial effects that when an application instance needs to perform data transmission, an application identifier is configured for the data, and the data is directly written into a local cache queue; a background process program of a server where the application instance is located scans data in the local cache queue in a time manner, aggregates the scanned data, and sends the aggregated data to the registration center; and the registration center needs to unpack the data packet after receiving the data packet, and obtains data transmitted by each application instance according to the application identifier. Compared with the method in the prior art in which each application instance needs to create a connection with the registration center to interact, the data transmission method of the application specially manages the connection to the same registration center in one server, and an application instance on one server only creates a connection with the same registration center, so that the number of connections of the registration center can be reduced, and the number of instances, the service performance, and the service efficiency of the registration center can be improved.

[0024] Further, the application further comprises:

[0025] When the registration center sends a task to the registration center client, the second data to be sent is configured according to the application identifier of the application instance, the configured second data is aggregated into a second data packet, and the second data packet is sent to the server where the application instance is located;

[0026] The server unpacks the second data packet through a corresponding background process program, and sends the unpacked second data to the local cache queue of the corresponding registration center client according to the application identifier.

[0027] From the above description, through the connection of the registration center and the server where the application is located, the tasks to be sent to the application instance can be aggregated and sent to the corresponding server. The server unpacks the aggregated data packet and distributes the second data to the local cache queue of the corresponding registration center client according to the application identifier. By connecting the registration center and the server where the application instance is located, the data transmission between the registration center client in the application instance and the registration center can be realized, and the registration center only needs to send one message to the server, further improving the efficiency of data transmission.

[0028] Further, the second data obtained by unpacking is sent to the local cache queue of the corresponding registration center client, and then includes:

[0029] The registration center client obtains the second data from the local cache queue and pulls task data from the registration center according to the second data.

[0030] From the above description, after the registration center client receives the task sent by the registration center, it can actively pull task data from the registration center, ensuring data transmission between the registration center client and the registration center.

[0031] Further, the server where the application instance is located scans the data of the local cache queue of the registration center client by a corresponding background process program at regular intervals, and before includes:

[0032] The registration center client judges whether the server where the application instance is located has a corresponding background process program. If yes, the background process program directly establishes a connection with the registration center, otherwise, a background process program is created and the background process program establishes a connection with the registration center.

[0033] From the above description, the registration center client and the registration center perform data transmission through the background process program of the server where the application instance is located. The background process program can be created by the registration center client corresponding to the application instance, ensuring that the connection between the server and the registration center is not interrupted.

[0034] Further, the background process program establishes a connection with the registration center, including:

[0035] When there are multiple registration centers, the background process program establishes a connection with the registration center and configures a unique identifier of the registration center for the connection.

[0036] From the above description, if there are multiple registration centers, there will be multiple connections, so the unique identifier of the registration center is used for differentiation, ensuring the normal operation of data transmission.

[0037] Please refer to Figure 2Another embodiment of the present application provides a distributed server-based data transmission system, comprising an application instance, a server and a registration center;

[0038] The registration center client of the application instance acquires first data to be transmitted, configures a corresponding application identifier for the first data to obtain to-be-transmitted data, and writes the to-be-transmitted data into a local cache queue of the registration center client;

[0039] The server where the application instance is located scans data of the local cache queue of the registration center client through a corresponding background process program in a time manner, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center;

[0040] The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

[0041] From the above description, it can be known that the present application has the following beneficial effects: when an application instance needs to perform data transmission, an application identifier is configured for data, and the data is directly written into a local cache queue; a background process program of a server where the application instance is located scans data in the local cache queue in a time manner, aggregates the scanned data, and sends the aggregated data to a registration center; the registration center receives a data packet, unpacks the data packet, and obtains data transmitted by each application instance according to an application identifier. Compared with a method in the prior art in which each application instance needs to create a connection with a registration center to interact, the data transmission method of the present application specially manages a connection to the same registration center in a server, and an application instance on a server only creates a connection with the same registration center, so that the number of connections of the registration center can be reduced, and the number of instances, service performance and efficiency of the registration center service can be improved.

[0042] Further, the present application further comprises:

[0043] When the registration center sends a task to the registration center client, the registration center configures second data to be sent according to an application identifier of the application instance, aggregates the configured second data into a second data packet, and sends the second data packet to a server where the application instance is located;

[0044] The server unpacks the second data packet through a corresponding background process program, and sends second data obtained by unpacking to a local cache queue of a corresponding registration center client according to the application identifier.

[0045] From the above description, through the connection of the registration center and the server where the application is located, the tasks to be sent to the application instance can be aggregated and sent to the corresponding server. The server unpacks the aggregated data packet and distributes the second data to the local cache queue of the corresponding registration center client according to the application identifier. By connecting the registration center and the server where the application instance is located, the data transmission between the registration center client in the application instance and the registration center can be realized, and the registration center only needs to send one message to the server, further improving the efficiency of data transmission.

[0046] Further, the second data obtained by unpacking is sent to the local cache queue of the corresponding registration center client, and then includes:

[0047] The registration center client obtains the second data from the local cache queue and pulls task data from the registration center according to the second data.

[0048] From the above description, after the registration center client receives the task sent by the registration center, it can actively pull task data from the registration center, ensuring data transmission between the registration center client and the registration center.

[0049] Further, the server where the application instance is located scans the data of the local cache queue of the registration center client through the corresponding background process program at regular intervals, and before includes:

[0050] The registration center client judges whether the server where the application instance is located has a corresponding background process program. If yes, the background process program directly establishes a connection with the registration center, otherwise, a background process program is created and the background process program establishes a connection with the registration center.

[0051] From the above description, the registration center client and the registration center perform data transmission through the background process program of the server where the application instance is located. The background process program can be created by the registration center client corresponding to the application instance, ensuring that the connection between the server and the registration center is not interrupted.

[0052] Further, the background process program establishes a connection with the registration center, which includes:

[0053] When there are multiple registration centers, the background process program establishes a connection with the registration center and configures a unique identifier of the registration center for the connection.

[0054] From the above description, if there are multiple registration centers, there will be multiple connections, so the unique identifier of the registration center is used for differentiation, ensuring the normal operation of data transmission.

[0055] The data transmission method and system based on the distributed server are suitable for the micro-service architecture, reduce the number of connections between the instances on the server and the registration center, and improve the number of instances, service performance and efficiency of the registration center service. The following will be described through specific embodiments.

[0056] Embodiment one

[0057] Please refer to Figure 1 A data transmission method based on a distributed server, comprising the steps of:

[0058] S1, the registration center client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the to-be-transmitted data, and writes the to-be-transmitted data into the local cache queue of the registration center client.

[0059] In this embodiment, it is assumed that there is already a registration center in the micro-service architecture, and an application instance is deployed on a server. At this time, the application instance integrates a registration center client, and a proxy layer is added to the registration center client.

[0060] The registration center client on the server and the background process program interact through the local cache queue. When the registration center client needs to send data, such as heartbeat information, it directly writes the heartbeat message into the local cache queue. The content written mainly includes the application unique identifier and the message type

heartbeat type

[0061] In this embodiment, the transmission of heartbeat information can also be processed, and the transmission of other data such as monitoring data can also be processed.

[0062] S2, the server where the application instance is located scans the data of the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center.

[0063] Specifically, the registration center client judges whether the server where the application instance is located exists the corresponding background process program. If yes, the background process program directly establishes a connection with the registration center, otherwise, the background process program is created, and the background process program establishes a connection with the registration center.

[0064] In the embodiment, it is judged by the proxy layer of the registry client whether there is a separate background process program on the server and the program has been started; if not, the proxy layer actively creates the background process program and a connection is created between the program and the registry; if yes, the program directly uses the connection established with the registry. Through the step, it is ensured that only one connection is communicated with the same registry on a server.

[0065] When there are multiple registries, the background process program establishes a connection with the registry and configures a unique identifier of the registry for the connection.

[0066] In the embodiment, if there are multiple registries, there will be multiple connections, which are distinguished by the unique identifiers of the registries.

[0067] S3, the registry unpacks the first data packet and obtains the first data transmitted by each application instance according to the application identifier.

[0068] In the embodiment, when the registry receives the heartbeat packet, the heartbeat packet needs to be unpacked, the corresponding heartbeat message is obtained according to different application unique identifiers, and subsequent processing is performed.

[0069] Embodiment Two

[0070] The embodiment is different from the embodiment one in that the data transmission method when the registry needs to notify the related application instances to perform tasks is further limited.

[0071] When the registry sends tasks to the registry client, the second data to be sent is configured according to the application identifier of the application instance, the configured second data is aggregated into a second data packet and sent to the server where the application instance is located.

[0072] The server unpacks the second data packet through the corresponding background process program and sends the unpacked second data to the local cache queue of the corresponding registry client according to the application identifier.

[0073] The registry client obtains the second data from the local cache queue and pulls task data from the registry according to the second data.

[0074] Specifically, when the registry needs to notify the related registry client to perform tasks, the aggregated message packet can be sent to the registry client in a batch aggregation manner, unpacked by the background process program and sent to the local cache queue, and read by the corresponding registry client.

[0075] For example, the registry center can send a message to request the client to reacquire the latest service list information on the registry center side. At this time, only one notification message needs to be sent, and after the client receives it, it actively pulls the service list information from the registry center. At this time, the registry center only needs to send one copy of the latest service list information to the connection.

[0076] Embodiment Three

[0077] Please refer to Figure 2 A data transmission system based on a distributed server includes an application instance, a server, and a registry center.

[0078] The registry center client of the application instance acquires first data to be transmitted, configures a corresponding application identifier for the first data to obtain to-be-transmitted data, and writes the to-be-transmitted data into a local cache queue of the registry center client.

[0079] The registry center client determines whether a corresponding background process program exists on the server where the application instance is located. If yes, the background process program directly establishes a connection with the registry center. Otherwise, the background process program is created, and the background process program establishes a connection with the registry center. When multiple registry centers exist, the background process program establishes a connection with the registry center, and configures a unique identifier of the registry center for the connection.

[0080] The server where the application instance is located scans the data in the local cache queue of the registry center client through the corresponding background process program in a timely manner, aggregates the scanned data into a first data packet, and sends the first data packet to the registry center.

[0081] The registry center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

[0082] In some embodiments, the method further includes:

[0083] When the registry center sends a task to the registry center client, the second data to be sent is configured according to the application identifier of the application instance, the configured second data is aggregated into a second data packet, and the second data packet is sent to the server where the application instance is located.

[0084] The server unpacks the second data packet through the corresponding background process program, and sends the unpacked second data to the local cache queue of the corresponding registry center client according to the application identifier.

[0085] The registry center client acquires the second data from the local cache queue, and pulls task data from the registry center according to the second data.

[0086] In summary, the application provides a data transmission method and system based on a distributed server. When an application instance needs to transmit data, an application identifier is configured for the data and written directly into a local cache queue. A background process program of a server where the application instance is located scans the data in the local cache queue at regular intervals, aggregates the scanned data, and sends the aggregated data to a registration center. The registration center receives the data packet, unpacks it, and obtains the data transmitted by each application instance according to the application identifier. Similarly, when the registration center needs to notify a related client to perform a task, a batch aggregation information mode can also be used to send an aggregated message packet to the client, and the background process program unpacks the message packet and sends it to the local cache queue, which is read by the corresponding client. Compared with the method in the prior art in which each application instance needs to create a connection with the registration center for interaction, the data transmission method of the application specially manages the connection to the same registration center in one server, and the application instance on one server only creates a connection with the same registration center, so that the number of connections to the registration center can be reduced, and the number of instances, service performance, and efficiency of the registration center can be improved.

[0087] The above description is only an embodiment of the application, and does not limit the patent scope of the application. Any equivalent transformation or direct or indirect application in the related technical field based on the content of the specification and drawings is also included in the patent protection scope of the application.

Claims

1. A data transmission method based on a distributed server, characterized by, The method comprises the steps of: The application instance of the registration center client obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registration center client; The server where the application instance is located scans the data in the local cache queue of the registration center client through the corresponding background process program in a timely manner, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center; The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier; The server where the application instance is located scans the data in the local cache queue of the registration center client through the corresponding background process program in a timely manner, and previously comprises the steps of: The registration center client judges whether the server where the application instance is located has a corresponding background process program, if yes, directly establishes a connection between the background process program and the registration center, otherwise, creates a background process program and establishes a connection between the background process program and the registration center.

2. The data transmission method based on distributed servers according to claim 1, characterized in that, Further comprising: When the registration center sends a task to the registration center client, the second data to be sent is configured according to the application identifier of the application instance, the configured second data is aggregated into a second data packet, and the second data packet is sent to the server where the application instance is located; The server unpacks the second data packet through the corresponding background process program, and sends the second data obtained by unpacking to the local cache queue of the corresponding registration center client according to the application identifier.

3. The data transmission method based on distributed servers according to claim 2, characterized in that, The second data obtained by unpacking is sent to the local cache queue of the corresponding registration center client, and then comprises the steps of: The registration center client obtains the second data from the local cache queue, and pulls task data from the registration center according to the second data.

4. The data transmission method based on distributed servers according to claim 1, characterized in that, The connection between the background process program and the registration center comprises the steps of: When there are multiple registration centers, the background process program establishes a connection with the registration center, and configures a unique identifier of the registration center for the connection.

5. A distributed server-based data transmission system, characterized in that The method comprises the steps of: The application instance of the registration center client obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registration center client; The server where the application instance is located scans the data in the local cache queue of the registration center client through the corresponding background process program in a timely manner, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center; The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier; The server where the application instance is located scans the data in the local cache queue of the registration center client through the corresponding background process program in a timely manner, and previously comprises the steps of: The registration center client judges whether the server where the application instance is located has a corresponding background process program, if yes, directly establishes a connection between the background process program and the registration center, otherwise, creates a background process program and establishes a connection between the background process program and the registration center.

6. The data transmission system based on distributed servers according to claim 5, characterized in that, Further comprising: When the registration center sends a task to the registration center client, the second data to be sent is configured according to the application identifier of the application instance, the configured second data is aggregated into a second data packet, and the second data packet is sent to the server where the application instance is located; The server unpacks the second data packet through a corresponding background process program, and sends the second data obtained by unpacking to the local cache queue of the corresponding registration center client according to the application identifier.

7. The data transmission system based on distributed servers according to claim 6, characterized in that, The second data obtained by unpacking is sent to the local cache queue of the corresponding registration center client, and then includes: The registration center client obtains the second data from the local cache queue, and pulls task data from the registration center according to the second data.

8. The data transmission system based on distributed servers according to claim 5, characterized in that, The connection between the background process program and the registration center includes: When there are multiple registration centers, the background process program establishes a connection with the registration center, and configures a unique identifier of the registration center for the connection.

Images