Method for synchronizing dynamic and static data of embedded cloud platform distributed high-availability architecture

Abstract

The application discloses a kind of dynamic and static data synchronization method of embedded cloud platform distributed high availability architecture, belong to the field of management embedded node distributed cloud platform, including steps: S1, build embedded cloud platform distributed high availability architecture, at least contain two and above cloud platform master server;S2, embedded cloud platform distributed high availability architecture system runs, including: when in real-time dynamic data synchronization, long connection is established with cloud platform server to embedded board card, and cloud platform generates the virtual agent of corresponding board card chip, and real-time broadcast chip information to all distributed cloud platform servers by virtual agent;When static data that needs to be synchronized by cloud platform is landed, cloud platform master node broadcasts data to all backup node servers by rsync component synchronization.This application can realize the real-time synchronization of dynamic and static data by pure software mode, without increasing system complexity, using relatively simple and less dependent component library.

Description

Technical Field

[0001] This invention relates to the field of distributed cloud platforms for managing embedded nodes, and more specifically, to a method for synchronizing dynamic and static data in a distributed high-availability architecture for embedded cloud platforms. Background Technology

[0002] A distributed high-availability architecture consists of two or more servers. During system operation, data needs to be synchronized in real time. Currently, the following methods are commonly used:

[0003] (1) Use distributed storage systems (ETCD, Redis) or message middleware (Kafka) to achieve data synchronization, replication, data consistency and other functions;

[0004] (2) Build a distributed database on the server and use the existing data functions of the distributed database to ensure time synchronization;

[0005] (3) Build a shared disk array for distributed services and use the shared disk to save the data to disk;

[0006] (4) Introduce third-party data synchronization related application libraries into the platform service and develop and implement custom data synchronization functions.

[0007] Because the embedded cloud platform service has limited OPS (operations per second), limited amount of synchronized data, static data is only the algorithm files, application files and configuration files required by the embedded nodes, and dynamic data is only the real-time board status and task status of the embedded nodes, the software operation and maintenance costs required by methods (1) and (2) are high, which greatly increases the complexity of the cloud platform system and occupies a certain amount of CPU and memory resources; method (3) requires the construction of a shared disk, changes the system hardware architecture, increases costs, and cannot make full use of the server disk storage resources, and cannot synchronize the data in memory, only sharing the data written to disk. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for synchronizing dynamic and static data in a distributed high-availability architecture of an embedded cloud platform. This method can achieve dynamic and static data synchronization in a distributed high-availability architecture of an embedded cloud platform without increasing system complexity, through a pure software approach and using a simpler, less dependent component library.

[0009] The objective of this invention is achieved through the following approach:

[0010] A method for synchronizing dynamic and static data in a distributed high-availability architecture of an embedded cloud platform includes:

[0011] Step S1: Build a distributed high-availability architecture for the embedded cloud platform, which includes at least two or more primary and backup cloud platform servers.

[0012] Step S2, during the operation of the embedded cloud platform distributed high availability architecture system, includes: when real-time dynamic data synchronization is performed, a long connection is established between the embedded board and the cloud platform server, the cloud platform generates a virtual agent for the corresponding board chip, and broadcasts chip information to all distributed cloud platform servers in real time through the virtual agent; when synchronizing static data that needs to be written to the cloud platform disk, the cloud platform master node synchronously broadcasts the data to all backup node servers.

[0013] Furthermore, in step S1, the embedded cloud platform's distributed high-availability architecture ensures that the system can handle abnormal failures and meet high availability requirements.

[0014] Furthermore, in step S1, the embedded cloud platform distributed high-availability architecture has distributed cloud platform services and manages multiple embedded board chips.

[0015] Furthermore, in step S2, after the embedded board establishes a long connection with the cloud platform server, the following sub-steps are included: real-time reporting of dynamic data of the embedded board chip, the dynamic data including status data and task data; and simultaneously synchronizing the static data uploaded to the cloud platform, the static data including algorithms, applications, and configuration files.

[0016] Furthermore, in step S2, the long connection is a TCP long connection.

[0017] Furthermore, in step S2, the establishment of a long connection between the embedded board and the cloud platform server, and the generation of a virtual agent for the corresponding board chip by the cloud platform, specifically includes the following sub-steps: the embedded board will actively establish a long connection with the cloud platform after the board goes online, and after the long connection is established, the cloud platform will establish a virtual agent for each chip in the board.

[0018] Furthermore, in step S2, the step of broadcasting chip information to all distributed cloud platform servers in real time through a virtual agent specifically includes the following sub-steps: the chip's virtual agent receives dynamic data reported by the board chip in real time and broadcasts it to the cloud platform server using gRPC.

[0019] Furthermore, in step S2, when the synchronization requires static data to be written to the cloud platform disk, the cloud platform master node synchronously broadcasts the data to all backup node servers. This specifically includes the following sub-steps: the cloud platform writes static data to disk and saves it in units of files, and uses the Linux system's general component rsync to synchronously broadcast the files to all backup cloud platform servers.

[0020] Furthermore, the real-time synchronization of static data stored on the cloud platform includes four scenarios: adding, deleting, modifying, and querying.

[0021] Furthermore, when the cloud platform saves static data to disk, if a cloud platform server is unavailable due to a failure, the process includes the following sub-step: comparing and pushing static data after the server recovers.

[0022] The beneficial effects of this invention include:

[0023] This invention is well-suited for managing distributed service architectures with embedded nodes. It achieves real-time synchronization of dynamic and static data in a purely software-based manner without increasing system complexity and using a simple, low-dependency component library. Furthermore, the data synchronization occurs on the cloud platform server side, eliminating the need for embedded nodes to add data synchronization business logic. It also meets the high availability requirements of distributed cloud platform servers, which can be scalably scaled. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the data synchronization system architecture according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the synchronous dynamic data flow according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the synchronous static data flow according to an embodiment of the present invention. Detailed Implementation

[0028] All features disclosed in all embodiments of this specification, or steps in all methods or processes implied in the disclosure, may be combined and / or extended or replaced in any way, except for mutually exclusive features and / or steps.

[0029] In view of the problems mentioned in the background, the inventors of this invention, targeting the characteristics of the distributed high-availability architecture of embedded cloud platforms and specifically addressing method 4, conceived a dynamic and static data synchronization method for the distributed high-availability architecture of embedded cloud platforms. This method is applied to a distributed cloud platform for managing embedded nodes and solves the following technical challenges:

[0030] 1) Data synchronization of dynamic and static data was achieved without increasing system complexity or changing the server hardware architecture (i.e., through pure software).

[0031] 2) A simple and convenient method is adopted to reduce the coupling between cloud platform services and third-party data synchronization libraries, thereby reducing dependencies.

[0032] 3) After the data synchronization function is configured, the distributed services can be dynamically added or removed to meet the requirements of elastic scaling and high availability during failures.

[0033] In the specific implementation method, according to as follows Figure 1 The data synchronization system architecture shown here needs to meet the requirements of a distributed high-availability architecture, meaning the system environment must meet the following requirements:

[0034] (1) A distributed high-availability architecture consists of two or more servers;

[0035] (2) The server operating system is Linux;

[0036] (3) Complete the rsync component configuration, configure the server account and password, and set the rsync component daemon passwordless data synchronization function;

[0037] (4) The cloud platform service completes the development of functions to receive embedded board TCP connections and generate board chip virtual agents;

[0038] (5) The cloud platform service completes the gRPC remote function call function and realizes the development of broadcast dynamic data function;

[0039] (6) The cloud platform service has completed the development of interfaces for receiving, adding, deleting, modifying and querying static data files.

[0040] like Figure 2 The diagram illustrates the synchronous dynamic data flow of this invention, which specifically includes the following steps:

[0041] (1) The embedded board actively establishes a long TCP connection with the cloud platform server;

[0042] (2) After receiving the embedded board connection, the cloud platform generates a virtual agent for the corresponding online board chip;

[0043] (3) The chip virtual agent is responsible for sending heartbeats to the corresponding board chip in real time, and monitoring the chip status and the task running status in the chip based on the information returned by the heartbeat.

[0044] (4) If the chip status is normal, the chip agent will run normally and send heartbeat commands periodically; if the chip virtual agent detects that the chip status is abnormal, the chip agent will broadcast the chip status information and task information to all distributed cloud platform services via gRPC.

[0045] like Figure 3The diagram illustrates the static disk data synchronization process of this invention, which specifically includes the following steps:

[0046] (1) The static data CRUD interfaces of the cloud platform service are called, and static data such as algorithms, applications, and configuration files are changed;

[0047] (2) The cloud platform service saves the changes to the corresponding static data files to the corresponding file directory on the local disk.

[0048] (3) The cloud platform service uses the rsync component to easily and quickly synchronize and broadcast changed static data, and finally the corresponding file directories of all backup node servers complete the static data synchronization.

[0049] It should be noted that, within the scope of protection defined in the claims of this invention, the following embodiments can be combined and / or extended or replaced in any logical manner from the above specific embodiments, such as the disclosed technical principles, disclosed technical features or implicitly disclosed technical features.

[0050] Example 1

[0051] A method for synchronizing dynamic and static data in a distributed high-availability architecture of an embedded cloud platform includes:

[0052] Step S1: Build a distributed high-availability architecture for the embedded cloud platform, which includes at least two or more primary and backup cloud platform servers.

[0053] Step S2, during the operation of the embedded cloud platform distributed high availability architecture system, includes: when real-time dynamic data synchronization is performed, a long connection is established between the embedded board and the cloud platform server, the cloud platform generates a virtual agent for the corresponding board chip, and broadcasts chip information to all distributed cloud platform servers in real time through the virtual agent; when synchronizing static data that needs to be written to the cloud platform disk, the cloud platform master node synchronously broadcasts the data to all backup node servers.

[0054] Example 2

[0055] Based on Example 1, in step S1, the embedded cloud platform distributed high availability architecture can ensure that the system can handle abnormal faults and meet high availability requirements.

[0056] Example 3

[0057] Based on Example 1, in step S1, the embedded cloud platform distributed high availability architecture has distributed cloud platform services and manages multiple embedded board chips.

[0058] Example 4

[0059] Based on Example 1, in step S2, after the embedded board establishes a long connection with the cloud platform server, the following sub-steps are included: real-time reporting of dynamic data of the embedded board chip, the dynamic data including status data and task data; and simultaneous synchronization of static data uploaded to the cloud platform, the static data including algorithms, applications, and configuration files.

[0060] Example 5

[0061] Based on Example 1, in step S2, the long connection is a TCP long connection.

[0062] Example 6

[0063] Based on Example 1, in step S2, the establishment of a long connection between the embedded board and the cloud platform server, and the generation of a virtual agent for the corresponding board chip by the cloud platform, specifically includes the following sub-steps: the embedded board will actively establish a long connection with the cloud platform after the board goes online, and after the long connection is established, the cloud platform will establish a virtual agent for each chip in the board.

[0064] Example 7

[0065] Based on Example 1, in step S2, the real-time broadcasting of chip information to all distributed cloud platform servers via a virtual agent specifically includes the following sub-steps: the chip's virtual agent receives dynamic data reported by the board chip in real time and broadcasts it to the cloud platform server using gRPC.

[0066] Example 8

[0067] Based on Example 1, in step S2, when the synchronization requires static data to be written to the cloud platform disk, the cloud platform master node synchronously broadcasts the data to all backup node servers. Specifically, this includes the following sub-steps: the cloud platform writes static data to disk and saves it in units of files, and uses the general component rsync of the Linux system to synchronously broadcast the files to all backup cloud platform servers.

[0068] Example 9

[0069] Based on Example 8, the real-time synchronization of static data stored on the cloud platform includes four scenarios: adding, deleting, modifying, and querying.

[0070] Example 10

[0071] Based on Example 8, when the cloud platform saves static data to disk, if a cloud platform server is unavailable due to a failure, the process includes the following sub-step: comparing and pushing static data after the server recovers.

[0072] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0073] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.

[0074] In another aspect, embodiments of the present invention also provide a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

Claims

1. A method for synchronizing dynamic and static data of an embedded cloud platform distributed high-availability architecture, characterized in that, include: Step S1: Build a distributed high-availability architecture for the embedded cloud platform, which includes at least two or more primary and backup cloud platform servers. Step S2, during the operation of the distributed high-availability architecture system of the embedded cloud platform, includes: when real-time dynamic data synchronization is performed, a long connection is established between the embedded board and the cloud platform server, the cloud platform generates a virtual agent for the corresponding board chip, and broadcasts chip information to all distributed cloud platform servers in real time through the virtual agent; when synchronizing static data that needs to be written to the cloud platform disk, the cloud platform master node synchronously broadcasts the data to all backup node servers. In step S2, establishing a long connection between the embedded board and the cloud platform server, and the cloud platform generating a virtual proxy for the corresponding board chip, specifically includes the following sub-steps: After the embedded board reports dynamic data, it will actively establish a long connection with the cloud platform after the board goes online. After the long connection is established, the cloud platform will establish a virtual agent for each chip in the board. In step S2, the step of broadcasting chip information to all distributed cloud platform servers in real time through a virtual agent specifically includes the following sub-steps: the chip's virtual agent receives dynamic data reported by the board chip in real time and broadcasts it to the cloud platform server using gRPC. In step S2, when the synchronization requires static data to be written to the cloud platform disk, the cloud platform master node synchronously broadcasts the data to all backup node servers. Specifically, this includes the following sub-steps: the cloud platform writes static data to disk and saves it in units of files, and uses the Linux system's general component rsync to synchronously broadcast the files to all backup cloud platform servers.

2. The method for synchronizing dynamic and static data in an embedded cloud platform distributed high-availability architecture according to claim 1, characterized in that, In step S1, the embedded cloud platform's distributed high-availability architecture ensures that the system can handle abnormal faults and meet high availability requirements.

3. The method for synchronizing dynamic and static data in an embedded cloud platform distributed high-availability architecture according to claim 1, characterized in that, In step S1, the embedded cloud platform distributed high availability architecture has distributed cloud platform services and manages multiple embedded board chips.

4. The method for synchronizing dynamic and static data in a distributed high-availability architecture of an embedded cloud platform according to claim 1, characterized in that, In step S2, after the embedded board establishes a long connection with the cloud platform server, the following sub-steps are included: real-time reporting of dynamic data of the embedded board chip, the dynamic data including status data and task data; and simultaneous synchronization of static data uploaded to the cloud platform, the static data including algorithms, applications, and configuration files.

5. The method of claim 1, wherein the method further comprises: In step S2, the long connection is a TCP long connection.

6. The method of claim 1, wherein the method further comprises: The real-time synchronization of static data stored on the cloud platform includes four scenarios: adding, deleting, modifying, and querying.

7. The method for synchronizing dynamic and static data in an embedded cloud platform distributed high-availability architecture according to claim 1, characterized in that, When the cloud platform saves static data to disk, if a cloud platform server is unavailable due to a failure, the process includes the following sub-steps: comparing and pushing static data after the server recovers.

Images