A cross-network database synchronization method and system of a cloud-native architecture

Abstract

The application relates to the technical field of databases, in particular to a cross-network database synchronization method and system of a cloud native architecture, which comprises the following steps: synchronization management service, synchronization resource control, scheduling management service, task execution and exchange transmission; the cloud native architecture cross-network database synchronization method and system have the advantages that the method and system combine the advantages of cloud native technology, light-weight data transmission technology and data acquisition technology, and provide more flexible, scalable, light-weight and easy-to-manage real-time synchronization capability for heterogeneous database synchronization.

Description

Technical Field

[0001] This invention relates to the field of database technology, specifically to a cross-network database synchronization method and system based on a cloud-native architecture. Background Technology

[0002] With the continuous development and popularization of cloud computing technology, cloud-native architecture has gradually become the mainstream choice for enterprises to build and maintain applications. Cloud-native architecture emphasizes the use of technologies such as containerization, microservices, and automated deployment and maintenance to improve the elasticity, scalability, and maintainability of applications. In a cloud-native environment, databases, as one of the core components of applications, face significant technical challenges in cross-network synchronization.

[0003] Traditional database synchronization methods often rely on fixed network environments and complex configurations, making them ill-suited to the flexibility and dynamism of cloud-native architectures. Especially in cross-network, weak network, or edge computing scenarios, the reliability and efficiency of database synchronization are difficult to guarantee. Furthermore, as enterprise businesses continue to expand, the requirements for real-time performance, consistency, and security in database synchronization are becoming increasingly stringent. Summary of the Invention

[0004] The purpose of this invention is to provide a cross-network database synchronization method and system based on a cloud-native architecture to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a cross-network database synchronization method for cloud-native architecture, the method comprising the following steps:

[0006] Synchronization management service provides a unified synchronization service interface and synchronization operation interface to the outside world;

[0007] Synchronization resource control provides management and control of custom Kubernetes resources required for synchronization, including instance resource controller, tenant resource controller, and task resource controller, which respectively manage custom instance resources, tenant resources, and task resources;

[0008] The scheduling and management service is responsible for generating and scheduling task execution plans, and maintaining the operational status of relevant metadata.

[0009] Task execution: Receives task execution plans from the scheduling and management service, executes the corresponding logic, and provides database collection and writing capabilities;

[0010] The switching transmission provides data transmission and caching capabilities, enabling data transmission in network isolation, weak network, and edge scenarios. It is divided into local switching transmission engine and central switching transmission engine.

[0011] Preferably, the specific operations for synchronous resource control include:

[0012] Synchronization resource control utilizes a set of Kubernetes custom controllers to create and manage custom resources. These controllers monitor and respond to the creation, update, and deletion of custom resources, providing lifecycle management for related resources and enabling automatic scaling, creation, and upgrades of related service modules as needed. The synchronization resource controllers include instance resource controllers, tenant resource controllers, and task resource controllers. Instance resource controllers correspond to synchronization service instance resources, describing the specifications and related configurations of basic services, and are responsible for the creation, scaling, and status monitoring of the synchronization service entity. Tenant resource controllers correspond to tenant-class custom resources, describing the specifications and configurations of a type of tenant-isolated computing and transmission resources. Tenant resource controllers are responsible for the creation, scaling, and destruction of execution units and local switching transmission engines. Task resource controllers correspond to task-class custom resources, describing the basic elements of a synchronization task, including the source of data, the flow of data, and the topological relationship information of data within the task. Through task controllers, the creation, update, and deletion of synchronization tasks are implemented, and the status of tasks can be modified through custom attributes, allowing for the starting, pausing, and stopping of resource-related task entities.

[0013] Preferably, the specific operations of the scheduling management service include:

[0014] The scheduling and management service is the core of database synchronization. It supports multi-replica deployment and is responsible for managing the system's metadata and maintaining the system's operating status. System metadata and operating status are classified into two categories: task information and tenant information. These are uniformly aggregated and transmitted through the exchange transmission engine. Tenant information specifically refers to the information of the execution unit, workload indicators, and their operating status, which is used to guide the scaling of tenant resources. Task information corresponds to the configuration information, execution results, and status of the subtasks associated with the synchronization task, supporting the decision-making of the distribution, migration, and scheduling of synchronization tasks.

[0015] Preferably, the specific operations for task execution include:

[0016] The system utilizes execution units, which are the actual execution units of synchronous tasks. These units include database collection and writing capabilities. Execution units are logical execution units, stateless components, and support horizontal scaling. A single execution unit contains a basic front-end agent and a common runtime environment. Agent plugins are distributed through the synchronization service based on the type of task, providing the ability to prune and hot update. Database data collected by the collection service within the execution unit is cached in the local exchange engine and then transmitted to the local exchange engine of the writing service through the central exchange engine, achieving stable cross-network data transmission. When tenant execution resources within the system are insufficient or idle, dynamic scaling of the execution unit is achieved by updating associated custom resources.

[0017] Preferably, the specific operations of the switching transmission include:

[0018] The switching and transmission engine is the communication foundation of the system and the central hub for exchanging data and events in the entire synchronization system. It provides the ability to route, transmit, and persistently cache messages and data between various components within the system. A single switching and transmission engine consists of multiple service nodes, which are interconnected and communicate with each other through a cluster protocol to build a message passing network. The switching and transmission engine includes a central switching and transmission engine and local switching and transmission engines. The switching and transmission engine mainly provides core capabilities based on NATS.

[0019] A cloud-native architecture cross-network database synchronization system, the system comprising a synchronization management module, a resource control module, a scheduling management module, an execution module, and a transmission module;

[0020] The synchronization management module provides a unified synchronization service interface and synchronization operation interface to the outside world;

[0021] The resource control module provides management and control of custom Kubernetes resources required for synchronization, including instance resource controller, tenant resource controller, and task resource controller, which respectively manage custom instance resources, tenant resources, and task resources;

[0022] The scheduling management module is responsible for generating and scheduling task execution plans, and maintaining the running status of relevant metadata.

[0023] The execution module receives the task execution plan issued by the scheduling management service, executes the corresponding logic, and provides database collection and writing capabilities.

[0024] The transmission module provides data transmission and caching capabilities, enabling data transmission in network isolation, weak network, and edge scenarios. It is divided into a local switching transmission engine and a central switching transmission engine.

[0025] Preferably, the resource control module utilizes a set of Kubernetes custom controllers to create and manage custom resources. These controllers monitor and respond to the creation, update, and deletion of custom resources, providing lifecycle management for related resources and enabling automatic scaling, creation, and upgrading of related service modules as needed. The synchronization resource controllers include instance resource controllers, tenant resource controllers, and task resource controllers. Instance resource controllers correspond to synchronization service instance resources, describing the specifications and related configurations of basic services, and are responsible for the creation, expansion, and status monitoring of the synchronization service entity. Tenant resource controllers correspond to tenant-class custom resources, describing the specifications and configurations of a type of tenant-isolated computing and transmission resources. Tenant resource controllers are responsible for the creation, expansion, and destruction of execution units and local switching transmission engines. Task resource controllers correspond to task-specific resources, describing the basic elements of a synchronization task, including the source of data, the flow of data, and the topological relationship information of data within the task. Through the task controllers, the creation, update, and deletion of synchronization tasks are implemented, and the status of tasks can be modified through custom attributes, allowing for the starting, pausing, and stopping of resource-related task entities.

[0026] Preferably, the scheduling management module, which is the core of database synchronization, supports multi-replica deployment and is responsible for managing the system's metadata and maintaining the system's operating status. The system metadata and operating status are classified into two categories: task information and tenant information, which are uniformly aggregated and transmitted through the exchange transmission engine. Tenant information specifically refers to the information of the execution unit, workload indicators, and their operating status, which is used to guide the scaling of tenant resources. Task information corresponds to the configuration information, execution results, and status of the subtasks associated with the synchronization task, supporting the decision-making of the distribution, migration, and scheduling of synchronization tasks.

[0027] Preferably, the execution module utilizes execution units, which are the actual execution units of synchronous tasks, including database acquisition and database writing capabilities. Execution units are logical execution units, stateless components, and support horizontal scaling. A single execution unit includes a basic front-end agent and a common runtime environment. Agent plugins are distributed through the synchronization service according to the type of task, providing the ability to prune and hot update. Database data acquired by the acquisition service in the execution unit is cached in the local exchange engine and then transmitted to the local exchange engine of the writing service through the central exchange engine, achieving stable cross-network data transmission. When tenant execution resources within the system are insufficient or idle, dynamic scaling of the execution unit is achieved by updating associated custom resources.

[0028] Preferably, the transmission module, the switching transmission engine, is the communication foundation of the system, the exchange center for data and events of the entire synchronization system, and provides the ability to route, transmit, and persistently cache messages and data between various components within the system. A single switching transmission engine consists of multiple service nodes, which are interconnected and communicate with each other through a cluster protocol to build a message passing network. The switching transmission engine includes a central switching transmission engine and local switching transmission engines, and the switching transmission engine mainly provides core capabilities based on NATS.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] The cloud-native architecture cross-network database synchronization method and system proposed in this invention combine the advantages of cloud-native technology, lightweight data transmission technology, and data acquisition technology to provide more flexible, scalable, lightweight, and easy-to-manage real-time synchronization capabilities for heterogeneous database synchronization. Attached Figure Description

[0031] Figure 1 This is a diagram of the overall architecture of the present invention;

[0032] Figure 2 This is a diagram of the switching transmission engine architecture of the present invention;

[0033] Figure 3 This is a diagram of the execution unit architecture of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the present invention clear and complete, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some, not all, embodiments of the present invention, and are merely illustrative of the embodiments of the present invention. They are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Example 1

[0036] Please see Figures 1 to 3 This invention provides a technical solution: a cross-network database synchronization method for cloud-native architecture, the method comprising the following steps:

[0037] It consists of five parts: synchronization management service, synchronization resource controller, scheduling management service, execution unit, and switching transmission engine.

[0038] The synchronization management service acts as the synchronization service layer, providing a unified synchronization service interface and operation interface. The synchronization resource controller manages the custom Kubernetes resources required for synchronization, including instance resource controllers, tenant resource controllers, and task resource controllers, which respectively manage custom instance resources, tenant resources, and task resources. The scheduling management service is responsible for generating and scheduling task execution plans, maintaining relevant metadata and system runtime status. The execution unit is the actual execution unit of the task, receiving the task execution plan from the scheduling management service, executing the corresponding logic, and providing database collection and writing capabilities. The switching and transmission engine provides data transmission and caching capabilities, enabling data transmission in network isolation, weak network, and edge scenarios. It is divided into a local switching and transmission engine and a central switching and transmission engine. The overall architecture is as follows: Figure 1 As shown.

[0039] Database synchronization mainly consists of two phases: the creation of resources associated with the synchronization task and the scheduling, distribution, and execution of the synchronization task. The specific process is as follows:

[0040] (1) Users input the metadata of the created synchronization task into the synchronization management service through the interface or API, including the source / destination database address, username, password; synchronized database table information; and basic information such as the attribute configuration of the synchronization task.

[0041] (2) After receiving user information, the synchronization management service calls the scheduling management service interface to determine whether there is an execution unit that meets the requirements. If there is an available execution unit, proceed to step 4; otherwise, proceed to step 3.

[0042] (3) Synchronization management service: Based on task configuration information (source and destination database information), call the tenant resource controller in the synchronization resource controller to create associated local switching transmission engine and execution unit.

[0043] (4) Synchronization Management Service: Based on the task configuration information (source, destination), it calls the task resource controller in the synchronization resource controller. The task resource controller then calls the relevant interfaces of the scheduling management service component to create a task.

[0044] (5) After receiving the created task, the scheduling management service generates a task execution plan, including the extraction end execution plan or the storage end execution plan, and sends each execution plan to the corresponding execution unit through the exchange transmission engine.

[0045] (6) The execution unit receives the task execution plan, constructs the extraction end or the storage end execution environment according to the specific type, and feeds back the execution results. It also collects or writes database data as needed.

[0046] 1 Synchronization Resource Controller

[0047] The Synchronization Resource Controller is a set of Kubernetes custom controllers responsible for creating and managing custom resources. The controller monitors and responds to creation, update, and deletion operations of custom resources, provides lifecycle management for related resources, and can automatically scale, create, and upgrade related service modules as needed.

[0048] The synchronization resource controller includes the instance resource controller, the tenant resource controller, and the task resource controller.

[0049] The instance resource controller corresponds to the synchronization service instance resources. It describes the specifications of the basic services and their related configurations, and is responsible for the creation, expansion, and status monitoring of the synchronization service main body (scheduling management service, central exchange transmission engine).

[0050] The tenant resource controller corresponds to the tenant class's custom resources. It mainly describes the specifications and configuration of a type of tenant-isolated computing and transmission resources. The tenant resource controller is responsible for the creation, expansion, and destruction of execution units and local switching transmission engines.

[0051] The task resource controller corresponds to the task-defined resource. It describes the basic elements of a synchronous task, mainly including the source of data in the task, the direction of data flow, and the topological relationship of data. Through the task controller, we can realize the creation, update, and deletion operations of synchronous tasks. At the same time, we can modify the status of the task through custom attributes, and start, pause, and stop the task entity associated with the resource.

[0052] 2. Dispatch Management Service

[0053] The scheduling and management service is the core of database synchronization, supporting multi-replica deployment. It is responsible for managing system metadata and maintaining the system's operational status. System metadata and operational status can be categorized into two types: task-related information and tenant-related information, which are uniformly aggregated and transmitted through the switching and transmission engine.

[0054] Tenant information specifically refers to the information of the execution unit, workload metrics, and its running status, which is used to guide the scaling of tenant resources.

[0055] Task-related information corresponds to the configuration information, execution results, and status of the associated subtasks of the synchronous task, supporting decisions on the distribution, migration, and scheduling of synchronous tasks.

[0056] 3 Switching Transmission Engine

[0057] The switching and transport engine is the communication foundation of the system, serving as the central hub for exchanging data and events within the entire synchronization system. It provides the capability for routing, transmitting, and persistently caching messages (control commands, monitoring information) and data (DDL events, DML events) between various components within the system. A single switching and transport engine (central or local) consists of multiple service nodes, which are interconnected and communicate with each other via a clustering protocol to build a message passing network. The switching and transport engine includes a central switching and transport engine and local switching and transport engines. The switching and transport engine primarily relies on NATS to provide its core capabilities. The network topology is as follows: Figure 2 As shown.

[0058] The central switching transmission engine primarily provides collaborative capabilities across cloud and regional networks. Multiple central switching transmission engines are connected via gateway protocols to enable message transmission between different central switching engines.

[0059] The local switching transmission engine provides basic message transmission capabilities, as well as local message persistence and caching capabilities, supporting secure and reliable data message transmission between worker computing nodes. Multiple local switching transmission engines can proactively connect to the central switching transmission engine through the edge extension protocol, enabling message routing, forwarding, and exchange transmission between local switching engines that are network isolated.

[0060] The switching transmission engine also supports data encryption and flow control, providing the system with a controllable and secure transmission channel. It decouples the relationships between components within the system, simplifies the network topology, provides a low-level abstraction of control and transmission channels, and enhances the overall flexibility, stability, and scalability of the system.

[0061] 4 execution units

[0062] An execution unit is the actual unit of execution for synchronous tasks, including database data collection and writing capabilities. Execution units are logical execution units, stateless components, and support horizontal scaling.

[0063] A single execution unit includes a basic front-end agent and a common runtime environment (common logic for synchronous sharing of heterogeneous databases). Agent plugins, based on task type, are distributed through a synchronization service, providing capabilities for trimming and hot updates. Database data collected by the collection service within the execution unit is cached in a local exchange engine and then transmitted through a central exchange engine to the local exchange engine of the write service, achieving stable cross-network data transmission. For example... Figure 3 As shown, the execution unit receives control commands through the front-end agent and performs related operations.

[0064] When tenants within the system lack sufficient or idle execution resources, dynamic scaling of execution units can be achieved by updating associated custom resources. Execution units are stateless, meaning tasks can be easily migrated between different computing units, and the synchronization service performs reasonable scheduling based on the availability and load of each execution unit.

[0065] Example 2

[0066] Based on Example 1, a cloud-native architecture cross-network database synchronization system is proposed. The system consists of a synchronization management module, a resource control module, a scheduling management module, an execution module, and a transmission module.

[0067] The synchronization management module provides a unified synchronization service interface and synchronization operation interface to the outside world;

[0068] The resource control module provides management and control of custom Kubernetes resources required for synchronization, including instance resource controllers, tenant resource controllers, and task resource controllers, which respectively manage custom instance resources, tenant resources, and task resources. Synchronization resource control utilizes a set of custom Kubernetes controllers responsible for creating and managing custom resources. These controllers monitor and respond to creation, update, and deletion operations of custom resources, provide lifecycle management of related resources, and can automatically scale, create, and upgrade related service modules as needed. The synchronization resource controllers include instance resource controllers, tenant resource controllers, and task resource controllers. The instance resource controller corresponds to the synchronization service instance resources and describes the basic... The basic service specification and related configuration are responsible for the creation, expansion, and status monitoring of the synchronization service entity; the tenant resource controller corresponds to the tenant class custom resource, describing the specifications and configuration of a type of tenant-isolated computing and transmission resources. The tenant resource controller is responsible for the creation, expansion, and destruction of execution units and local switching transmission engines; the task resource controller corresponds to the task custom resource, describing the basic elements of a synchronization task, including the source of data in the task, the direction of data flow, and the topological relationship information of data. Through the task controller, the creation, updating, and deletion operations of synchronization tasks can be realized. At the same time, the status of the task can be modified through custom attributes, and the task entity associated with the resource can be started, paused, or stopped.

[0069] The scheduling management module is responsible for generating and scheduling task execution plans and maintaining the operational status of relevant metadata. The scheduling management service is the core of database synchronization, supports multi-replica deployment, and is responsible for managing the system's metadata and maintaining the system's operational status. System metadata and operational status are classified into two categories: task-related information and tenant-related information, which are uniformly aggregated and transmitted through the exchange transmission engine. Tenant-related information specifically refers to the information of execution units, workload indicators, and their operational status, which is used to guide the scaling of tenant-related resources. Task-related information corresponds to the configuration information, execution results, and status of subtasks associated with synchronization tasks, supporting decisions on the distribution, migration, and scheduling of synchronization tasks.

[0070] The execution module receives task execution plans from the scheduling and management service, executes the corresponding logic, and provides database collection and writing capabilities. It utilizes execution units, which are the actual execution units for synchronous tasks, including database collection and writing capabilities. Execution units are logical execution units, stateless components, and support horizontal scaling. A single execution unit includes a basic front-end agent and a common runtime environment. Agent plugins are distributed through the synchronization service based on the task type, providing the ability to prune and hot-update. Database data collected by the collection service within the execution unit is cached in the local exchange engine and then transmitted through the central exchange engine to the local exchange engine of the writing service, achieving stable cross-network data transmission. When tenant execution resources within the system are insufficient or idle, dynamic scaling of the execution unit is achieved by updating associated custom resources.

[0071] The transmission module provides data transmission and caching capabilities, enabling data transmission in network isolation, weak network, and edge scenarios. It is divided into a local exchange transmission engine and a central exchange transmission engine. The exchange transmission engine is the communication foundation of the system, serving as the central hub for exchanging data and events throughout the entire synchronization system. It provides the ability to route, transmit, and persistently cache messages and data between various components within the system. A single exchange transmission engine consists of multiple service nodes, which are interconnected and communicate with each other through a cluster protocol to build a message passing network. The exchange transmission engine includes a central exchange transmission engine and a local exchange transmission engine. The exchange transmission engine primarily provides core capabilities based on NATS.

[0072] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for cross-network database synchronization of a cloud-native architecture, characterized in that: The method comprises the following steps: A synchronization management service provides a unified synchronization service interface and a synchronization operation interface to the outside; A synchronization resource control provides management and control of K8s custom resources required for synchronization, including instance resource controllers, tenant resource controllers, and task resource controllers, which manage custom instance resources, tenant resources, and task resources, respectively; A scheduling management service is responsible for task execution plan generation and scheduling, and maintains the running state of related metadata; Task execution receives the task execution plan issued by the scheduling management service, executes the corresponding logic, and provides database collection and writing capabilities; Exchange transmission provides data transmission and caching capabilities, and provides data transmission in network isolation, weak network, and edge scenarios, and is divided into local exchange transmission engines and central exchange transmission engines; The specific operations of the synchronization resource control include: The synchronization resource control utilizes a set of K8s custom controllers to create and manage custom resources. The controllers monitor and respond to the creation, update, and deletion of custom resources, provide lifecycle management of related resources, and can provide automatic scaling, creation, and upgrading of related service modules according to demand. The synchronization resource controller includes instance resource controllers, tenant resource controllers, and task resource controllers. Instance resource controllers correspond to synchronization service instance resources, describe the specifications and related configurations of basic services, and are responsible for the creation, scaling, and state monitoring of synchronization service subjects. Tenant resource controllers correspond to tenant custom resources, describe a class of tenant-isolated computing and transmission resource specifications and their configurations, and are responsible for the creation, scaling, and destruction of execution units and local exchange transmission engines. Task resource controllers correspond to task custom resources, describe the basic elements of a synchronization task, including the source of data in the task, the flow of data, and the topology relationship information of data. Through the task controller, the creation, update, and deletion operations of the synchronization task are implemented, and the status of the task is modified through custom attributes, such as starting, pausing, and stopping the resource-associated task entity. The specific operations of the scheduling management service include: The scheduling management service is the core of database synchronization, supports multiple replica deployments, manages system metadata, and maintains system running states. System metadata and running states are divided into two categories: task information and tenant information, which are unified by exchange transmission engines. Tenant information refers to execution unit information, workload indicators, and their running states, which are used to guide the scaling of tenant resources. Task information corresponds to the configuration information of synchronization task-associated subtasks, execution results, and states, which support the distribution, migration, and scheduling decisions of synchronization tasks. The specific operations of task execution include: The system utilizes execution units, which are the actual execution units of synchronous tasks. These units include database collection and writing capabilities. Execution units are logical execution units, stateless components, and support horizontal scaling. A single execution unit contains a basic front-end agent and a common runtime environment. Agent plugins are distributed through the synchronization service based on the type of task, providing the ability to prune and hot update. Database data collected by the collection service within the execution unit is cached in the local exchange engine and then transmitted to the local exchange engine of the writing service through the central exchange engine, achieving stable cross-network data transmission. When tenant execution resources within the system are insufficient or idle, dynamic scaling of the execution unit is achieved by updating associated custom resources.

2. The cloud-native architecture cross-network database synchronization method of claim 1, wherein: The specific operations of switching and transmission include: The switching and transmission engine is the communication foundation of the system and the central hub for exchanging data and events in the entire synchronization system. It provides the ability to route, transmit, and persistently cache messages and data between various components within the system. A single switching and transmission engine consists of multiple service nodes, which are interconnected and communicate with each other through a cluster protocol to build a message passing network. The switching and transmission engine includes a central switching and transmission engine and local switching and transmission engines. The switching and transmission engine mainly provides core capabilities based on NATS.

3. The cloud-native cross-network database synchronization system of the cloud-native cross-network database synchronization method according to any one of claims 1-2, characterized in that: The system consists of a synchronization management module, a resource control module, a scheduling management module, an execution module, and a transmission module. The synchronization management module provides a unified synchronization service interface and synchronization operation interface to the outside world; The resource control module provides management and control of custom Kubernetes resources required for synchronization, including instance resource controller, tenant resource controller, and task resource controller, which respectively manage custom instance resources, tenant resources, and task resources; The scheduling management module is responsible for generating and scheduling task execution plans, and maintaining the running status of relevant metadata. The execution module receives the task execution plan issued by the scheduling management service, executes the corresponding logic, and provides database collection and writing capabilities. The transmission module provides data transmission and caching capabilities, enabling data transmission in network isolation, weak network, and edge scenarios. It is divided into a local switching transmission engine and a central switching transmission engine.

4. The cross-network database synchronization system of a cloud-native architecture according to claim 3, characterized in that: The resource control module utilizes a set of Kubernetes custom controllers to create and manage custom resources. These controllers monitor and respond to the creation, update, and deletion of custom resources, providing lifecycle management for related resources and enabling automatic scaling, creation, and upgrades of related service modules as needed. The synchronization resource controllers include instance resource controllers, tenant resource controllers, and task resource controllers. Instance resource controllers correspond to synchronization service instance resources, describing the specifications and related configurations of basic services, and are responsible for the creation, scaling, and status monitoring of the synchronization service entity. Tenant resource controllers correspond to tenant-class custom resources, describing the specifications and configurations of a type of tenant-isolated computing and transmission resources, and are responsible for the creation, scaling, and destruction of execution units and local switching transmission engines. The task resource controller corresponds to the task-defined resource and describes the basic elements of a synchronous task, including the source of data in the task, the direction of data flow, and the topological relationship information of data. Through the task controller, the creation, updating, and deletion operations of synchronous tasks can be realized. At the same time, the status of the task can be modified through custom attributes, and the task entity associated with the resource can be started, paused, or stopped.

5. The cloud-native architecture cross-network database synchronization system according to claim 3, wherein: The scheduling management module, which is the core of database synchronization, supports multi-replica deployment and is responsible for managing the system's metadata and maintaining its operational status. System metadata and operational status are categorized into two types: task-related information and tenant-related information, which are uniformly aggregated and transmitted through the exchange transmission engine. Tenant-related information specifically refers to the information of execution units, workload indicators, and their operational status, used to guide the scaling of tenant-related resources. Task-related information corresponds to the configuration information, execution results, and status of subtasks associated with synchronization tasks, supporting decisions on the distribution, migration, and scheduling of synchronization tasks.

6. The cross-network database synchronization system of a cloud-native architecture according to claim 3, wherein: The execution module utilizes execution units, which are the actual execution units of synchronous tasks. These units include database acquisition and writing capabilities. Execution units are logical execution units, stateless components, and support horizontal scaling. A single execution unit contains a basic front-end agent and a common runtime environment. Agent plugins are distributed through the synchronization service based on the task type, providing the ability to prune and hot-update. Database data collected by the acquisition service in the execution unit is cached in the local exchange engine and then transmitted to the local exchange engine of the writing service through the central exchange engine, achieving stable cross-network data transmission. When tenant execution resources within the system are insufficient or idle, dynamic scaling of the execution unit is achieved by updating associated custom resources.

7. The cross-network database synchronization system of a cloud-native architecture according to claim 3, characterized in that: The transmission module, the switching transmission engine, is the communication foundation of the system and the exchange center for data and events in the entire synchronization system. It provides the ability to route, transmit, and persistently cache messages and data between various components within the system. A single switching transmission engine consists of multiple service nodes, which are interconnected and communicate with each other through a cluster protocol to build a message passing network. The switching transport engine includes a central switching transport engine and a local switching transport engine. The switching transport engine primarily provides core capabilities based on NATS.

Images