Database table structure synchronization method and device and electronic equipment

Abstract

The invention discloses a database table structure synchronization method and device and electronic equipment, and relates to the field of distributed technologies or other related technologies, and the method comprises the steps that a subscription relation chain is established; under the condition that the table structure of the source database is updated, a change event is generated, the change event is used for representing statement information of a first statement, and the first statement is used for updating the table structure of the source database; pushing the change event to a target database based on the subscription relationship chain; and updating the table structure of the target database based on the change event. According to the method and the device, the technical problem that the effect of synchronizing the distributed database table structure in the multi-level environment based on the prior art is poor is solved.

Description

Technical Field

[0001] This application relates to the field of distributed technology or other related technical fields, and more specifically, to a method, apparatus, and electronic device for synchronizing database table structures. Background Technology

[0002] With the promotion of distributed service technology, enterprises providing large-scale distributed database services are facing increasing development pressure. In order to quickly realize user business needs, enterprises generally adopt a project iteration approach during R&D, that is, to achieve project R&D support by rotating multiple R&D environments.

[0003] Because large-scale distributed database services involve numerous applications, each with multiple development environments requiring parallel operation and maintenance, the operational pressure on these environments increases accordingly. During the development process, when changes to the distributed database table structure are involved, traditional methods such as manual comparison and script synchronization, scheduled task detection, and one-way replication using database middleware gradually reveal the following shortcomings:

[0004] (1) Inaccurate synchronization: Manual operation is prone to oversight and errors, leading to deviations in table structures between multi-level environments, affecting the integrity and consistency of functional verification. In addition, solutions based on specific database middleware often fail when crossing heterogeneous database environments, causing synchronization obstacles between platforms.

[0005] (2) Synchronous but not real-time: Scheduled tasks or manually executed scripts cannot respond to changes in the table structure in a timely manner. The delay may be as long as several hours or even a day, which greatly hinders the efficiency of agile development and makes it difficult for the team to respond quickly to market changes and customer needs.

[0006] There is currently no effective solution to the above problems. Summary of the Invention

[0007] This application provides a method, apparatus, and electronic device for synchronizing database table structures, to at least solve the technical problem of poor performance in synchronizing distributed database table structures in multi-level environments based on existing technologies.

[0008] According to one aspect of this application, a method for synchronizing database table structures is provided, comprising: establishing a subscription relationship chain, wherein the subscription relationship chain is used to represent the subscription relationship between databases in a development environment and a testing environment; generating a change event when the table structure of the source database is updated, wherein the change event is used to represent statement information of a first statement used to update the table structure of the source database, the source database being a database subscribed to in the subscription relationship chain; pushing the change event to a target database based on the subscription relationship chain, wherein the target database is a database in the subscription relationship chain that subscribes to the source database; and updating the table structure of the target database based on the change event.

[0009] Optionally, in the case of an update to the table structure of the source database, a change event is generated, including: determining N event elements based on the statement information of the first statement, wherein the N event elements are at least one of the following: a first element, used to characterize the operation type of the database operation that updates the table structure of the source database; a second element, used to characterize the table name of the database table in the source database whose table structure is being updated; a third element, used to characterize the type of the source database; and encapsulating the N event elements and the source code corresponding to the first statement based on a preset protocol to obtain a change event, wherein the change event corresponds to an event identifier and a version identifier, and the version identifier of the change event is used to characterize the iteration version of the code in the environment to which the source database belongs.

[0010] Optionally, based on the subscription relationship chain, the change event is pushed to the target database, including: based on the subscription relationship chain, determining the partition allocation strategy corresponding to the preset platform; based on the partition allocation strategy, determining the target partition corresponding to the change event, wherein the target partition is the partition in the preset platform used to store the change event; after writing the change event to the target partition, pushing the change event in the target partition to the target database.

[0011] Optionally, based on the change event, the table structure of the target database is updated, including: obtaining a metadata mapping table, wherein the metadata mapping table includes at least the mapping relationships between fields corresponding to different types of databases; parsing the change event to obtain a parsing result, wherein the parsing result includes at least the code function corresponding to the first statement and the source code corresponding to the first statement; filtering the metadata mapping table based on the type of the target database to obtain the mapping rules for the source code corresponding to the first statement; updating the fields in the source code corresponding to the first statement based on the mapping rules to obtain the second statement; and updating the table structure of the target database based on the second statement.

[0012] Optionally, after filtering in the metadata mapping table based on the type of the target database, the database table structure synchronization method further includes: if there is no mapping rule for the source code corresponding to the first statement in the metadata mapping table, obtaining a schema transformation matrix, wherein the schema transformation matrix includes at least equivalent implementation schemes of the code functions implemented by database operation statements in different types of databases; filtering in the schema transformation matrix based on the type of the target database to obtain equivalent implementation schemes of the code functions corresponding to the first statement; and generating a second statement based on the equivalent implementation schemes of the code functions corresponding to the first statement.

[0013] Optionally, after updating the table structure of the target database based on the change event, the database table structure synchronization method further includes: executing a preset command in the target database table to obtain the target table structure, wherein the target table structure is the table structure of the target database obtained after executing the second statement; obtaining the similarity between the source table structure and the target table structure, wherein the source table structure is the table structure of the source database obtained after executing the first statement; determining that the database table structure synchronization between the target database and the source database is successful if the similarity is greater than or equal to a preset similarity; determining that the database table structure synchronization between the target database and the source database fails if the similarity is less than the preset similarity; and executing the rollback operation corresponding to the second statement and generating a warning message if the database table structure synchronization between the target database and the source database fails.

[0014] Optionally, after writing the change event to the target partition, the database table structure synchronization method further includes: if the target database is a database in the test environment, generating filtering rules based on the test requirements of the test environment, wherein each filtering rule corresponds to a rule identifier and a version identifier, and the version identifier of the filtering rule is used to characterize the iteration version of the code in the test environment; if the version identifier corresponding to the change event is less than the version identifier of the filtering rule, the change event in the target partition is intercepted; if the version identifier corresponding to the change event is greater than or equal to the version identifier of the filtering rule, the change event in the target partition is pushed to the target database.

[0015] According to another aspect of this application, a database table structure synchronization device is also provided, comprising: an establishment unit for establishing a subscription relationship chain, wherein the subscription relationship chain represents the subscription relationship between databases in a research and development environment and a testing environment; an event generation unit for generating a change event when the table structure of the source database is updated, wherein the change event represents statement information of a first statement used to update the table structure of the source database, the source database being a subscribed database in the subscription relationship chain; an event push unit for pushing the change event to a target database based on the subscription relationship chain, wherein the target database is a database in the subscription relationship chain that subscribes to the source database; and a table structure update unit for updating the table structure of the target database based on the change event.

[0016] According to another aspect of this application, a computer program product is also provided, which stores a computer program, wherein a synchronization method for controlling the computer program product to execute any of the above-mentioned database table structures is provided during the execution of the computer program.

[0017] According to another aspect of this application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a synchronization method for a database table structure as described above.

[0018] In this application, a subscription relationship chain is first established, which represents the subscription relationship between databases in the R&D environment and the test environment. Then, when the table structure of the source database is updated, a change event is generated, which represents the statement information of a first statement used to update the table structure of the source database, which is a database subscribed to in the subscription relationship chain. Then, based on the subscription relationship chain, the change event is pushed to the target database, which is a database in the subscription relationship chain that subscribes to the source database. Subsequently, based on the change event, the table structure of the target database is updated.

[0019] As can be seen from the above, this application achieves the goal of accurately and in real-time synchronizing the distributed database table structure in a multi-level R&D environment by constructing a subscription relationship chain and using table update events to drive the synchronous update of table structure data in different environments. This application avoids the introduction of human negligence and errors, as well as the inherent delay of scheduled tasks, thereby achieving the technical effect of improving data consistency and synchronization response speed between multi-level R&D environments, and thus solving the technical problem of poor effect of synchronizing the distributed database table structure in a multi-level environment based on existing technologies. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a synchronization method for database table structures according to an embodiment of this application.

[0022] Figure 2 This is a flowchart of an optional database table structure synchronization method according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of an optional subscription relationship chain according to an embodiment of this application;

[0024] Figure 4 This is a flowchart of an optional automatic synchronization method for a database table structure according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of a synchronization device for an optional database table structure according to an embodiment of this application;

[0026] Figure 6 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] It should also be noted that all information and data involved in this application (including but not limited to information used for display and analysis) and data (including but not limited to table structure data and executed database operation statements in the source / target databases) are information and data authorized by the user or fully authorized by all parties. For example, if there is an interface between this system and the relevant user or organization, before obtaining the relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information only after receiving consent from the aforementioned user or organization.

[0030] Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of relevant information and data involved in this application all comply with the relevant laws, regulations, and standards of the relevant regions, and necessary confidentiality measures have been taken. This application does not violate public order and good morals. In addition, this application provides a corresponding operation entry point for users to choose to agree to or refuse authorization. If the user chooses to refuse authorization, the corresponding expert decision-making process will be initiated.

[0031] In one optional embodiment, the following methods are provided for synchronizing database table structures in a multi-level development environment, including:

[0032] (1) Manual comparison and script synchronization: The DDL (Data Definition Language) statements used to update the distributed database table structure are manually exported from the source database by the operation and maintenance personnel, and the DDL statements are executed in batches in the target database by the script.

[0033] (2) Scheduled task scanning: The database table structure update records of the version control system are detected by scheduled tasks.

[0034] (3) Database middleware synchronization: Relying on the one-way replication tool provided by the database vendor to realize the synchronization of table structure.

[0035] However, the technical defects of the technical solutions provided by the above embodiments are as follows:

[0036] (1) High risk of manual operation: Relying on manual execution of changes is prone to omission of key change items or incorrect operation sequence, which directly leads to inconsistent configuration of the operating environment.

[0037] (2) Insufficient timeliness: The mechanism based on timed scanning has inherent delays, cannot respond to changes in real time, and is difficult to meet the requirements of agile development for rapid iteration and deployment.

[0038] (3) Strong technical coupling: Middleware solutions are often strongly coupled to specific database platforms / technology stacks, lack platform independence, and are difficult to achieve compatible deployment across heterogeneous environments.

[0039] (4) Lack of cascading control mechanism: It cannot support automated, sequential, and synchronous processes between multi-level environments.

[0040] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0041] Distributed database table structure: Metadata definitions such as fields, indexes, and constraints of database tables distributed across multiple physical nodes.

[0042] Subscription rules: A configuration strategy that specifies which source environment's table structure change events the target environment should listen to.

[0043] Relay synchronization: A chain-like synchronization mechanism in which table structure changes are passed sequentially from the initial environment to downstream multi-level environments.

[0044] Test sandbox: A test environment that is isolated for safely executing code, experimenting with new features, or verifying system behavior without affecting the production environment or other critical systems. Its core objectives are isolation and reproducibility.

[0045] The present invention will now be described in detail with reference to various embodiments.

[0046] Example 1

[0047] According to an embodiment of this application, a method for synchronizing a database table structure is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0048] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1This is a hardware structure block diagram of a computer terminal (or mobile device) according to an optional embodiment of this application for implementing a method for synchronizing database table structures. For example... Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0049] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0050] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the database table structure synchronization method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned database table structure synchronization method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0052] The display can be configured as a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0053] Under the aforementioned operating environment, this application provides a database table structure synchronization system (hereinafter referred to as the synchronization system) for executing the database table structure synchronization method of this application. Figure 2 This is a flowchart of an optional database table structure synchronization method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0054] Step S201: Establish a subscription relationship chain, wherein the subscription relationship chain is used to represent the subscription relationship between the databases in the R&D environment and the test environment.

[0055] Alternatively, the R&D environment and the testing environment refer to the environment settings for different stages of software development.

[0056] Optionally, the subscription relationship between databases specifies the direction and logic of information transfer between the source database and the target database.

[0057] Optionally, Figure 3 This is a schematic diagram of an optional subscription relationship chain according to an embodiment of this application, such as... Figure 3 As shown, if the table structure of iteration environment A changes, iteration environment B, which subscribes to this environment, will change the table structure by consuming messages from the preset platform; iteration environment C and the sandbox environment, which subscribe to iteration environment B, will change the table structure by consuming messages from the preset platform.

[0058] Optionally, the dynamic update process of subscription relationships in the subscription relationship chain is as follows:

[0059] (1) Configuration change trigger: Write new subscription rules to the metadata Topic (a type of message that is transmitted in a preset platform) via API (Application Programming Interface).

[0060] (2) Consumer group monitoring: All cascaded nodes subscribe to this Topic and obtain new rules in real time through the consumer group.

[0061] (3) Hot loading mechanism: After parsing the new rules, the consumer instance of the preset platform is dynamically rebuilt without restarting the service.

[0062] Optionally, when the environment changes, the automated processing of subscription relationships is as follows:

[0063] (1) Dynamic node discovery:

[0064] The Consumer Group, which relies on the preset platform, automatically detects the addition / exit of new nodes.

[0065] By combining ZooKeeper (a distributed coordination service) or the Kubernetes API (an interface for managing all resources and services on a cluster), node state changes can be monitored, triggering updates to metadata topics.

[0066] (2) Elastic stretching:

[0067] When scaling horizontally, new nodes automatically subscribe to the required data streams by reading metadata topics.

[0068] When scaling down, partition ownership is automatically transferred through Consumer Group's Rebalance mechanism (used to dynamically adjust workloads across multiple nodes / partitions within the cluster).

[0069] Optionally, the conflict resolution process for subscription relationships is as follows:

[0070] (1) Optimistic locking control: Metadata updates adopt the CAS (Compare And Swap, atomic operation in lock-free algorithms) mechanism, and avoid concurrent modification conflicts through the version number (version field).

[0071] (2) Audit Log: All changes are recorded in the preset Topic, supporting rollback and source tracing.

[0072] Optionally, by constructing a subscription relationship chain, the synchronization system enables changes to the database table structure to be synchronized sequentially from the source environment to the databases in subsequent iterations of the development environment along the predetermined path indicated by the subscription relationship chain. This effectively eliminates the uncertainty and synchronization delay caused by human intervention, achieving automated propagation and intelligent processing of distributed database table structure data changes, and improving the accuracy and real-time performance of table structure synchronization. Simultaneously, the cascading subscription design enhances the robustness and flexibility of the synchronization system, thus better supporting multi-level environment management and complex testing requirements.

[0073] Step S202: In the case of an update to the table structure of the source database, a change event is generated. The change event is used to represent the statement information of the first statement, which is used to update the table structure of the source database. The source database is the database that is subscribed to in the subscription relationship chain.

[0074] Optionally, the source database refers to the starting point in the subscription relationship chain, that is, the database environment in which the table structure change first occurs.

[0075] Optionally, table structure update refers to changes in the definition of a database table, including but not limited to adding, deleting, or modifying database table fields or attributes.

[0076] Optionally, a change event is an encapsulated expression of the definition update action of a database table. The change event contains complete database table structure update instructions (i.e., the first statement) information, so as to be transmitted in the subscription relationship chain corresponding to the multi-level development environment.

[0077] Optionally, the synchronization system generates change events, enabling each table structure update to be immediately converted into a transmittable information unit. Subsequently, the synchronization system uses change events as the medium for transmitting database table structure update information, reducing the complexity of table update information propagation and ensuring the integrity and consistency of change instructions during transmission.

[0078] Step S203: Based on the subscription relationship chain, push the change event to the target database, where the target database is the database in the subscription relationship chain that subscribes to the source database.

[0079] Optionally, the target database refers to the next node in the subscription relationship chain that subscribes to the source database, i.e., the database environment that needs to receive table structure and table update information.

[0080] Optionally, the change event push mechanism fully leverages the advantages of the subscription relationship chain to achieve precise targeting and efficient transmission of change information. This process not only reduces unnecessary data transmission and saves network resources, but also ensures that table structure and change information can be transmitted in a predetermined order and hierarchy, avoiding confusion and conflicts during the synchronization process. Through this method, the synchronization system can quickly and accurately transmit change events from the source database to the target database, ensuring that databases in all relevant environments can respond to and adapt to changes in a timely manner, thereby supporting the needs of agile development.

[0081] Step S204: Update the table structure of the target database based on the change event.

[0082] Optionally, the synchronization system simplifies the steps of applying changes by directly applying change events to the target database, improves the efficiency of synchronizing database table structure and table change information, and ensures the accurate implementation of table structure changes in the target environment. This mechanism reduces the manual workload of developers and operations personnel, avoids errors and delays caused by manual operations, and realizes the automation and standardization of table structure synchronization.

[0083] In summary, this application achieves accurate and real-time synchronization of distributed database table structures in multi-level R&D environments by constructing a subscription relationship chain and using table update events to drive the synchronous update of table structure data in different environments. This application avoids the introduction of human error and the inherent delay of scheduled tasks, thereby improving the technical effect of data consistency and synchronization response speed between multi-level R&D environments, and thus solving the technical problem of poor synchronization effect of distributed database table structures in multi-level environments based on existing technologies.

[0084] In one optional embodiment, in order to generate a change event, the synchronization system first determines N event elements based on the statement information of the first statement. Then, the synchronization system encapsulates the N event elements and the source code corresponding to the first statement based on a preset protocol to obtain a change event. The change event corresponds to an event identifier and a version identifier. The version identifier of the change event is used to characterize the iterative version of the code in the environment to which the source database belongs.

[0085] Optionally, the first statement refers to the table structure modification statement actually executed in the source database, i.e., the DDL statement.

[0086] Optionally, the N event elements are at least one of the following:

[0087] The first element is the type of database operation used to represent the database operation that updates the table structure of the source database;

[0088] The second element is the name of the database table whose structure is being updated in the source database.

[0089] The third element is used to characterize the type of the source database.

[0090] Optionally, examples of change events encapsulated through a preset protocol are as follows:

[0091] {

[0092] "operation": "ALTER_TABLE",

[0093] "object": "orders",

[0094] "platform": "XXX", / / Indicate the source platform type

[0095] "command": "ADD COLUMN status VARCHAR(20)"

[0096] }

[0097] Optionally, the first element refers to the key information in the change event used to describe the type of database operation, such as "ALTER", "CREATE", "DROP", etc.

[0098] Optionally, explicit labeling of operation types enables change events to maintain their semantic integrity and consistency during transmission. At the same time, explicit labeling of operation types also allows the synchronization system to make smarter decisions based on operation types, such as distinguishing between DDL and DML (Data Manipulation Language) changes, or prioritizing specific types of operations.

[0099] Optionally, the second element refers to the identifier element corresponding to the name of the database table affected by the change in the change event.

[0100] Optionally, by explicitly specifying the table names of the database tables whose table structures are being updated, the synchronization system can ensure that the target database only updates tables related to the change events, avoiding unnecessary resource consumption and potential errors, while improving the overall system's resource utilization efficiency and synchronization accuracy.

[0101] Optionally, the third element refers to the database platform that identifies the origin of the change event.

[0102] Optionally, the annotation of database type provides a reference for the cross-system propagation of change events. The synchronization system can perform appropriate syntax conversion or function substitution adaptation of the first statement based on the platform characteristics of the target database, ensuring the success rate of synchronizing table structure change information between heterogeneous database environments.

[0103] Optionally, the synchronization system determines N event elements based on the statement information of the first statement, thereby decomposing complex database operations into manageable and transmittable elements. On the one hand, this can reduce the complexity of constructing change events and improve the efficiency of the system in handling changes; on the other hand, clear division of event elements helps to achieve accurate synchronization in different database environments and reduce synchronization failures or inaccuracies caused by environmental differences.

[0104] Optionally, the synchronization system encapsulates N event elements and the source code corresponding to the first statement based on a preset protocol to obtain a change event including an event identifier and a version identifier. The encapsulation of the change event not only ensures the integrity of the database table structure update information, but also achieves data standardization through the preset protocol, thereby enhancing the traceability and manageability of the change event. This enables the generated change event to support difference comparison and rollback operations between different code iteration versions, improving the flexibility and robustness of change synchronization.

[0105] In summary, by refining the construction of change events, the synchronization system enhances the ability to synchronize distributed database table structures in multi-level R&D and testing environments, thereby overcoming the limitations of traditional technologies and achieving efficient and accurate propagation of database table structure change information.

[0106] In one optional embodiment, the synchronization system first determines the partition allocation strategy corresponding to the preset platform based on the subscription relationship chain; then, the synchronization system determines the target partition corresponding to the change event based on the partition allocation strategy, wherein the target partition is the partition in the preset platform used to store the change event; then, after writing the change event to the target partition, the synchronization system pushes the change event in the target partition to the target database.

[0107] Optionally, the partition allocation strategy refers to the strategy used to organize and manage change event data in the preset platform. The partition allocation strategy determines which partitions the change events will be written to and how load balancing will be performed among these partitions.

[0108] Optionally, when rebalancing consumer groups, if the partition allocation strategy needs to be updated, the synchronization system should retain the original partition allocation strategy as much as possible, thereby reducing the consumption pause time caused by changes in subscription relationships.

[0109] Optionally, when constructing a targeted subscription relationship in the subscription relationship chain, the synchronization system supports targeted subscription of table structure change messages, thereby avoiding full broadcast of table structure change messages.

[0110] Optionally, the synchronization system determines a partition allocation strategy, which enables the reasonable allocation of multiple change events. This allows change events to be stored quickly and evenly in the preset platform, avoiding the formation of hotspot partitions and improving the overall data throughput of the preset platform. At the same time, the customization of the strategy can better serve the subscription relationship chain. For example, the synchronization system can dynamically adjust the allocation strategy according to the type of the source database, the priority of the change events, or the load of the target database, thereby enabling more refined resource management and more flexible event processing.

[0111] Optionally, the target partition refers to a partition in the default platform used to store specific change events.

[0112] Optionally, by writing change events to the corresponding target partition, the synchronization system can leverage the parallel processing capabilities of the pre-defined platform to accelerate the processing speed of change events, while ensuring the orderly storage and consumption of change events. This approach not only improves the writing efficiency of change events but also facilitates subsequent accurate synchronization and event backtracking, enhancing the maintainability of the system and the consistency of the data.

[0113] Optionally, after writing the change event to the target partition, the synchronization system pushes the change event in the target partition to the target database through a preset platform, thereby realizing real-time and targeted synchronization of change events from the source database to the target database, shortening the time interval for table structure updates between multi-level development environments, and improving the real-time performance and accuracy of table structure change information synchronization.

[0114] In summary, the synchronization system improves the performance and reliability of distributed database table structure synchronization by optimizing the management and transmission process of change events within the preset platform.

[0115] Optionally, the default platform achieves high reliability through sequential writing, multi-replica synchronization, and ISR (In-Sync Replicas) election; the default platform ensures high throughput by combining zero-copy and page caching; in failure scenarios, the default platform relies on replica resynchronization and strict consistency configuration to ensure no data loss, thus making it suitable for scenarios such as financial-grade near real-time database table structure synchronization.

[0116] Optionally, when deploying the default platform, latency, durability, and number of partitions need to be weighed according to business requirements.

[0117] Optionally, the default platform uses sequential writing and log structure storage when storing table update messages. Each Topic partition in the default platform is split into multiple fixed-size Segment (the basic storage unit that constitutes each topic partition) files (1GB by default), and new messages are only appended to the end of the active Segment.

[0118] Optionally, the default platform uses sequential disk writing to achieve high-performance change message writing (3 orders of magnitude faster than random writing). Each segment file is accompanied by an index file, which is used to accelerate the location of change messages through memory mapping, thereby avoiding full scan.

[0119] Optionally, the default platform adopts a zero-copy transmission method, that is, change message data is directly transferred from the disk file to the network card via DMA (Direct Memory Access), bypassing user space and reducing CPU overhead and context switching.

[0120] Optionally, the default platform adopts a page cache optimization mechanism to avoid GC (Garbage Collection) overhead, and determines the frequency of data synchronization to disk through parameter control.

[0121] Optionally, the ISR election mechanism includes the following:

[0122] (1) ISR mechanism: Each partition has 1 Leader (master node) and N-1 Followers (slave nodes). The Leader is responsible for reading and writing events, and the Followers are responsible for asynchronously / synchronously pulling event data. Furthermore, the Followers must catch up with the Leader within a preset time period, otherwise they will be removed from the ISR list.

[0123] (2) ACK (Acknowledgement, used to control whether the producer's acknowledgment message is successfully received and stored by the consumer cluster) acknowledgment strategy:

[0124] The producer configuration is as follows:

[0125] acks=0: No confirmation required, data may be lost.

[0126] acks=1 (default): Leader writes are acknowledged immediately, but may be lost if the Leader fails.

[0127] acks=all: Acknowledgements are only made after all ISR replicas have been written, ensuring strong consistency.

[0128] (3) Leader election: Monitor the leader status. If the leader fails, elect a new leader from the ISR (preferably the latest copy) to avoid data inconsistency issues.

[0129] Optionally, the default platform fault recovery process is as follows:

[0130] (1) Broker (server node of the default platform) crash recovery:

[0131] Replica synchronization: The restored Broker pulls the missing data from the new Leader, catches up, and rejoins the ISR.

[0132] Data integrity check: Integrity verification is performed based on the cyclic checksum of each message.

[0133] (2) Disk failure handling:

[0134] Automatic isolation: If the Broker detects a disk failure (such as read / write timeout), it will automatically switch to another healthy disk.

[0135] Replica migration: If the failure is unrecoverable, the administrator can manually migrate the replica to another broker.

[0136] (3) Data loss protection:

[0137] Set a minimum number of synchronized replicas to ensure that writes are still possible even if one replica fails; prevent non-ISR replicas from becoming Leaders to prevent data loss.

[0138] Optionally, each partition in the preset platform can independently handle read and write requests, thereby horizontally scaling to improve throughput; the producers in the preset platform reduce network overhead by accumulating messages and sending them in batches; the partitions in the preset platform are assigned to different corresponding consumers, realizing parallel consumption and thus improving the concurrency performance of the preset platform.

[0139] In one optional embodiment, the synchronization system first obtains a metadata mapping table, which includes at least the mapping relationships between fields corresponding to different types of databases. Then, the synchronization system parses the change events to obtain a parsing result, which includes at least the code function corresponding to the first statement and the source code corresponding to the first statement. Then, based on the type of the target database, the synchronization system filters the metadata mapping table to obtain the mapping rules for the source code corresponding to the first statement. Subsequently, based on the mapping rules, the synchronization system updates the fields in the source code corresponding to the first statement to obtain the second statement. After that, based on the second statement, the synchronization system updates the table structure of the target database.

[0140] Optionally, the metadata mapping table records in detail the rules for field mapping between different database types, including but not limited to field names, data types, and field conversion logic.

[0141] For example, the metadata mapping table is shown in List 1 below:

[0142] Table 1

[0143]

[0144] Optionally, by obtaining data from the metadata mapping table, the synchronization system can resolve syntax inconsistencies and functional differences during cross-database type synchronization. This ensures that the meaning of database table fields and their corresponding functions are accurately preserved during the transmission of change events from one database to another. This mechanism enhances the compatibility and flexibility of the synchronization system, enabling the same change management process to be applied to multiple database platforms. It reduces synchronization obstacles caused by environmental differences, accelerates the iteration speed of application development, and optimizes the development process of software application code.

[0145] Optionally, the synchronization system parses change events to obtain at least the code functionality and source code corresponding to the first statement. This parsing process, by obtaining the complete source code corresponding to the first statement, not only facilitates intelligent conversion of the source code in subsequent steps but also enables auditing or backtracking of change events, enhancing the transparency and security of change management. Furthermore, the standardized format of the parsing results facilitates integration with metadata mapping tables, improving the automation level of the entire synchronization process.

[0146] Optionally, the system can filter the metadata mapping table according to the type of the target database to obtain the mapping rules of the source code corresponding to the first statement. This ensures the syntactic correctness and functional equivalence of the first statement in the target database environment and avoids conversion errors caused by differences in database platforms. Through intelligent filtering of the metadata mapping table, the system can automatically identify and adapt to the characteristics of the target database without manual intervention, thereby improving the efficiency and accuracy of synchronization and reducing maintenance costs.

[0147] Optionally, the synchronization system updates the fields in the source code corresponding to the first statement based on mapping rules to obtain the second statement. That is, the synchronization system uses predefined mapping rules to convert the fields in the first statement into equivalent fields in the target database environment, ensuring the correct execution of the change instructions. The converted second statement can be executed directly on the target database, realizing seamless synchronization of table structures in different environments, enhancing the system's cross-platform capabilities, and reducing the workload of developers and operations personnel in manually adjusting each database environment.

[0148] Optionally, the synchronization system updates the table structure of the target database based on the second statement, ensuring that every change to the table structure in the source database is accurately implemented in the target database. This avoids the uncertainty and risks of manual operation. This mechanism supports the needs of agile development and continuous integration, enabling operations personnel to quickly and safely promote table structure changes in multi-level environments, improving the speed and stability of application software code iteration. At the same time, this mechanism simplifies the change testing and verification process. The target database can be directly updated using the second statement without additional adjustments or verification, further accelerating the software code development cycle.

[0149] In summary, the synchronization system, by acquiring the metadata mapping table, parsing change events, intelligently filtering and applying mapping rules, and generating and executing a second statement, constructs a comprehensive solution for real-time synchronization of table structure changes between heterogeneous database environments. The above steps not only solve the problems of inaccurate and non-real-time synchronization in existing technologies, but also improve software development efficiency and reduce the error rate and latency of synchronizing database table structures in heterogeneous environments.

[0150] In one optional embodiment, after filtering the metadata mapping table based on the type of the target database, if there is no mapping rule for the source code corresponding to the first statement in the metadata mapping table, the synchronization system first obtains a schema transformation matrix, wherein the schema transformation matrix includes at least equivalent implementation schemes of the code functions implemented by database operation statements in different types of databases. Then, the synchronization system filters the schema transformation matrix based on the type of the target database to obtain equivalent implementation schemes of the code functions corresponding to the first statement. Finally, the synchronization system generates the second statement based on the equivalent implementation schemes of the code functions corresponding to the first statement.

[0151] For example, the mode transition matrix is ​​shown in Table 2 below:

[0152] Table 2

[0153]

[0154] Optionally, if there is no mapping rule for the source code corresponding to the first statement in the metadata mapping table, the introduction of the schema transformation matrix fills the field transformation gap in the metadata mapping table. Especially when facing functions or operations unique to different types of databases, the equivalent implementation schemes stored in the schema transformation matrix can ensure that the synchronization system can still find alternative or equivalent implementation methods even without direct field mapping rules, thereby maintaining the integrity and compatibility of table structure change synchronization.

[0155] Optionally, the synchronization system filters the schema transformation matrix based on the target database type, and can intelligently identify and select alternatives with code functions equivalent to the first statement. This not only solves the compatibility problem of database operation statements in heterogeneous environments, but also ensures that the core functions of the change instructions can be preserved and reproduced in the target database. This mechanism improves the accuracy and effectiveness of database table structure change synchronization and avoids functional loss or erroneous execution caused by differences in database platforms.

[0156] In summary, by filtering the hierarchical alternatives in the schema transformation matrix, the synchronization system solves the technical challenge of synchronizing table structure changes in heterogeneous database environments. It enables accurate and efficient synchronization of table structure changes even in the absence of direct mapping rules, thereby enhancing the system's functional adaptability and flexibility.

[0157] Optionally, the pseudocode for the fault tolerance handling process during the conversion from the first statement to the second statement in the synchronization system is set as follows:

[0158] A[Conversion failed]-->B{Error type};

[0159] B -- Syntax error --> C [Recorded to the corresponding Topic];

[0160] B -- Functionality missing --> D [Triggering degradation module];

[0161] B -- Data Conflict --> E [Invoke Repair Script];

[0162] C-->F [Manual Intervention Interface];

[0163] D-->G [Generate application layer compensation code];

[0164] E-->H [Automatically retry 3 times].

[0165] In one optional embodiment, after updating the table structure of the target database based on a change event, the synchronization system first executes a preset command in the target database table to obtain the target table structure, wherein the target table structure is the table structure of the target database obtained after executing the second statement; then, the synchronization system obtains the similarity between the source table structure and the target table structure, wherein the source table structure is the table structure of the source database obtained after executing the first statement; subsequently, if the similarity is greater than or equal to a preset similarity, the synchronization system determines that the database table structure synchronization between the target database and the source database is successful; if the similarity is less than the preset similarity, the synchronization system determines that the database table structure synchronization between the target database and the source database has failed; in the case of database table structure synchronization failure between the target database and the source database, the synchronization system executes the rollback operation corresponding to the second statement and generates a warning message.

[0166] Optionally, a preset command refers to a data extraction command that can fully reflect the database table structure. By executing the preset command on the target database, the synchronization system can verify whether the converted DDL semantics can achieve functional equivalence.

[0167] Optionally, the synchronization system obtains the target database table structure by executing preset commands, enabling the synchronization system to accurately evaluate the target table structure and thus detect whether the change instructions in the target database have been correctly executed. By directly obtaining the latest state of the table structure from the database, the synchronization system provides a non-intrusive means of verifying the database table structure, avoiding interference with the operation of the target database.

[0168] Optionally, by obtaining the similarity between the source table structure and the target table structure, the synchronization system can measure the degree of structural matching between the source table structure and the target table structure, and thus intuitively detect the degree of conformity between the target database table structure and the source database table structure. This helps the synchronization system determine whether the change information of the database table structure has been accurately copied from the source environment to the target environment.

[0169] Optionally, the preset similarity is a pre-set threshold used to determine whether the table structure synchronization between the source database and the target database meets the expected standard. By setting the preset similarity, the synchronization system can automatically determine the success status of the synchronization operation, reducing the need for manual review and also reducing errors caused by subjective judgment.

[0170] Optionally, if the similarity is less than the preset similarity, the synchronization system determines that the database table structure synchronization between the target database and the source database has failed. The synchronization system converts each DDL statement into a sub-transaction and triggers a compensation operation when the synchronization fails.

[0171] Optionally, a rollback operation is used when the table structure of the source database fails to be accurately synchronized to the state of the target database, and the synchronization system performs operations to undo or repair the changes to the database table structure caused by the second statement.

[0172] Optionally, by performing a rollback operation, the previous state of the target database can be quickly restored as soon as a synchronization failure is detected, avoiding data corruption or environmental chaos and protecting the stable operation of the software system. At the same time, the generation of early warning information facilitates the rapid response and resolution of database synchronization problems by operation and maintenance personnel, improving the efficiency of the overall development process.

[0173] In summary, through the above steps, the synchronization system not only verifies the execution effect of the change instructions and ensures a high degree of consistency in the table structure between the target database and the source database, but also builds a complete error detection and correction mechanism through automatic rollback and early warning information generation, thereby improving the reliability and robustness of the synchronization system.

[0174] In one optional embodiment, after writing the change event to the target partition, if the target database is a database in the test environment, the synchronization system generates filtering rules based on the test requirements of the test environment. Each filtering rule corresponds to a rule identifier and a version identifier. The version identifier of the filtering rule is used to characterize the iteration version of the code in the test environment. If the version identifier corresponding to the change event is less than the version identifier of the filtering rule, the synchronization system intercepts the change event in the target partition. If the version identifier corresponding to the change event is greater than or equal to the version identifier of the filtering rule, the synchronization system pushes the change event in the target partition to the target database.

[0175] Optionally, the synchronization system uses JSON (JavaScript Object Notation, a lightweight data exchange format) or SQL-like (Structured Query Language) syntax to define filtering rules, wherein the filtering rules support multi-level nested logic; the filtering rules adopt the default platform's compact persistent storage strategy.

[0176] Optionally, the synchronization system performs rule filtering on the consumer side. That is, the consumer periodically pulls rule snapshots from the topics that store the filtering rules. During the snapshot pulling process, the synchronization system uses a Bloom filter to pre-filter topics that store irrelevant rules.

[0177] Optionally, when updating filtering rules, the synchronization system adopts a dual rule set buffering mechanism. That is, the synchronization system uses Copy-On-Write-Array-List (a thread-safe collection class) to store rules. When updating, a new rule set is first built, and then the old reference rule set is atomically replaced.

[0178] Optionally, each filtering rule also carries a version number, and consumers will discard outdated versions of the filtering rules.

[0179] Optionally, the synchronization system can intelligently filter and control change events based on the iterative versions of the test environment and current test requirements by dynamically generating filtering rules. This mechanism ensures that the database in the test environment only receives change events that match the current test requirements, avoiding interference from irrelevant changes in the testing process and improving the relevance and efficiency of testing. At the same time, the introduction of rule identifiers and version identifiers enhances the traceability of change event handling and version management capabilities, making it easier for testers to track change history and conduct functional verification and performance testing after changes.

[0180] Optionally, if the version identifier corresponding to a change event is less than the version identifier of the filtering rule, the synchronization system intercepts change events in the target partition, ensuring that only change events matching the current test environment version can pass through. This avoids damage to the test environment caused by outdated or incompatible database table structure changes. This interception mechanism not only protects the purity and consistency of the test environment but also ensures the accuracy and validity of test results, reducing test delays and resource waste caused by version mismatch.

[0181] In summary, the synchronization system, by dynamically generating filtering rules and intelligently screening and controlling change events based on version identifiers, not only optimizes the change management process of the testing environment and improves the efficiency and accuracy of testing, but also ensures the stability and purity of the testing environment through version control strategies, supporting the smooth progress of software projects in continuous iteration.

[0182] As can be seen from the above, this application achieves the goal of accurately and in real-time synchronizing the distributed database table structure in a multi-level R&D environment by constructing a subscription relationship chain and using table update events to drive the synchronous update of table structure data in different environments. This application avoids the introduction of human negligence and errors, as well as the inherent delay of scheduled tasks, thereby achieving the technical effect of improving data consistency and synchronization response speed between multi-level R&D environments, and thus solving the technical problem of poor effect of synchronizing the distributed database table structure in a multi-level environment based on existing technologies.

[0183] In one alternative embodiment, Figure 4This is a flowchart of an optional automatic synchronization method for a database table structure according to an embodiment of this application, such as... Figure 4 As shown, the method includes the following steps:

[0184] Step 401: Construct a cascading synchronization control module. Through a preset platform, formulate chain subscription rules between multiple preceding and subsequent development environments. That is, the subsequent version environment needs to subscribe to the distributed database table structure change messages of the preceding version environment.

[0185] Optionally, chained subscription rules are used to define environment topology relationships (e.g., environment A iteration version -> environment B iteration version -> environment C iteration version), and different environments pass change messages hierarchically through a preset platform.

[0186] Optionally, the cascaded synchronization control module execution order guarantee design includes:

[0187] (1) Attach a version serial number (e.g., [v1.3]-[DDL-002]) to each change event.

[0188] (2) Strictly verify the continuity of the sequence when consuming in the downstream environment, and trigger an alarm message when the sequence is interrupted.

[0189] Optionally, the cascading synchronization control module adopts an automatic recursive execution method, that is, after the B iteration version environment completes local synchronization, it automatically pushes messages to the C iteration version environment to achieve relay synchronization of the A iteration version environment -> B iteration version environment -> C iteration version environment.

[0190] Step 402: Construct a heterogeneous platform adaptation layer, which controls each environment to transmit table structure change messages according to subscription rules. By formulating a unified message protocol and using a dynamic conversion engine, changes to different database table structures can be realized.

[0191] Optionally, the specific steps for building a heterogeneous platform adaptation layer are as follows:

[0192] (1) Define a unified message protocol: that is, define a general structure description language that is independent of the database.

[0193] (2) Set up a dynamic conversion engine: a converter is built into the consumer side to translate the source DDL statement into the native DDL corresponding to the target database in real time according to the target database type.

[0194] Step 403: Build a sandbox environment subscription engine. Through a preset platform, formulate subscription rules between special testing environments and development environments. For example, the sandbox testing environment needs to subscribe to the distributed database table structure change messages of the current version environment.

[0195] Optionally, the sandbox environment subscription engine can be configured as follows:

[0196] (1) Fine-grained rule configuration, defining sandbox instance ID, subscription source environment, filtering rules, etc.

[0197] (2) Message routing mechanism: The sandbox environment filters and consumes specific change sets according to rules.

[0198] Step 404: When the database table structure of the subscribed environment changes, the distributed database table structure change message of the subscribed environment is pushed to a preset queue. Each subscribed environment consumes the distributed database table structure change message and changes the distributed database table structure of its own environment.

[0199] Step 405: Implement changes to the database table structure in the subscription environment.

[0200] As can be seen from the above, this application, through the message queue and dynamic subscription rule engine provided by the preset platform, achieves real-time and accurate synchronization of distributed database table structures with automated protection, and can achieve the following functions:

[0201] (1) Improved accuracy of cascading: By controlling the version sequence, the risk of errors in the order of manual operation is eliminated, and the consistency of the environment is improved.

[0202] (2) Platform scalability breakthrough: Supports table structure synchronization between different types of databases.

[0203] (3) Operation and maintenance efficiency optimization: The time taken for database table structure synchronization in the sandbox environment has been reduced from 30 minutes to seconds, and resource consumption has been reduced by 70%, thus improving operation and maintenance efficiency.

[0204] Example 2

[0205] This application embodiment also provides a database table structure synchronization device. It should be noted that the database table structure synchronization device of this application embodiment can be used to execute the database table structure synchronization method provided in this application embodiment. The following describes the database table structure synchronization device provided in this application embodiment.

[0206] According to an embodiment of this application, an apparatus for implementing the synchronization method of the above-described database table structure is also provided. Figure 5 This is a schematic diagram of a synchronization device for an optional database table structure according to an embodiment of this application, such as... Figure 5 As shown, the device includes: a creation unit 501, an event generation unit 502, an event push unit 503, and a table structure update unit 504.

[0207] Optionally, the establishment unit 501 is used to establish a subscription relationship chain, wherein the subscription relationship chain is used to represent the subscription relationship between databases in the R&D environment and the test environment; the event generation unit 502 is used to generate a change event when the table structure of the source database is updated, wherein the change event is used to represent the statement information of the first statement, the first statement is used to update the table structure of the source database, and the source database is the database subscribed to in the subscription relationship chain; the event push unit 503 is used to push the change event to the target database based on the subscription relationship chain, wherein the target database is the database that subscribes to the source database in the subscription relationship chain; and the table structure update unit 504 is used to update the table structure of the target database based on the change event.

[0208] In one optional embodiment, the event generation unit 502 includes an event element determination subunit and an encapsulation subunit.

[0209] Optionally, the event element determination subunit is used to determine N event elements based on the statement information of the first statement, wherein the N event elements are at least one of the following: a first element, used to characterize the operation type of the database operation that updates the table structure of the source database; a second element, used to characterize the table name of the database table whose table structure is being updated in the source database; and a third element, used to characterize the type of the source database. The encapsulation subunit is used to encapsulate the N event elements and the source code corresponding to the first statement based on a preset protocol to obtain a change event, wherein the change event corresponds to an event identifier and a version identifier, and the version identifier of the change event is used to characterize the iteration version of the code in the environment to which the source database belongs.

[0210] In one optional embodiment, the event push unit 503 includes: a partitioning strategy determination subunit, a target partition determination subunit, and an event push subunit.

[0211] Optionally, the partitioning strategy determination subunit is used to determine the partitioning allocation strategy corresponding to the preset platform based on the subscription relationship chain; the target partition determination subunit is used to determine the target partition corresponding to the change event based on the partitioning allocation strategy, wherein the target partition is the partition in the preset platform used to store the change event; and the event push subunit is used to push the change event in the target partition to the target database after writing the change event to the target partition.

[0212] In one optional embodiment, the table structure update unit 504 includes: a mapping table acquisition subunit, an event parsing subunit, a rule filtering subunit, a field update subunit, and a table structure update subunit.

[0213] Optionally, the mapping table retrieval subunit is used to retrieve a metadata mapping table, wherein the metadata mapping table includes at least the mapping relationships between fields corresponding to different types of databases; the event parsing subunit is used to parse change events and obtain parsing results, wherein the parsing results include at least the code function corresponding to the first statement and the source code corresponding to the first statement; the rule filtering subunit is used to filter the metadata mapping table based on the type of the target database to obtain the mapping rules for the source code corresponding to the first statement; the field update subunit is used to update the fields in the source code corresponding to the first statement based on the mapping rules to obtain the second statement; and the table structure update subunit is used to update the table structure of the target database based on the second statement.

[0214] In one optional embodiment, the database table structure synchronization device further includes: a schema conversion unit, a scheme filtering unit, and a statement generation unit.

[0215] Optionally, the schema conversion unit is used to obtain a schema conversion matrix when there is no mapping rule for the source code corresponding to the first statement in the metadata mapping table, wherein the schema conversion matrix includes at least equivalent implementation schemes of the code functions implemented by database operation statements in different types of databases; the scheme filtering unit is used to filter in the schema conversion matrix based on the type of the target database to obtain equivalent implementation schemes of the code functions corresponding to the first statement; and the statement generation unit is used to generate a second statement based on the equivalent implementation schemes of the code functions corresponding to the first statement.

[0216] In one optional embodiment, the database table structure synchronization device further includes: a command execution unit, a similarity determination unit, a first synchronization result determination unit, a second synchronization result determination unit, and a rollback unit.

[0217] Optionally, the command execution unit is used to execute a preset command in the target database table to obtain the target table structure, wherein the target table structure is the table structure of the target database obtained after executing the second statement; the similarity determination unit is used to obtain the similarity between the source table structure and the target table structure, wherein the source table structure is the table structure of the source database obtained after executing the first statement; the first synchronization result determination unit is used to determine that the database table structure synchronization between the target database and the source database is successful if the similarity is greater than or equal to a preset similarity; the second synchronization result determination unit is used to determine that the database table structure synchronization between the target database and the source database fails if the similarity is less than a preset similarity; and the rollback unit is used to execute the rollback operation corresponding to the second statement and generate warning information if the database table structure synchronization between the target database and the source database fails.

[0218] In one optional embodiment, the database table structure synchronization device further includes: a filtering rule generation unit, a first event processing unit, and a second event processing unit.

[0219] Optionally, the filtering rule generation unit is used to generate filtering rules based on the testing requirements of the testing environment when the target database is a database in the testing environment. The filtering rule corresponds to a rule identifier and a version identifier, and the version identifier of the filtering rule is used to represent the iteration version of the code in the testing environment. The first event processing unit is used to intercept the change event in the target partition when the version identifier corresponding to the change event is less than the version identifier of the filtering rule. The second event processing unit is used to push the change event in the target partition to the target database when the version identifier corresponding to the change event is greater than or equal to the version identifier of the filtering rule.

[0220] It should be noted that the aforementioned establishment unit 501, event generation unit 502, event push unit 503, and table structure update unit 504 correspond to steps S201 to S204 in the method embodiment. The instances and application scenarios implemented by the aforementioned units and corresponding steps are the same, but are not limited to the content disclosed in the aforementioned embodiments. It should be noted that the aforementioned modules or units may be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The aforementioned modules may also be part of a device and can run in the computer terminal 10 provided in the embodiment.

[0221] As can be seen from the above, this application achieves the goal of accurately and in real-time synchronizing the distributed database table structure in a multi-level R&D environment by constructing a subscription relationship chain and using table update events to drive the synchronous update of table structure data in different environments. This application avoids the introduction of human negligence and errors, as well as the inherent delay of scheduled tasks, thereby achieving the technical effect of improving data consistency and synchronization response speed between multi-level R&D environments, and thus solving the technical problem of poor effect of synchronizing the distributed database table structure in a multi-level environment based on existing technologies.

[0222] Example 3

[0223] Embodiments of this application can also provide an electronic device. Figure 6 This is a structural block diagram of an electronic device according to an embodiment of this application, such as... Figure 6 As shown, the electronic device includes: one or more ( Figure 6 (Only one is shown) Processor 602, memory 604, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0224] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and devices in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, to realize the above-mentioned method for synchronizing the database table structure.

[0225] The memory may include high-speed random access memory (RAM), and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, which can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0226] The processor can invoke information and applications stored in memory via a transmission device to perform the following steps: establishing a subscription relationship chain, wherein the subscription relationship chain represents the subscription relationship between databases in the R&D environment and the test environment; generating a change event when the table structure of the source database is updated, wherein the change event represents the statement information of a first statement used to update the table structure of the source database, the source database being the database subscribed to in the subscription relationship chain; pushing the change event to the target database based on the subscription relationship chain, wherein the target database is the database in the subscription relationship chain that subscribes to the source database; and updating the table structure of the target database based on the change event.

[0227] The processor can invoke information and application programs stored in memory via a transmission device to perform the following steps: Based on the statement information of the first statement, determine N event elements, wherein the N event elements are at least one of the following: a first element, used to characterize the operation type of the database operation that updates the table structure of the source database; a second element, used to characterize the table name of the database table in the source database whose table structure is being updated; a third element, used to characterize the type of the source database; based on a preset protocol, encapsulate the N event elements and the source code corresponding to the first statement to obtain a change event, wherein the change event corresponds to an event identifier and a version identifier, and the version identifier of the change event is used to characterize the iterative version of the code in the environment to which the source database belongs.

[0228] The processor can access information and applications stored in memory via a transmission device to perform the following steps: determining the partition allocation strategy corresponding to the preset platform based on the subscription relationship chain; determining the target partition corresponding to the change event based on the partition allocation strategy, wherein the target partition is the partition in the preset platform used to store the change event; and after writing the change event to the target partition, pushing the change event in the target partition to the target database.

[0229] The processor can invoke information and applications stored in memory via a transmission device to perform the following steps: obtaining a metadata mapping table, wherein the metadata mapping table includes at least the mapping relationships between fields corresponding to different types of databases; parsing change events to obtain parsing results, wherein the parsing results include at least the code function corresponding to the first statement and the source code corresponding to the first statement; filtering the metadata mapping table based on the type of the target database to obtain the mapping rules for the source code corresponding to the first statement; updating the fields in the source code corresponding to the first statement based on the mapping rules to obtain the second statement; and updating the table structure of the target database based on the second statement.

[0230] The processor can invoke information and applications stored in memory via a transmission device to perform the following steps: if there is no mapping rule for the source code corresponding to the first statement in the metadata mapping table, obtain a schema transformation matrix, wherein the schema transformation matrix includes at least equivalent implementation schemes of the code functions implemented by database operation statements in different types of databases; based on the type of the target database, filter in the schema transformation matrix to obtain equivalent implementation schemes of the code functions corresponding to the first statement; and generate a second statement based on the equivalent implementation schemes of the code functions corresponding to the first statement.

[0231] The processor can invoke information and application programs stored in memory via a transmission device to perform the following steps: Execute a preset command in the target database table to obtain the target table structure, wherein the target table structure is the table structure of the target database obtained after executing the second statement; obtain the similarity between the source table structure and the target table structure, wherein the source table structure is the table structure of the source database obtained after executing the first statement; if the similarity is greater than or equal to a preset similarity, determine that the database table structure synchronization between the target database and the source database is successful; if the similarity is less than the preset similarity, determine that the database table structure synchronization between the target database and the source database has failed; if the database table structure synchronization between the target database and the source database has failed, execute the rollback operation corresponding to the second statement and generate a warning message.

[0232] The processor can access information and applications stored in memory via a transmission device to perform the following steps: If the target database is a database in the test environment, generate filtering rules based on the test requirements of the test environment. Each filtering rule corresponds to a rule identifier and a version identifier. The version identifier of the filtering rule is used to characterize the iterative version of the code in the test environment. If the version identifier corresponding to a change event is less than the version identifier of the filtering rule, intercept the change event in the target partition. If the version identifier corresponding to a change event is greater than or equal to the version identifier of the filtering rule, push the change event in the target partition to the target database.

[0233] This application provides a scheme for synchronizing database table structures. By constructing a subscription relationship chain and using table update events to drive the synchronous updating of table structure data in different environments, this application achieves accurate and real-time synchronization of distributed database table structures in multi-level development environments. This avoids the introduction of human error and the inherent delays of scheduled tasks, thereby improving data consistency and synchronization response speed between multi-level development environments. It also solves the technical problem of poor synchronization performance of distributed database table structures in multi-level environments based on existing technologies.

[0234] Those skilled in the art will understand that Figure 6 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, PDAs, mobile internet devices, PADs, and other terminal devices. Figure 6 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 6 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 6 The different configurations shown.

[0235] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0236] Example 4

[0237] Embodiments of this application may also provide a storage medium.

[0238] Optionally, in this embodiment of the application, the storage medium can be used to store the program code executed by the synchronization method of the database table structure provided in the above method embodiment.

[0239] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0240] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing synchronization method steps of a database table structure.

[0241] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0242] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0243] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0244] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0245] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0246] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0247] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of synchronizing database table structures, characterized by, include: Establish a subscription relationship chain, wherein the subscription relationship chain is used to represent the subscription relationship between the databases in the R&D environment and the test environment; When the table structure of the source database is updated, a change event is generated, wherein the change event is used to characterize the statement information of the first statement, the first statement is used to update the table structure of the source database, and the source database is the database subscribed to in the subscription relationship chain; Based on the subscription relationship chain, the change event is pushed to the target database, wherein the target database is the database in the subscription relationship chain that subscribes to the source database; Based on the change event, the table structure of the target database is updated.

2. The method of synchronizing database table structures of claim 1, wherein, When the table structure of the source database is updated, change events are generated, including: Based on the statement information of the first statement, N event elements are determined, wherein the N event elements are at least one of the following: The first element is used to characterize the type of database operation that updates the table structure of the source database; The second element is the name of the database table whose structure is being updated in the source database. The third element is used to characterize the type of the source database; Based on a preset protocol, the N event elements and the source code corresponding to the first statement are encapsulated to obtain the change event. The change event corresponds to an event identifier and a version identifier. The version identifier of the change event is used to characterize the iterative version of the code in the environment to which the source database belongs.

3. The method of claim 1, wherein, Based on the subscription relationship chain, the change event is pushed to the target database, including: Based on the subscription relationship chain, determine the partition allocation strategy corresponding to the preset platform; Based on the partition allocation strategy, the target partition corresponding to the change event is determined, wherein the target partition is the partition in the preset platform used to store the change event; After the change event is written to the target partition, the change event in the target partition is pushed to the target database.

4. The method of claim 1, wherein, Based on the change event, the table structure of the target database is updated, including: Obtain a metadata mapping table, wherein the metadata mapping table includes at least the mapping relationship between fields corresponding to different types of databases; The change event is parsed to obtain a parsing result, wherein the parsing result includes at least the code function corresponding to the first statement and the source code corresponding to the first statement; Based on the type of the target database, the metadata mapping table is filtered to obtain the mapping rules for the source code corresponding to the first statement; Based on the mapping rules, the fields in the source code corresponding to the first statement are updated to obtain the second statement; Based on the second statement, the table structure of the target database is updated.

5. The database table structure synchronization method according to claim 4, characterized in that, After filtering the metadata mapping table based on the type of the target database, the method for synchronizing the database table structure further includes: If there is no mapping rule for the source code corresponding to the first statement in the metadata mapping table, obtain the schema transformation matrix, wherein the schema transformation matrix includes at least equivalent implementation schemes of the code functions implemented by database operation statements in different types of databases; Based on the type of the target database, the equivalent implementation scheme of the code function corresponding to the first statement is obtained by filtering in the pattern conversion matrix. The second statement is generated based on an equivalent implementation scheme of the code function corresponding to the first statement.

6. The database table structure synchronization method according to claim 1, characterized in that, After updating the table structure of the target database based on the change event, the database table structure synchronization method further includes: Execute a preset command in the target database table to obtain the target table structure, wherein the target table structure is the table structure of the target database obtained after executing the second statement; Obtain the similarity between the source table structure and the target table structure, wherein the source table structure is the table structure of the source database obtained after executing the first statement; If the similarity is greater than or equal to a preset similarity, it is determined that the database table structure synchronization between the target database and the source database is successful. If the similarity is less than the preset similarity, it is determined that the database table structure synchronization between the target database and the source database has failed. If the database table structure synchronization between the target database and the source database fails, execute the rollback operation corresponding to the second statement and generate a warning message.

7. The database table structure synchronization method according to claim 3, characterized in that, After writing the change event to the target partition, the database table structure synchronization method further includes: When the target database is a database in the test environment, filtering rules are generated based on the test requirements of the test environment. Each filtering rule corresponds to a rule identifier and a version identifier. The version identifier of the filtering rule is used to characterize the iterative version of the code in the test environment. If the version identifier corresponding to the change event is less than the version identifier of the filtering rule, the change event in the target partition is intercepted. If the version identifier corresponding to the change event is greater than or equal to the version identifier of the filtering rule, the change event in the target partition will be pushed to the target database.

8. A synchronization device for a database table structure, characterized in that, include: A unit is established to establish a subscription relationship chain, wherein the subscription relationship chain is used to characterize the subscription relationship between databases in the R&D environment and the test environment; An event generation unit is used to generate a change event when the table structure of the source database is updated. The change event is used to characterize the statement information of a first statement, which is used to update the table structure of the source database, and the source database is the database subscribed to in the subscription relationship chain. An event push unit is used to push the change event to a target database based on the subscription relationship chain, wherein the target database is a database in the subscription relationship chain that subscribes to the source database; The table structure update unit is used to update the table structure of the target database based on the change event.

9. A computer program product, characterized in that, The computer program product includes a computer program, wherein, when the computer program is executed, it controls the computer program product to perform the synchronization method of the database table structure according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the synchronization method of the database table structure according to any one of claims 1 to 7.

Images