Data Synchronization Method, Device, Computer Equipment and Storage Medium

Abstract

The present application relates to a data synchronization method, apparatus, computer device, and storage medium. The method includes: obtaining the business data that has been uploaded in the service database of the professional group within a preset CMDB synchronization framework platform; performing Quartz periodic scheduling logic on the business data to determine the data to be synchronized in the business data; and updating the configuration data in the CMDB database according to the data to be synchronized. In the present application, the full amount of business data is obtained through a preset CMDB synchronization framework platform, and then the data to be synchronized in the business data is determined by performing periodic scheduling logic on the CMDB synchronization framework platform, and then data synchronization is performed. Through the preset CMDB synchronization framework platform, the synchronization management in the CMDB data synchronization process can be effectively realized.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a data synchronization method, apparatus, computer device, and storage medium. Background Art

[0002] With the rapid development of computer technology, database technology is also constantly evolving. The Configuration Management Database (CMDB) is a logical database that contains information about the entire life cycle of configuration items and the relationships between configuration items, including physical relationships, real-time communication relationships, non-real-time communication relationships, and dependency relationships. The CMDB stores and manages various configuration information of devices in an enterprise's IT architecture. It is closely associated with all service support and service delivery processes, supports the operation of these processes, realizes the value of configuration information, and at the same time depends on relevant processes to ensure data accuracy.

[0003] Currently, the CMDB obtains business data transmitted from each interface through an interface and stores it in a separate database for data synchronization processing. However, for this data synchronization process, the CMDB cannot perform real-time monitoring, and log maintenance is also relatively difficult, making it impossible to effectively manage the data synchronization situation. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a data synchronization method, apparatus, computer device, and storage medium that can manage the data synchronization process under the CMDB.

[0005] A data synchronization method, the method comprising:

[0006] Obtain the business data uploaded in a preset CMDB synchronization framework platform, wherein the preset CMDB synchronization framework platform includes professional groups, the service databases of the professional groups are constructed based on the entity dependency relationships within the professional groups, and are respectively connected to the business application database and the CMDB database. The preset CMDB synchronization framework is used to obtain the business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database;

[0007] Perform Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data;

[0008] Update the configuration data in the CMDB database according to the data to be synchronized.

[0009] In one embodiment, before obtaining the business data uploaded in the preset CMDB synchronization framework platform, the method further includes:

[0010] Obtain entities and entity attributes in the professional group within the preset CMDB synchronization framework platform;

[0011] Obtain the association relationships between the entities according to the entity attributes;

[0012] Stitch the entities according to the association relationships to construct the entity dependency relationships within the professional group.

[0013] In one embodiment, after determining the data to be synchronized in the service data and before updating the service data in the CMDB database according to the data to be synchronized, it further includes:

[0014] Query the unique constraint key corresponding to the data to be synchronized;

[0015] Perform DEL processing on the data to be synchronized according to the unique constraint key and record the DEL status;

[0016] Verify the correctness and integrity of the data to be synchronized according to the DEL status through the convention of the unique constraint key;

[0017] The updating of the service data in the CMDB database according to the data to be synchronized includes:

[0018] Update the configuration data in the CMDB database according to the data to be synchronized that has passed the correctness verification and integrity verification.

[0019] In one embodiment, the determining of the data to be synchronized in the service data by performing Quartz periodic scheduling logic determination on the service data includes:

[0020] Obtain the historical synchronization times of various data synchronization operations in the previous round of data synchronization in the historical record;

[0021] Determine the current synchronization time according to the historical synchronization times;

[0022] Determine the data to be synchronized corresponding to the current synchronization time according to the service data received within the current synchronization time;

[0023] When reaching the Quartz periodic scheduling logic time node of the next round, return to the step of obtaining the historical synchronization times of various data synchronization operations in the previous round of data synchronization in the historical record.

[0024] In one embodiment, after updating the configuration data in the CMDB database according to the data to be synchronized, it further includes:

[0025] When the data synchronization is completed, a data synchronization message in a preset agreed format is fed back according to the data content corresponding to the data to be synchronized.

[0026] In one embodiment, before obtaining the business data uploaded in the preset CMDB synchronization framework platform, it further includes:

[0027] Construct a first KAFKA channel between the business application database corresponding to the business data and the professional group, and a second KAFKA channel between the professional group and the CMDB database;

[0028] Uniformly configure the proxy addresses of the first KAFKA channel and the second KAFKA channel.

[0029] A data synchronization device, the device includes:

[0030] A data acquisition module, configured to acquire the business data uploaded in the preset CMDB synchronization framework platform, where the preset CMDB synchronization framework platform includes a professional group, the service database of the professional group is constructed based on the entity dependency relationship within the professional group, and is respectively connected to the business application database and the CMDB database, and the preset CMDB synchronization framework is used to acquire the business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database;

[0031] A synchronization data screening module, configured to perform Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data;

[0032] A data synchronization module, configured to update the configuration data in the CMDB database according to the data to be synchronized.

[0033] In one embodiment, it further includes a dependency construction module, configured to: acquire the entities and entity attributes in the professional group in the preset CMDB synchronization framework platform; acquire the association relationship between the entities according to the entity attributes; splice the entities according to the association relationship to construct the entity dependency relationship within the professional group.

[0034] A computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Obtain the business data uploaded within the preset CMDB synchronization framework platform. Among them, the preset CMDB synchronization framework platform includes professional groups. The service databases of the professional groups are constructed based on the entity dependency relationships within the professional groups, and are respectively connected to the business application database and the CMDB database. The preset CMDB synchronization framework is used to obtain the business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database;

[0036] Perform Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data;

[0037] Update the configuration data in the CMDB database according to the data to be synchronized.

[0038] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0039] Obtain the business data uploaded within the preset CMDB synchronization framework platform. Among them, the preset CMDB synchronization framework platform includes professional groups. The service databases of the professional groups are constructed based on the entity dependency relationships within the professional groups, and are respectively connected to the business application database and the CMDB database. The preset CMDB synchronization framework is used to obtain the business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database;

[0040] Perform Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data;

[0041] Update the configuration data in the CMDB database according to the data to be synchronized.

[0042] The above data synchronization method, device, computer device and storage medium obtain the business data uploaded within the preset CMDB synchronization framework platform; perform Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data; update the configuration data in the CMDB database according to the data to be synchronized. This application obtains the uploaded business data through the preset CMDB synchronization framework platform, and then determines the data to be synchronized in the business data through the periodic scheduling logic of the CMDB synchronization framework platform, and then performs data synchronization. Obtain business data through the professional groups of the preset CMDB synchronization framework platform and perform grouped data synchronization, which can more effectively implement the synchronization management in the CMDB data synchronization process. Description of the Drawings

[0043] Figure 1It is an application environment diagram of the data synchronization method in an embodiment;

[0044] Figure 2 It is a schematic flowchart of the data synchronization method in an embodiment;

[0045] Figure 3 It is a synchronization schematic diagram of the cloud CMDB synchronization framework platform with the professional group and the cloud CMDB platform in an embodiment;

[0046] Figure 4 It is a schematic flowchart of the steps for building the entity dependency relationship within the professional group in an embodiment;

[0047] Figure 5 It is a schematic diagram of the target entities within the professional group and the relationships between the entities in an embodiment;

[0048] Figure 6 It is a schematic flowchart of the steps for performing data verification in an embodiment;

[0049] Figure 7 For an embodiment Figure 2 It is a sub - flowchart diagram of step 203 in

[0050] Figure 8 It is a structural block diagram of the data synchronization device in an embodiment;

[0051] Figure 9 It is an internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0052] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] The data synchronization method provided by the present application can be applied to, for example Figure 1In the application environment shown. Among them, multiple business servers 102 are connected to the data synchronization server 104 through a network. The data synchronization server 104 includes a preset CMDB synchronization framework platform. In one embodiment, each professional group is included in the preset CMDB synchronization framework platform, and the professional groups correspond to the business servers 102 one by one. A service database is included in each professional group. At the same time, the data synchronization server 104 is connected to the CMDB database 106 of the cloud CMDB platform through a network. Specifically, the preset CMDB synchronization framework platform can obtain the full amount of business data from each business server and save it to the service database of the corresponding professional group in the preset CMDB synchronization framework platform. The data synchronization server 104 obtains the business data that has been uploaded in the preset CMDB synchronization framework platform; performs Quartz periodic scheduling logic on the business data to determine the data to be synchronized in the business data; and updates the configuration data in the CMDB database according to the data to be synchronized. Among them, the business server 102 and the data synchronization server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0054] In one embodiment, as Figure 2 shown, a data synchronization method is provided. Taking the data synchronization server 104 in Figure 1 as an example, the method includes the following steps:

[0055] Step 201, obtain the business data that has been uploaded in the preset CMDB synchronization framework platform. The preset CMDB synchronization framework platform includes professional groups. The service databases of the professional groups are constructed based on the entity dependencies within the professional groups, and are respectively connected to the business application database and the CMDB database. The preset CMDB synchronization framework is used to obtain business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database.

[0056] Among them, the CMDB database, that is, the Configuration Management Database, is a logical database that contains information about the entire life cycle of configuration items and the relationships between configuration items, including physical relationships, real-time communication relationships, non-real-time communication relationships, and dependency relationships. The Configuration Management Database stores and manages various configuration information of devices in the enterprise IT architecture. It is closely connected to all service support and service delivery processes, supports the operation of these processes, gives play to the value of configuration information, and at the same time depends on relevant processes to ensure the accuracy of data. The preset CMDB synchronization framework platform refers to the platform pre-constructed in this application for synchronizing this part of the business data related to configuration. The preset CMDB synchronization framework platform can be connected to the databases of business applications, receive business data from these databases, and then synchronize it to the CMDB database. The preset CMDB synchronization framework platform contains professional groups, such as testing, production / public cloud, and external network, etc., which are respectively used to receive business data from different sources. The service database specifically refers to database services, such as RDS (Relational Database Service). RDS is an on-demand, stable, reliable, and elastically scalable online database service. It has multiple security protection measures and a perfect performance monitoring system, and provides professional database backup, recovery, and optimization solutions, making it more convenient for application development and business development. As Figure 3 shown, the solution of this application is used to synchronize data from the old CMDB database (which can be equivalent to the business application database) to the cloud CMDB database. At this time, the testing professional group can receive business data from the old CMDB database. Then, after processing, it is synchronously saved to the cloud CMDB database. At the same time, the professional group specifically includes a service database, including CVM (Cloud Virtual Machine) or RDS. After extracting cloud data through the professional group, it is saved to the cloud CMDB database. By specifying data partitions through the professional group and then performing data synchronization processing, the problems of consumption blockage and slow database storage during the data synchronization process can be effectively solved.

[0057] Step 203, perform a Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data.

[0058] Among them, Quartz is a timing task framework and another open-source project of the OpenSymphony open-source organization in the Job scheduling field. It is developed based on the Java language and can be used to execute timing tasks, similar to java.util.Timer. However, compared with Timer, Quartz has added many functions, such as persistent jobs, that is, maintaining the scheduling timing state; and job management, that is, effectively managing scheduling jobs.

[0059] Specifically, in the solution of this application, the timed data synchronization logic is mainly implemented through Quartz. Quartz can be used to perform periodic logical checks on business data to determine the data to be synchronized that needs to be synchronized in each period.

[0060] Step 205: Update the configuration data in the CMDB database according to the data to be synchronized.

[0061] Specifically, after filtering out the data that needs to be synchronized in the current period, the configuration data in the CMDB database can be updated according to this data to be synchronized, thereby realizing the corresponding data synchronization. In this application, the data synchronization server 104 can perform CMDB-related data synchronization management through a preset CMDB synchronization framework platform. First, it receives the relevant business data uploaded by the business server, and then realizes synchronization through the service database of the professional group in the preset CMDB synchronization framework platform, ensuring that the information in the CMDB can reflect the update situation of the combined data source, and determining the CMDB update based on the update frequency of the combined data source. It helps to improve the data accuracy rate and achieve unified centralized collection and management of the CMDB data synchronization process.

[0062] The above data synchronization method includes: obtaining the business data uploaded in the preset CMDB synchronization framework platform; performing Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data; updating the configuration data in the CMDB database according to the data to be synchronized. In this application, the preset CMDB synchronization framework platform is used to obtain the uploaded business data, and then the periodic scheduling logic of the CMDB synchronization framework platform is used to determine the data to be synchronized in the business data, and then data synchronization is performed. The professional group of the preset CMDB synchronization framework platform obtains the business data and then performs grouped data synchronization, which can more effectively realize the synchronization management in the CMDB data synchronization process.

[0063] In one embodiment, as Figure 4 shown, before step 201, it further includes:

[0064] Step 401: Obtain the entities and entity attributes in the professional group in the preset CMDB synchronization framework platform.

[0065] Step 403: Obtain the association relationship between entities according to the entity attributes.

[0066] Step 405: Splice the entities according to the association relationship to construct the entity dependency relationship within the professional group.

[0067] Among them, the entities of the preset CMDB synchronization framework platform in this application specifically include entities such as computer rooms, cabinets, U positions, physical devices (physical hosts and network devices), physical operating systems (Operating System), applications (MYSQL / REDIS / LB / JETTY / OSS / WAF), and services. It also includes entities in terms of networks such as tenants, virtual port channels (VPCs), network segments, IPs (Internet Protocol), regions, pods, and DNS. In addition, it includes virtualization and container entities such as KVM / DOCKER, and file system entities such as VOLUME GFS / NAS. Before constructing the professional group, the data synchronization server can first obtain these entities and their attributes, then determine the entity association relationships within each professional group based on the entity attributes, and complete the splicing of the entities based on the association relationships, thereby constructing the entity dependency relationships within the professional group, and then obtaining the available professional groups.

[0068] Specifically, entities such as computer rooms, cabinets, U positions, physical devices (physical hosts and network devices), physical operating systems, applications, and services are sequentially associated and have a one-way dependency relationship. The entities within the professional group can be determined in advance, and then the dependencies of the entities can be constructed based on the association relationships of the entities. In one embodiment, the target entities within the professional group and the relationships between the entities can specifically refer to Figure 5 , where the one-way arrow indicates that there is a one-to-one or one-to-N relationship between two entities, and the two-way arrow indicates that there is an N-to-N relationship between two entities. In this embodiment, by obtaining the entities and their attributes of the professional group, constraint relationships are established accordingly, which can ensure the accuracy of the cloud CMDB synchronization framework and also ensure the correctness of the data flow during the data synchronization process.

[0069] As Figure 6 shown, in one embodiment, before step 205, it further includes:

[0070] Step 601, query the unique constraint key corresponding to the data to be synchronized.

[0071] Step 603, perform DEL processing on the data to be synchronized according to the unique constraint key, and record the DEL status.

[0072] Step 605, according to the DEL status, verify the correctness and integrity of the data to be synchronized through the convention of the unique constraint key.

[0073] Step 205 includes:

[0074] Step 607: Update the configuration data in the CMDB database according to the data to be synchronized that has passed the correctness verification and integrity verification.

[0075] The unique constraint key is the UK (Unique Key) key. The unique constraint can ensure the uniqueness of records. The DEL process is the deletion process, and the data is verified through the status of the DEL process.

[0076] Specifically, in this application, the verification of data storage is performed through the UK verification based on the UK key. First, obtain the UK key, and then perform the DEL process on each piece of data to be synchronized with the UK as the condition, and record the status of the DEL. Through the DEL process, if there is a lower-level sub-table in the sub-table of the data to be synchronized, a 1452 error (the inserted foreign key data does not exist in the primary key of another table) will be prompted. The reason for this error is that the dependency relationship of some business data is incorrect. The integrity and correctness of the data to be synchronized cannot be guaranteed. By verifying all the data to be synchronized, the corresponding verification results are obtained. Then, according to the data to be synchronized that has passed the correctness verification and integrity verification, the configuration data in the CMDB database is updated. To ensure the integrity and correctness of the data synchronized into the CMDB database. Specifically, the CMDB database is a logical database used to store and manage various configuration information of devices in the enterprise IT architecture, specifically including information on the entire life cycle of configuration items and the relationships between configuration items (including physical relationships, real-time communication relationships, non-real-time communication relationships, and dependency relationships). When the CMDB database receives the data to be synchronized that has passed the correctness verification and integrity verification, the database will search for the original configuration information in the database corresponding to the device identifier based on the device identifier corresponding to this part of the data to be synchronized, and replace the original configuration information according to the data to be synchronized. For the case where there is no original configuration information in the database, this newly added configuration information is directly saved into the database. In addition, the data to be synchronized can also include information for performing data deletion, which is used to delete specified configuration information in the CMDB database. In this embodiment, the correctness and integrity of the data are ensured through the UK convention. It is necessary to synchronize the dependency information upstream first, and then downstream can synchronize to ensure alignment with the complete data source. Different professional groups are distinguished according to the theme, and the UK is uniformly verified and judged, which can effectively prevent key business data from being modified.

[0077] In one embodiment, as Figure 7 shown, step 203 includes:

[0078] Step 702: Obtain the historical synchronization times of various data synchronization operations in the previous round of data synchronization in the historical record.

[0079] Step 704: Determine the current synchronization time based on the historical synchronization time.

[0080] Step 706: Determine the data to be synchronized corresponding to the current synchronization time based on the service data received within the current synchronization time.

[0081] In addition, when reaching the time node of the Quartz periodic scheduling logic for the next round, return to the step of obtaining the historical synchronization time of various data synchronization operations in the previous round of data synchronization in the historical record. This step is to execute the Quartz periodic scheduling logic. First, determine the synchronization time of the previous round of data from the historical data, and then combine the period to determine the synchronization update execution time corresponding to the current round. Based on the service data received as of the current synchronization time, it is possible to determine the service data received within the current synchronization period among this part of the received service data, and this part is the data to be synchronized corresponding to the current synchronization time.

[0082] Among them, the historical synchronization time specifically refers to the synchronization time of various data in the CMDB database last time. Based on the last synchronization time and the preset data update delay, it is possible to determine which types of data need to be updated in the current update, and then data updates can be performed according to the determined data types. The preset data update delay can be regarded as the scheduling period of the Quartz periodic scheduling logic. Within each scheduling period, the server will perform a data synchronization to synchronize the data to be synchronized received within the scheduling period to the CMDB database.

[0083] Specifically, in this application, the last synchronization time is sorted out according to the preset synchronization logic. The types of synchronization specifically include data addition, data update, and data deletion. In addition, for these data synchronization types, the latest synchronization time is queried according to the unique dimension: professional group + table + system code + database. Based on the service data received within the current synchronization time, it is possible to screen and determine the data to be synchronized corresponding to the current synchronization time. Data addition refers to adding new configuration data in the CMDB database, data update refers to updating and replacing the original configuration data in the CMDB database, and data deletion refers to deleting the original configuration data in the CMDB database. Through the scheduling logic for iterative loop processing, effective management of data synchronization can be ensured within each logical period. In this embodiment, data screening is performed through the historical synchronization time of various data synchronization operations, which can effectively ensure the accuracy of the screened synchronization data.

[0084] In one embodiment, after step 205, it further includes: when the data synchronization is completed, feedback a data synchronization message in a preset agreed format according to the data content corresponding to the data to be synchronized.

[0085] Specifically, in this application, when the data synchronization is completed, corresponding data synchronization messages can be fed back according to the specific data synchronization situation to facilitate the direct traceability and verification of the data synchronization process. For example, the data synchronization situation can be assembled into the JSON format, then saved to the status table, and then message pushing is performed based on the status table. The pushed messages should conform to the JSON format convention and also follow the JSON parsing habits of Java and Python. The agreed format needs to be adhered to to ensure the readability of the data synchronization messages: it must be "source_count", an integer field. At the same time, check whether there are new records or updates in the corresponding table. And ensure that there is a push count for each synchronization, especially for the synchronization of inserting data and deleting data. In addition, in one embodiment, the data storage situation of the preset CMDB synchronization framework as a consumer can be pushed to the CMDB database in real time, and then the CMDB data and the preset CMDB synchronization framework are used as producers in turn to push back to the business server to facilitate the direct traceability and verification by the business server side. In this embodiment, by feeding back the data synchronization messages in the preset agreed format, the traceability and verifiability during the data synchronization process can be effectively ensured.

[0086] In one embodiment, before step 201, it further includes: constructing a first KAFKA channel for the business data corresponding to the business application database and the professional group, and a second KAFKA channel for the professional group and the CMDB database; uniformly configuring the proxy addresses of the first KAFKA channel and the second KAFKA channel.

[0087] Among them, the KAFKA channel is generally used as a big data channel for data transmission and can meet the requirements of a unified, efficient, low-latency, high-throughput, and highly available platform for big data. By pre-constructing the channel and configuring the proxy address, the accuracy of the business data and the data to be synchronized during the data transmission process can be effectively ensured. In this embodiment, the data channel between the business application and the professional group in the preset CMDB synchronization framework platform is constructed through the KAFKA channel to perform data transmission, and at the same time, the proxy addresses of the first KAFKA channel and the second KAFKA channel are uniformly configured to ensure high recognition.

[0088] It should be understood that although Figure 2-7 the steps in the flowchart Figure 2-7At least some of the steps may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0089] In one embodiment, as Figure 8 shown, a data synchronization device is provided, including: a data acquisition module 801, a synchronization data screening module 803, and a data synchronization module 805, where:

[0090] The data acquisition module 801 is configured to acquire the service data uploaded in the preset CMDB synchronization framework platform. The preset CMDB synchronization framework platform includes professional groups, and the service databases of the professional groups are constructed based on the entity dependencies within the professional groups and are respectively connected to the service application database and the CMDB database. The preset CMDB synchronization framework is used to acquire the service data from the service application database and synchronize the data to be synchronized in the service data to the CMDB database.

[0091] The synchronization data screening module 803 is configured to perform Quartz periodic scheduling logic determination on the service data to determine the data to be synchronized in the service data.

[0092] The data synchronization module 805 is configured to update the configuration data in the CMDB database according to the data to be synchronized.

[0093] In one of the embodiments, a dependency construction module is further included, which is configured to: acquire the entities and entity attributes in the professional groups in the preset CMDB synchronization framework platform; acquire the association relationships and cardinalities between the entities according to the entity attributes; splice the entities according to the association relationships and cardinalities to construct the entity dependencies within the professional groups.

[0094] In one of the embodiments, a data verification module is further included, which is configured to: query the unique constraint key corresponding to the data to be synchronized; perform DEL processing on the data to be synchronized according to the unique constraint key and record the DEL status; verify the correctness and integrity of the data to be synchronized according to the DEL status through the convention of the unique constraint key. The data synchronization module 805 is further configured to: update the configuration data in the CMDB database according to the data to be synchronized that has passed the correctness verification and integrity verification.

[0095] In one embodiment, the synchronous data screening module 803 is further configured to: obtain the historical synchronization time of various data synchronization operations in the previous round of data synchronization in the historical record; determine the current synchronization time according to the historical synchronization time; determine the data to be synchronized corresponding to the current synchronization time according to the service data received within the current synchronization time; and when reaching the Quartz periodic scheduling logic time node of the next round, return to the step of obtaining the historical synchronization time of various data synchronization operations in the previous round of data synchronization in the historical record.

[0096] In one embodiment, it further includes a message feedback module, which is configured to: when the data synchronization is completed, feedback a data synchronization message in a preset agreed format according to the data content corresponding to the data to be synchronized.

[0097] In one embodiment, it further includes a channel construction module, which is configured to: construct a first KAFKA channel between the business application database corresponding to the service data and the professional group, and a second KAFKA channel between the professional group and the CMDB database; and uniformly configure the proxy addresses of the first KAFKA channel and the second KAFKA channel.

[0098] For the specific limitations of the data synchronization device, reference can be made to the limitations on the data synchronization method in the foregoing text, which will not be elaborated here. Each module in the above data synchronization device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0099] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 9 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data synchronization data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a data synchronization method.

[0100] Those skilled in the art can understand, Figure 9The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0101] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0102] Obtain the business data that has been uploaded in the preset CMDB synchronization framework platform. Among them, the preset CMDB synchronization framework platform includes professional groups. The service database of the professional groups is constructed based on the entity dependency relationships within the professional groups, and is respectively connected to the business application database and the CMDB database. The preset CMDB synchronization framework is used to obtain the business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database;

[0103] Perform Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data;

[0104] Update the configuration data in the CMDB database according to the data to be synchronized.

[0105] In one embodiment, when the processor executes the computer program, the following steps are also implemented: Obtain the entities and entity attributes in the professional groups in the preset CMDB synchronization framework platform; Obtain the association relationships between the entities according to the entity attributes; Splice the entities according to the association relationships to construct the entity dependency relationships within the professional groups.

[0106] In one embodiment, when the processor executes the computer program, the following steps are also implemented: Query the unique constraint key corresponding to the data to be synchronized; Perform DEL processing on the data to be synchronized according to the unique constraint key, and record the DEL status; According to the DEL status, verify the correctness and integrity of the data to be synchronized through the convention of the unique constraint key; Update the configuration data in the CMDB database according to the data to be synchronized that passes the correctness verification and integrity verification.

[0107] In one embodiment, when the processor executes the computer program, the following steps are also implemented: Obtain the historical synchronization times of various data synchronization operations in the previous round of data synchronization in the historical record; Determine the current synchronization time according to the historical synchronization time; Determine the data to be synchronized corresponding to the current synchronization time according to the business data received within the current synchronization time.

[0108] In one embodiment, when the processor executes the computer program, the following steps are further implemented: when the data synchronization is completed, a data synchronization message in a preset agreed format is fed back according to the data content corresponding to the data to be synchronized.

[0109] In one embodiment, when the processor executes the computer program, the following steps are further implemented: constructing a first KAFKA channel between the business data corresponding business application database and the professional group, and a second KAFKA channel between the professional group and the CMDB database; uniformly configuring the proxy addresses of the first KAFKA channel and the second KAFKA channel.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0111] Obtaining the business data uploaded in the preset CMDB synchronization framework platform, wherein the preset CMDB synchronization framework platform includes professional groups, the service databases of the professional groups are constructed based on the entity dependency relationships within the professional groups, and are respectively connected to the business application database and the CMDB database. The preset CMDB synchronization framework is used to obtain the business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database;

[0112] Performing a Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data;

[0113] Updating the configuration data in the CMDB database according to the data to be synchronized.

[0114] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining the entities and entity attributes in the professional group in the preset CMDB synchronization framework platform; obtaining the association relationships between the entities according to the entity attributes; splicing the entities according to the association relationships to construct the entity dependency relationships within the professional group.

[0115] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: querying the unique constraint key corresponding to the data to be synchronized; performing a DEL process on the data to be synchronized according to the unique constraint key, and recording the DEL status; verifying the correctness and integrity of the data to be synchronized according to the DEL status through the convention of the unique constraint key; updating the configuration data in the CMDB database according to the data to be synchronized that passes the correctness verification and integrity verification.

[0116] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining the historical synchronization times of various data synchronization operations in the previous round of data synchronization in the historical record; determining the current synchronization time according to the historical synchronization times; and determining the data to be synchronized corresponding to the current synchronization time according to the service data received within the current synchronization time.

[0117] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: when the data synchronization is completed, feeding back a data synchronization message in a preset agreed format according to the data content corresponding to the data to be synchronized.

[0118] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: constructing a first KAFKA channel between the service application database corresponding to the service data and the professional group, and a second KAFKA channel between the professional group and the CMDB database; and uniformly configuring the proxy addresses of the first KAFKA channel and the second KAFKA channel.

[0119] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0120] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0121] The above embodiments only represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A data synchronization method, the method comprises: Obtain the business data uploaded in a preset CMDB synchronization framework platform. Among them, the preset CMDB synchronization framework platform includes professional groups. The service databases of the professional groups are constructed based on the entity dependency relationships within the professional groups, and are respectively connected to the business application database and the CMDB database. The preset CMDB synchronization framework is used to obtain the business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database; Perform Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data; Query the unique constraint key corresponding to the data to be synchronized; Perform DEL processing on the data to be synchronized according to the unique constraint key, and record the DEL status; According to the DEL status, verify the correctness and integrity of the data to be synchronized through the convention of the unique constraint key; Update the configuration data in the CMDB database according to the data to be synchronized that has passed the correctness verification and integrity verification.

2. The method according to claim 1, wherein, Before obtaining the business data uploaded in the preset CMDB synchronization framework platform, it further includes: Obtain the entities and entity attributes in the professional groups in the preset CMDB synchronization framework platform; Obtain the association relationships between the entities according to the entity attributes; Splice the entities according to the association relationships to construct the entity dependency relationships within the professional groups.

3. The method according to claim 1, wherein, The performing Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data includes: Obtain the historical synchronization times of various data synchronization operations in the previous round of data synchronization in the historical record; Determine the current synchronization time according to the historical synchronization times; Determine the data to be synchronized corresponding to the current synchronization time according to the business data received within the current synchronization time; When reaching the Quartz periodic scheduling logic time node of the next round, return to the step of obtaining the historical synchronization times of various data synchronization operations in the previous round of data synchronization in the historical record.

4. The method according to claim 1, wherein, After updating the configuration data in the CMDB database according to the data to be synchronized that has passed the correctness verification and integrity verification, it further includes: When the data synchronization is completed, feedback a data synchronization message in a preset agreed format according to the data content corresponding to the data to be synchronized.

5. The method according to claim 1, wherein, Before obtaining the business data uploaded in the preset CMDB synchronization framework platform, it further includes: Construct a first KAFKA channel between the business application database corresponding to the business data and the professional group, and a second KAFKA channel between the professional group and the CMDB database; Unify the configuration of the proxy addresses of the first KAFKA channel and the second KAFKA channel.

6. A data synchronization device, wherein, The device includes: A data acquisition module, configured to acquire the business data uploaded in a preset CMDB synchronization framework platform. The preset CMDB synchronization framework platform includes professional groups. The service databases of the professional groups are constructed based on the entity dependencies within the professional groups, and are respectively connected to the business application database and the CMDB database. The preset CMDB synchronization framework is used to acquire the business data from the business application database and synchronize the data to be synchronized in the business data to the CMDB database; A synchronized data screening module, configured to perform Quartz periodic scheduling logic determination on the business data to determine the data to be synchronized in the business data; A data verification module, configured to query the unique constraint key corresponding to the data to be synchronized; perform DEL processing on the data to be synchronized according to the unique constraint key, and record the DEL status; verify the correctness and integrity of the data to be synchronized according to the DEL status and the convention of the unique constraint key; A data synchronization module, configured to update the configuration data in the CMDB database according to the data to be synchronized that has passed the correctness verification and integrity verification.

7. The apparatus according to claim 6, wherein, it further includes a dependency construction module, configured to: acquire the entities and entity attributes in the professional groups in the preset CMDB synchronization framework platform; acquire the association relationships between the entities according to the entity attributes; splice the entities according to the association relationships to construct the entity dependencies within the professional groups.

8. The apparatus according to claim 6, wherein, it further includes a channel construction module, configured to: construct a first KAFKA channel between the business data corresponding business application database and the professional groups, and a second KAFKA channel between the professional groups and the CMDB database; uniformly configure the proxy addresses of the first KAFKA channel and the second KAFKA channel.

9. A computer device, including a memory and a processor, where the memory stores a computer program, wherein, when the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images