Database comparison method and device, electronic equipment and storage medium

Abstract

The invention discloses a database comparison method and device, electronic equipment and a storage medium, and relates to the technical field of big data or other related technical fields, and the method comprises the steps: determining a database to be compared, obtaining the attribute information of the database, and constructing a database list based on the database and the attribute information of the database; creating a comparison task for a database in the database list based on the attribute information; establishing connection with a target database based on the comparison task and executing the comparison task to obtain a comparison result; and performing structured storage on the comparison result, and configuring an index for the stored comparison result. According to the method and the device, the technical problem that a comparison result is inaccurate due to a writing error because a technician depends on manually writing an SQL script or executing a command to define a database comparison rule and carry out database comparison in related technologies is solved.

Description

Technical Field

[0001] This invention relates to the field of big data technology or other related technical fields. Specifically, it relates to a database comparison method and apparatus, electronic device and storage medium. Background Technology

[0002] With the advancement of the information technology application innovation (IT innovation) strategy, many key industries, such as aviation and finance, are accelerating their migration from traditional non-IT innovation databases to IT innovation databases. In this process, structural comparison and data consistency verification between databases have become crucial steps to ensure successful migration and business continuity.

[0003] In related technologies, it is necessary for technicians to manually write SQL scripts or execute commands to define database comparison rules for database comparison. This places high demands on technicians, which not only increases the learning and operation costs, but also makes it easy for errors to occur, resulting in inaccurate database comparison results and affecting the normal operation of the database.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a database comparison method, apparatus, electronic device, and storage medium to at least solve the technical problem in related technologies where technical personnel rely on manually writing SQL scripts or executing commands to define database comparison rules and perform database comparisons, which can easily lead to inaccurate comparison results due to writing errors.

[0006] According to one aspect of the present invention, a database comparison method is provided, comprising: determining a database to be compared and obtaining attribute information of the database; constructing a database list based on the database and the attribute information of the database; creating a comparison task for the databases in the database list based on the attribute information; establishing a connection with a target database based on the comparison task and executing the comparison task to obtain a comparison result; storing the comparison result in a structured manner and configuring an index for the stored comparison result.

[0007] Further, the step of creating a comparison task for the databases in the database list based on the attribute information includes: extracting the airline information, database type, database interface address, and field information stored in the database corresponding to the database based on the attribute information; configuring the comparison type, data synchronization requirements, and scheduled execution strategy for the database based on the airline information and the database type; configuring the connection strategy for the database based on the database interface address; configuring the comparison strategy for the database based on the field information stored in the database; and creating a comparison task for the database based on the comparison type, the data synchronization requirements, the scheduled execution strategy, the connection strategy, and the comparison strategy.

[0008] Further, the comparison task includes at least one of the following: a pre-file generation subtask, a database connection subtask, and a database comparison main task. The step of creating a comparison task for the database based on the comparison type, the data synchronization requirements, the scheduled execution strategy, the connection strategy, and the comparison strategy includes: creating the pre-file generation subtask based on the comparison type, the scheduled execution strategy, and the comparison strategy, wherein the pre-file generation subtask is used to generate a pre-file, which represents a configuration file required for performing the database comparison; creating the database connection subtask based on the connection strategy; and creating the database comparison main task based on the pre-file and the data synchronization requirements.

[0009] Furthermore, after creating the database comparison master task based on the pre-existing files and the data synchronization requirements, the process further includes: scheduling the pre-existing file generation subtask, the database connection subtask, and the database comparison master task using a chain of responsibility pattern, wherein the completion of the pre-existing file generation subtask is the trigger condition for executing the database connection subtask, and the completion of the database connection subtask is the trigger condition for executing the database comparison master task.

[0010] Furthermore, after creating a comparison task for the databases in the database list based on the attribute information, the method further includes: adding the database comparison main task to the task queue; and configuring the maximum number of concurrent tasks for the task queue using the token bucket algorithm.

[0011] Furthermore, after establishing a connection with the target database based on the comparison task and executing the comparison task, the method further includes: acquiring the database comparison execution log in real time; parsing the database comparison execution log to obtain comparison indicator data; and displaying the comparison indicator data through a visualization interface.

[0012] Furthermore, after obtaining the comparison result, the method further includes: if the comparison result is abnormal, calling the operation and maintenance terminal configuration list; extracting the target receiving address from the operation and maintenance terminal configuration list based on the database identifier of the target database; generating alarm information for the abnormal comparison result, and sending the alarm information to the target receiving address.

[0013] According to another aspect of the present invention, a database comparison apparatus is also provided, comprising: an acquisition unit, configured to determine a database to be compared, acquire attribute information of the database, and construct a database list based on the database and the attribute information of the database; a creation unit, configured to create a comparison task for the databases in the database list based on the attribute information; an execution unit, configured to establish a connection with the target database based on the comparison task and execute the comparison task to obtain a comparison result; and a storage unit, configured to perform structured storage of the comparison result and configure an index for the stored comparison result.

[0014] Further, the acquisition unit includes: a first extraction module, used to extract the airline information, database type, database interface address, and field information stored in the database corresponding to the database based on the attribute information; a first configuration module, used to configure the comparison type, data synchronization requirements, and scheduled execution strategy for the database based on the airline information and the database type; a second configuration module, used to configure the connection strategy for the database based on the database interface address; a third configuration module, used to configure the comparison strategy for the database based on the field information stored in the database; and a first creation module, used to create a comparison task for the database based on the comparison type, the data synchronization requirements, the scheduled execution strategy, the connection strategy, and the comparison strategy.

[0015] Further, the comparison task includes at least one of the following: a pre-file generation subtask, a database connection subtask, and a database comparison main task. The first creation module includes: a first creation submodule, used to create the pre-file generation subtask based on the comparison type, the timing execution strategy, and the comparison strategy, wherein the pre-file generation subtask is used to generate a pre-file, which represents the configuration file required to perform the database comparison; a second creation submodule, used to create the database connection subtask based on the connection strategy; and a third creation submodule, used to create the database comparison main task based on the pre-file and the data synchronization requirements.

[0016] Furthermore, the first creation module also includes: a first scheduling submodule, used to schedule the pre-file generation subtask, the database connection subtask, and the database comparison main task using a chain of responsibility pattern, wherein the completion of the pre-file generation subtask is the trigger condition for executing the database connection subtask, and the completion of the database connection subtask is the trigger condition for executing the database comparison main task.

[0017] Furthermore, the database comparison device further includes: a first addition module, used to add the database comparison main task to the task queue; and a fourth configuration module, used to configure the maximum number of concurrent tasks for the task queue using the token bucket algorithm.

[0018] Furthermore, the database comparison device further includes: a first acquisition module, used to acquire database comparison execution logs in real time; and a first parsing module, used to parse the database comparison execution logs to obtain comparison indicator data, and to display the comparison indicator data through a visual interface.

[0019] Furthermore, the database comparison device further includes: a first invocation module, used to invoke the operation and maintenance terminal configuration list when the comparison result is abnormal; a second extraction module, used to extract the target receiving address from the operation and maintenance terminal configuration list based on the database identifier of the target database; and a first generation module, used to generate alarm information for the abnormal comparison result and send the alarm information to the target receiving address.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described database comparison methods.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described database comparison methods.

[0022] According to another aspect of the present invention, a computer program product is also provided, the computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements any of the above-described database comparison methods.

[0023] In this application, the following steps are performed: determining the database to be compared and obtaining the attribute information of the database; constructing a database list based on the database and the attribute information of the database; creating a comparison task for the database in the database list based on the attribute information; establishing a connection with the target database based on the comparison task and executing the comparison task to obtain the comparison result; storing the comparison result in a structured manner and configuring an index for the stored comparison result.

[0024] In this application, a database list is constructed based on the attribute information of all databases to be compared, and a comparison task is created through a configuration interface. The comparison task is then executed to complete the database comparison. This decouples the underlying parameters required for the comparison task from the upper-level execution logic, improving the flexibility of task creation. It eliminates the need to manually write SQL scripts or execute commands, reducing the risk of errors and improving the accuracy and reliability of database comparison. This solves the technical problem in related technologies where technicians rely on manually writing SQL scripts or executing commands to define database comparison rules and perform database comparisons, which can easily lead to inaccurate comparison results due to writing errors. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0026] Figure 1 This is a flowchart of an optional database comparison method according to an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of an optional configuration process according to an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of an optional comparison task management process according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of an optional database comparison device according to an embodiment of the present invention;

[0030] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for performing a database comparison method according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] It should be noted that the database comparison method and apparatus in this application can be used in the field of big data technology for data comparison of databases, and can also be used in any field other than big data technology for data comparison of databases. This application does not limit the application field of the database comparison method and apparatus.

[0034] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of such data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0035] It should be noted that in this application, when collecting and analyzing customer information, users are provided with corresponding operation entry points to choose whether to agree to or reject the automated decision-making results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0036] The following embodiments of the present invention can be applied to various database comparison systems / applications / devices. The present invention achieves full-process coverage of configuration management, job scheduling, historical tracing, and task maintenance through modularization. By automatically executing database comparisons through the creation of comparison tasks, it decouples the underlying parameters of the comparison tasks from the upper-level execution logic, improving the flexibility and reusability of task creation, and enhancing the accuracy and reliability of comparisons.

[0037] The present invention will now be described in detail with reference to various embodiments.

[0038] Example 1

[0039] According to an embodiment of the present invention, an embodiment of a database comparison method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0040] Figure 1 This is a flowchart of an optional database comparison method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0041] With the advancement of the domestic IT innovation strategy, key industries such as civil aviation are accelerating their migration from existing non-domestic IT innovation databases to domestic IT innovation databases. Database comparison is a necessary means to verify data integrity and consistency during the migration process, ensuring that data is accurately transferred from the source to the target database. This involves complex operations such as field mapping and type conversion, making comparison the last line of defense for verifying the success of the migration. Non-domestic IT innovation databases and domestic IT innovation databases may differ in design and architecture. Through database comparison, these structural differences can be identified and recorded, including table structure, indexes, and constraints. This helps to make necessary structural adjustments to the source database before migration to adapt to the requirements of the target database and ensure the consistency of the data model. During data synchronization or data integration, database comparison is used to verify data consistency, that is, whether the data of the same records in the source and target databases are completely identical. This is crucial for data quality management, especially in fields such as finance and aviation where data accuracy requirements are extremely high; any slight data difference can affect the normal operation of business.

[0042] In this embodiment of the invention, a modular interface design breaks down the comparison process into four steps: configuration, execution, monitoring, and traceability, with each step exposing only necessary parameters. The backend service provides standardized comparison interfaces (such as structural difference detection and full / incremental data synchronization), while the frontend automatically generates interface parameters and creates execution tasks through visual components (such as tree table selectors and difference configuration wizards) to perform the comparison operation.

[0043] Step S101: Determine the database to be compared and obtain the attribute information of the database, and construct a database list based on the database and the attribute information of the database.

[0044] In step S101 above, all databases to be compared are first identified, including traditional databases in non-IT-based environments and new databases in IT-based environments. By scanning database resources in the system, key attribute information is extracted, such as database type, version, location, and connection parameters (including hostname, port, database name, username, and password). Based on the acquired attribute information, a comprehensive database list is constructed. This list not only includes the database's metadata but also integrates business information associated with the database, such as the airline and business domain. This database list will serve as the basis for subsequent comparison task configuration and execution.

[0045] By constructing a database list, the system can centrally manage and monitor database resources, facilitating subsequent comparison job scheduling and execution. At the same time, the attribute information in the list provides the necessary parameters for configuring the comparison task, ensuring the smooth progress of the comparison process.

[0046] Step S102: Create a comparison task for the databases in the database list based on the attribute information.

[0047] In step S102 above, a customized comparison task is created for each database in the list to ensure that the comparison process can be precisely configured according to the specific situation of the database. The comparison task is automatically created based on attribute information. The comparison task includes all necessary execution details, such as the database connection string, the list of tables and fields to be compared, and the execution time, to prepare for subsequent database comparison execution steps.

[0048] Furthermore, the steps for creating a comparison task for the databases in the database list based on attribute information include: extracting the airline information, database type, database interface address, and field information stored in the database based on the attribute information; configuring the comparison type, data synchronization requirements, and scheduled execution strategy for the database based on the airline information and database type; configuring the connection strategy for the database based on the database interface address; configuring the comparison strategy for the database based on the field information stored in the database; and creating a comparison task for the database based on the comparison type, data synchronization requirements, scheduled execution strategy, connection strategy, and comparison strategy.

[0049] Specifically, the system automatically parses the attribute information of each database, including airline information (i.e., the airline to which the database belongs), database type (e.g., non-IT-compliant database or IT-compliant database), database interface address, and field information stored in the database. This information together constitutes the basic parameters for the comparison task.

[0050] Based on airline information and database type, the system automatically generates preset comparison types (such as structural comparison and data comparison), data synchronization requirements (whether data synchronization is needed), and scheduled execution strategies (such as executing at 2 AM every day). This configuration process aims to ensure that the comparison operation can match the specific business scenario and database characteristics, achieving efficient and accurate comparison goals.

[0051] Based on the database interface address, the system automatically configures the database connection policy to ensure that the comparison tool can connect to the target database correctly and securely. The connection policy includes authentication methods, connection timeouts, and reconnection mechanisms, aiming to optimize database access efficiency while ensuring data security.

[0052] Furthermore, based on the field information stored in the database, personalized field-level comparison strategies are set. This includes field mapping, data type conversion rules, and comparison algorithms (such as precise comparison or fuzzy comparison), ensuring that the comparison task can be detailed down to the level of each field, thereby improving the depth and accuracy of the comparison.

[0053] Ultimately, the system integrates comparison types, data synchronization requirements, scheduled execution strategies, connection strategies, and field-level comparison strategies into complete comparison task parameters. Users can confirm and start these comparison tasks through a graphical interface or API. Each task will execute the comparison process according to customized parameters, achieving automated and intelligent database comparison operations.

[0054] By automatically extracting key attribute information from the database, the configuration of comparison tasks is automated, improving operational efficiency and accuracy. Customized settings for comparison types, data synchronization requirements, timing strategies, and connection strategies ensure that comparison tasks can adapt to the unique needs of different databases, achieving more accurate comparison results that better suit business scenarios.

[0055] Furthermore, the comparison task includes at least one of the following: a pre-file generation subtask, a database connection subtask, and a database comparison main task. The steps of creating a database comparison task based on the comparison type, data synchronization requirements, scheduled execution strategy, connection strategy, and comparison strategy include: creating a pre-file generation subtask based on the comparison type, scheduled execution strategy, and comparison strategy, wherein the pre-file generation subtask is used to generate pre-files, which represent the configuration files required to perform the database comparison; creating a database connection subtask based on the connection strategy; and creating a database comparison main task based on the pre-files and data synchronization requirements.

[0056] Specifically, to achieve efficient and automated database comparison, this embodiment of the invention subdivides the comparison task into at least one of the following subtasks: a pre-file generation subtask, a database connection subtask, and a main database comparison task. Based on the comparison type and strategy, the system automatically creates the pre-file generation subtask. The purpose of this subtask is to generate the pre-file, which is essentially a configuration file required for performing the database comparison, containing key information such as comparison rules and field mappings. Simultaneously, the system ensures that the pre-file is created at the appropriate time according to a scheduled execution strategy, such as "execute at 2 AM every day," to support subsequent comparison operations.

[0057] Based on the connection strategy, a database connection subtask is created. This subtask is responsible for establishing a connection with the database, including handling the database authentication method, connection timeout settings, and reconnection mechanisms, to ensure that the comparison tool can access the target database stably and securely, providing reliable connection support for the execution of the comparison job.

[0058] After the pre-file generation subtask and database connection subtask are completed, the system creates the database comparison main task based on the configuration information and data synchronization requirements in the pre-file. The main task will use the comparison rules in the pre-file to perform comparison operations at the structural or data level.

[0059] This invention ensures the continuity and efficiency of the comparison process by controlling the execution order of the pre-processing file generation subtask, the database connection subtask, and the main database comparison task. The pre-processing file generation subtask and the database connection subtask serve as prerequisites for the main comparison task; once completed, they automatically trigger the execution of the main comparison task, thus achieving process automation and logical dependencies between tasks.

[0060] Furthermore, after creating the main database comparison task based on the requirements of pre-file and data synchronization, the process also includes: scheduling the pre-file generation subtask, the database connection subtask, and the main database comparison task using the chain of responsibility pattern. The completion of the pre-file generation subtask is the trigger condition for executing the database connection subtask, and the completion of the database connection subtask is the trigger condition for executing the main database comparison task.

[0061] Specifically, after creating the main database comparison task, the system further employs the chain of responsibility pattern to schedule and coordinate the pre-processor file generation subtask, the database connection subtask, and the main database comparison task. The chain of responsibility pattern links request handlers into a chain, with the request passed along the chain until a handler successfully processes it. In this context, it ensures the correct execution order and interdependencies between tasks. First, the system schedules the execution of the pre-processor file generation subtask. This task generates standardized pre-processor files required for database comparison based on the database's attribute information and comparison strategy. These pre-processor files contain detailed comparison rules, field mappings, database connection information, and other key parameters, serving as a guide for subsequent comparison operations.

[0062] After the pre-file generation subtask is completed and successfully generates the pre-file, the database connection subtask is triggered. This task attempts to establish a connection with the target database according to the connection strategy in the pre-file. The connection process includes handling authentication information, connection timeout settings, and reconnection mechanisms to ensure that the database comparison can run in a stable and secure connection environment.

[0063] Once the database connection subtask is completed and the database connection is successfully established, the system automatically schedules the execution of the main database comparison task. Based on the comparison rules and database connection information in the pre-file, the main task begins comparing the databases for structural or data-level differences. The comparison process is adjusted according to data synchronization requirements (such as real-time or scheduled synchronization) to ensure the accuracy and timeliness of the comparison results.

[0064] By adopting the chain of responsibility model, the system ensures the correct execution order among the pre-file generation subtask, the database connection subtask, and the main database comparison task. The completion of the pre-file generation subtask serves as a condition for triggering the database connection subtask, while the completion of the database connection subtask serves as a prerequisite for executing the main database comparison task. This decouples the underlying operations of the comparison task (such as parameter configuration and communication connection) from the upper-level execution logic, thereby improving the flexibility and intelligence of task execution.

[0065] Furthermore, after creating comparison tasks for the databases in the database list based on attribute information, the process also includes: adding the main database comparison task to the task queue; and configuring the maximum number of concurrent tasks for the task queue using the token bucket algorithm.

[0066] This invention optimizes the execution efficiency of comparison tasks and system resource management through task queue management and concurrency control mechanisms. Specifically, after a comparison master task is created, the system automatically adds these tasks to a predefined task queue. The task queue is a first-in, first-out (FIFO) data structure used to store the comparison master tasks to be executed, ensuring that tasks are scheduled and executed in the order they are created. This step provides the foundation for subsequent task scheduling, enabling the comparison tasks to be managed uniformly by the system and avoiding conflicts or disordered order between tasks. To control the number of concurrent executions of comparison tasks, the system uses a token bucket algorithm to limit the number of tasks that can be executed simultaneously in the task queue. By setting a token bucket, a certain number of tokens are contained in the bucket, and each comparison task needs to consume one token before execution. The generation rate of tokens in the bucket determines the number of tasks that the system can execute in parallel. Through this mechanism, the execution efficiency of comparison tasks can be balanced with the stable utilization rate of system resources, avoiding system overload or response delays caused by too many concurrent tasks.

[0067] Step S103: Establish a connection with the target database based on the comparison task and execute the comparison task to obtain the comparison results.

[0068] In step S103 above, during the comparison task, the pre-file generation subtask and the database connection subtask are executed sequentially to configure comparison parameters and establish a connection with the target database to be compared. Then, the comparison logic is executed according to the comparison strategy in the comparison task. The comparison strategy covers the type of comparison (such as structural comparison, data consistency check), field mapping rules, data synchronization requirements, etc. Based on these strategies, the system compares the table structure, field types, and data values ​​in the database one by one to identify the differences between the source database and the target database.

[0069] During the comparison process, the system collects comparison results in real time, including the number of consistent and inconsistent items, specific differences, and other information. The comparison results are not limited to the final output but also include intermediate states during the comparison process, facilitating subsequent troubleshooting and result traceability. Furthermore, the system performs a preliminary analysis of the comparison results, marking key differences to help users quickly pinpoint the problem.

[0070] After the comparison is completed, the system will feed back the comparison results to the front-end user interface. Users can view the comparison results intuitively through a visual interface, including charts, statistical data, and other display formats, making it easy to understand and interpret. At the same time, the system will store the comparison results in the history record for convenient subsequent auditing, analysis, and re-comparison.

[0071] Furthermore, after establishing a connection with the target database based on the comparison task and executing the comparison task, the process also includes: obtaining the database comparison execution log in real time; parsing the database comparison execution log to obtain comparison indicator data; and displaying the comparison indicator data through a visual interface.

[0072] Specifically, during the data comparison process, the system closely interacts with the target database through backend services to capture all log information in real time during the comparison task execution. These logs not only include detailed comparison process information, such as the number of reads and writes, and comparison speed, but also cover any abnormal events, such as connection failures and data inconsistency alarms. Acquiring these real-time logs is fundamental for subsequent analysis of comparison results and tracking of execution status. The acquired log information is then parsed to extract key comparison metrics, such as the number of consistent entries, new entries, deleted entries, and updated entries. These metrics reflect the progress and effectiveness of the database comparison execution, providing data support for subsequent analysis and visualization. The extracted comparison metrics are then displayed in real time on the front-end visualization interface. The system uses a high-performance chart library to construct a dynamically updated comparison execution progress bar, statistical indicator dashboard, and anomaly pop-up prompts. Users can intuitively see the real-time progress of the comparison, changes in key indicators, and immediate notifications of any abnormal situations, improving user awareness of the task execution status and facilitating rapid decision-making and problem handling.

[0073] Step S104: The comparison results are stored in a structured manner, and an index is configured for the stored comparison results.

[0074] In step S104 above, the comparison results are converted into a structured data format to ensure data readability and standardization. The structuring process involves classifying and labeling various indicators in the comparison results (such as the number of consistent entries, the number of new entries, the number of deleted entries, and the number of updated entries) to form an easily processed data structure, facilitating subsequent data storage and retrieval. A relational database is used as the storage medium, and the most suitable data model is selected based on the nature of the results (such as time series or spatial relationships). This strategy ensures long-term preservation and efficient access to the comparison results.

[0075] To improve the efficiency of retrieving comparison results, the system configures indexes for key fields during storage. Index selection is based on the query pattern of the results (e.g., by time, by difference type, by database type) to ensure rapid location of frequently queried fields. The index configuration process includes analyzing the access frequency of fields, assessing the impact of the index on storage space, and determining the index type. Stored comparison results may include comparison timestamps, database names, comparison types, and inconsistent data content. This information aids in subsequent auditing and problem tracing, ensuring the transparency and auditability of the data migration process.

[0076] Furthermore, after obtaining the comparison results, the process also includes: in the event of an anomaly in the comparison results, calling the operation and maintenance terminal configuration list; extracting the target receiving address from the operation and maintenance terminal configuration list based on the database identifier of the target database; generating alarm information for the anomaly in the comparison results, and sending the alarm information to the target receiving address.

[0077] Specifically, the system first performs anomaly detection on the comparison results to identify unexpected anomalies such as inconsistent data, connection failures, and execution errors. Anomaly detection is based on preset comparison standards and rules, automatically identifying and marking anomaly entries. If an anomaly is detected, the system automatically calls a pre-configured list of maintenance terminals. This list contains the mailing addresses of maintenance personnel for each target database, ensuring that alarm information is accurately sent to the responsible parties. The system precisely matches and extracts the corresponding target mailing address from the maintenance terminal configuration list based on the unique identifier of the target database (such as database name or ID). This process ensures that alarm information is sent to the maintenance personnel responsible for that database, improving the targeting and efficiency of alarm response. For each anomaly, the system automatically generates detailed alarm information, including the anomaly type, occurrence time, database identifier, and specific differences. The alarm information is then sent via email to the corresponding target mailing address, ensuring that maintenance personnel receive timely notification and take appropriate measures to handle the anomaly.

[0078] Through the above steps, the database to be compared is determined, and the attribute information of the database is obtained. A database list is constructed based on the database and its attribute information. A comparison task is created for the databases in the database list based on the attribute information. A connection is established with the target database based on the comparison task and the comparison task is executed to obtain the comparison result. The comparison result is stored in a structured manner, and an index is configured for the stored comparison result.

[0079] In this embodiment, a database list is constructed based on the attribute information of all databases to be compared. A comparison task is created through a configuration interface, and the comparison task is executed to complete the database comparison. This decouples the underlying parameters required for the comparison task from the upper-level execution logic, improving the flexibility of task creation. It eliminates the need to manually write SQL scripts or execute commands, reducing the risk of errors and improving the accuracy and reliability of database comparison. This solves the technical problem in related technologies where technicians rely on manually writing SQL scripts or executing commands to define database comparison rules and perform database comparisons, which is prone to inaccurate comparison results due to writing errors.

[0080] The following describes in detail another optional implementation method.

[0081] The database comparison job management page integrates the entire process management of comparison tasks, including creation, execution, monitoring, and feedback. It is the core interface for user operation and aims to reduce operational complexity and improve task execution efficiency and result observability through visualization. Figure 2 This is a schematic diagram of an optional configuration process according to an embodiment of the present invention, such as... Figure 2 As shown, the configuration process specifically includes:

[0082] Step 1: The operations and maintenance personnel log in to the configuration interface;

[0083] Step 2, address information configuration;

[0084] After logging into the system, operations and maintenance personnel can access the Configuration Management - Address Information page and view the configured backend API addresses through the table component. Clicking the "Add" button will bring up a form; users can enter the address, port, and protocol type and submit to enable data interaction between the frontend and backend.

[0085] Step 3, configure the recipient list;

[0086] Configure recipients for each database through the Configuration Management - Recipient List page, and build a recipient list based on the database and its corresponding recipients (corresponding to the above-mentioned terminal configuration list). The recipient list can promptly feed back the comparison results to the person in charge of managing the database after obtaining the comparison results.

[0087] Step 4: Configure database information.

[0088] Technical personnel can access the Configuration Management - Table Information Management page, click the "Add" button to bring up a form, enter the database, database identifier, database type, and select whether to generate a comparison pre-file.

[0089] Obtain the attribute information of each database and create a database list based on the database and its attribute information.

[0090] Figure 3 This is a schematic diagram of an optional comparison task management process according to an embodiment of the present invention, such as... Figure 3 As shown, the comparison task management process includes:

[0091] Step 1: Compare parameter configurations;

[0092] Select the comparison type through the tabs on the job management page. Click the "Add" button to bring up a pop-up window, where you can select the backend API address, airline, whether to synchronize data, and the timing strategy (e.g., "Execute at 2 AM every day"). The system will dynamically load the corresponding configuration parameters. The new pop-up window integrates a cascading selector, enabling multiple airline selections and API address-linked selection.

[0093] Step 2: Create a comparison task;

[0094] When a table comparison type is selected and the generation of prerequisite files is configured, the system automatically creates a chain of subtasks. The chain of subtasks is managed using a chain of responsibility model. The main task (comparison) depends on the completion of the subtasks (file generation), and the main task is triggered upon completion of the subtasks. Personnel can view the status of subtasks in the task list (e.g., "WAITING" or "COMPLETE").

[0095] Step 3: Perform the comparison task;

[0096] During task execution, comparison metrics (number of reads, number of writtens, number of inconsistent entries, number of deleted entries, and number of updated entries) are dynamically updated through a visual interface. Execution logs are pushed to the front end in real time, and a fragmented transmission strategy is used to optimize network performance.

[0097] Personnel can select tasks in batches, click the batch execute or batch modify status button, set the maximum number of concurrent tasks, and then trigger the operation.

[0098] The token bucket algorithm is used to limit the maximum number of concurrent tasks, and task scheduling synchronization in a cluster environment is achieved through distributed locks.

[0099] Step 4: Store the comparison results.

[0100] The comparison results are stored in a structured format, including execution timestamps, airline codes, comparison types, and difference statistics, and indexes are configured for the stored comparison results. Personnel can access the historical task management page to query historical tasks by index (such as time range or execution status) and click on the task row to view detailed differences.

[0101] This invention achieves full-process coverage of configuration management, job scheduling, historical tracing, and task maintenance through modularization. By creating comparison tasks and automatically performing database comparisons, the underlying parameters of the comparison tasks are decoupled from the upper-level execution logic, improving the flexibility and reusability of task creation, and enhancing the accuracy and reliability of comparisons.

[0102] The following is a detailed description with reference to another embodiment.

[0103] Example 2

[0104] The database comparison device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in the above embodiment one. The specific implementation method and beneficial effects can be referred to the foregoing method embodiment, and will not be repeated here.

[0105] Figure 4 This is a schematic diagram of an optional database comparison device according to an embodiment of the present invention, such as... Figure 4As shown, the database comparison device may include: an acquisition unit 41, a creation unit 42, an execution unit 43, and a storage unit 44, wherein,

[0106] The acquisition unit 41 is used to determine the database to be compared and to acquire the attribute information of the database, and to construct a database list based on the database and the attribute information of the database.

[0107] Create unit 42, which is used to create a comparison task for the databases in the database list based on attribute information;

[0108] Execution unit 43 is used to establish a connection with the target database based on the comparison task and execute the comparison task to obtain the comparison result;

[0109] Storage unit 44 is used to structure the comparison results and configure an index for the stored comparison results.

[0110] The aforementioned database comparison device determines the database to be compared by the acquisition unit 41 and acquires the attribute information of the database, and constructs a database list based on the database and the attribute information of the database; the creation unit 42 creates a comparison task for the databases in the database list based on the attribute information; the execution unit 43 establishes a connection with the target database based on the comparison task and executes the comparison task to obtain the comparison result; and the storage unit 44 performs structured storage of the comparison result and configures an index for the stored comparison result.

[0111] In this embodiment, a database list is constructed based on the attribute information of all databases to be compared. A comparison task is created through a configuration interface, and the comparison task is executed to complete the database comparison. This decouples the underlying parameters required for the comparison task from the upper-level execution logic, improving the flexibility of task creation. It eliminates the need to manually write SQL scripts or execute commands, reducing the risk of errors and improving the accuracy and reliability of database comparison. This solves the technical problem in related technologies where technicians rely on manually writing SQL scripts or executing commands to define database comparison rules and perform database comparisons, which is prone to inaccurate comparison results due to writing errors.

[0112] Furthermore, the acquisition unit includes: a first extraction module, used to extract the airline information, database type, database interface address, and field information stored in the database corresponding to the database based on attribute information; a first configuration module, used to configure the comparison type, data synchronization requirements, and scheduled execution strategy for the database based on the airline information and database type; a second configuration module, used to configure the connection strategy for the database based on the database interface address; a third configuration module, used to configure the comparison strategy for the database based on the field information stored in the database; and a first creation module, used to create a comparison task for the database based on the comparison type, data synchronization requirements, scheduled execution strategy, connection strategy, and comparison strategy.

[0113] Furthermore, the comparison task includes at least one of the following: a pre-file generation subtask, a database connection subtask, and a database comparison main task. The first creation module includes: a first creation submodule, used to create a pre-file generation subtask based on the comparison type, the scheduled execution strategy, and the comparison strategy, wherein the pre-file generation subtask is used to generate a pre-file, which represents the configuration file required to perform the database comparison; a second creation submodule, used to create a database connection subtask based on the connection strategy; and a third creation submodule, used to create a database comparison main task based on the pre-file and data synchronization requirements.

[0114] Furthermore, the first creation module also includes a first scheduling submodule, which is used to schedule the pre-file generation subtask, the database connection subtask, and the database comparison main task using the chain of responsibility pattern. The completion of the pre-file generation subtask is the trigger condition for executing the database connection subtask, and the completion of the database connection subtask is the trigger condition for executing the database comparison main task.

[0115] Furthermore, the database comparison device also includes: a first addition module, used to add the database comparison main task to the task queue; and a fourth configuration module, used to configure the maximum number of concurrent tasks for the task queue using the token bucket algorithm.

[0116] Furthermore, the database comparison device also includes: a first acquisition module, used to acquire database comparison execution logs in real time; and a first parsing module, used to parse the database comparison execution logs to obtain comparison indicator data, and to display the comparison indicator data through a visual interface.

[0117] Furthermore, the database comparison device also includes: a first calling module, used to call the operation and maintenance terminal configuration list when there is an anomaly in the comparison result; a second extraction module, used to extract the target receiving address from the operation and maintenance terminal configuration list based on the database identifier of the target database; and a first generating module, used to generate alarm information for the anomaly in the comparison result and send the alarm information to the target receiving address.

[0118] The aforementioned database comparison device may also include a processor and a memory. The aforementioned acquisition unit 41, creation unit 42, execution unit 43, storage unit 44, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0119] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the database comparison is performed by adjusting kernel parameters.

[0120] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0121] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute any of the above-described database comparison methods.

[0122] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described database comparison methods.

[0123] According to another aspect of the present invention, a computer program product is also provided, the computer program product including a computer program, wherein the computer program, when executed by a processor, implements any of the above-described database comparison methods.

[0124] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: determining the database to be compared and obtaining the attribute information of the database; constructing a database list based on the database and the attribute information of the database; creating a comparison task for the databases in the database list based on the attribute information; establishing a connection with the target database based on the comparison task and executing the comparison task to obtain the comparison result; storing the comparison result in a structured manner and configuring an index for the stored comparison result.

[0125] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: extracting airline information, database type, database interface address, and field information stored in the database based on attribute information; configuring a comparison type, data synchronization requirements, and scheduled execution strategy for the database based on the airline information and database type; configuring a connection strategy for the database based on the database interface address; configuring a comparison strategy for the database based on the field information stored in the database; and creating a comparison task for the database based on the comparison type, data synchronization requirements, scheduled execution strategy, connection strategy, and comparison strategy.

[0126] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: the comparison task includes at least one of the following: a pre-file generation subtask, a database connection subtask, and a database comparison main task; the step of creating a comparison task for the database based on the comparison type, data synchronization requirements, scheduled execution strategy, connection strategy, and comparison strategy includes: creating a pre-file generation subtask based on the comparison type, scheduled execution strategy, and comparison strategy, wherein the pre-file generation subtask is used to generate a pre-file, which represents a configuration file required for performing the database comparison; creating a database connection subtask based on the connection strategy; and creating a database comparison main task based on the pre-file and data synchronization requirements.

[0127] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: scheduling a pre-file generation subtask, a database connection subtask, and a database comparison main task using a chain of responsibility pattern, wherein the completion of the pre-file generation subtask is a trigger condition for executing the database connection subtask, and the completion of the database connection subtask is a trigger condition for executing the database comparison main task.

[0128] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: adding the database comparison main task to the task queue; and configuring the maximum number of concurrent tasks for the task queue using the token bucket algorithm.

[0129] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: real-time acquisition of database comparison execution logs; parsing of the database comparison execution logs to obtain comparison indicator data; and displaying the indicator data through a visual interface.

[0130] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: in the event of an anomaly in the comparison result, calling the maintenance terminal configuration list; extracting the target receiving address from the maintenance terminal configuration list based on the database identifier of the target database; generating alarm information for the anomaly in the comparison result, and sending the alarm information to the target receiving address.

[0131] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for performing a database comparison method according to an embodiment of the present invention. Figure 5 As shown, an electronic device may include one or more processors ( Figure 5The processor, denoted by 502a, 502b, ..., 502n, can include, but is not limited to, a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA), and a memory 504 for storing data. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports in the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0132] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0133] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0135] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0137] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0138] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A database comparison method, characterized in that, include: Identify the databases to be compared and obtain the attribute information of the databases, and construct a database list based on the databases and the attribute information of the databases; Based on the attribute information, create a comparison task for the databases in the database list; A connection is established with the target database based on the comparison task, and the comparison task is executed to obtain the comparison result; The comparison results are stored in a structured manner, and an index is configured for the stored comparison results.

2. The method according to claim 1, characterized in that, The steps for creating a comparison task for the databases in the database list based on the attribute information include: Based on the attribute information, extract the airline information, database type, database interface address, and field information stored in the database corresponding to the database; Based on the airline information, the database type is configured with the database comparison type, data synchronization requirements, and scheduled execution strategy; Configure a connection strategy for the database based on the database interface address; Configure a comparison strategy for the database based on the field information stored in the database; A comparison task is created for the database based on the comparison type, the data synchronization requirements, the timed execution strategy, the connection strategy, and the comparison strategy.

3. The method according to claim 2, characterized in that, The comparison task includes at least one of the following: a pre-file generation subtask, a database connection subtask, and a database comparison main task. The step of creating a comparison task for the database based on the comparison type, the data synchronization requirements, the scheduled execution strategy, the connection strategy, and the comparison strategy includes: Based on the comparison type, the timed execution strategy, and the comparison strategy, a pre-file generation subtask is created, wherein the pre-file generation subtask is used to generate a pre-file, which represents the configuration file required to perform the database comparison; The database connection subtask is created based on the connection strategy; The database comparison master task is created based on the aforementioned prerequisite files and the aforementioned data synchronization requirements.

4. The method according to claim 3, characterized in that, After creating the database comparison master task based on the prerequisite files and the data synchronization requirements, the following steps are also included: The chain of responsibility pattern is used to schedule the pre-file generation subtask, the database connection subtask, and the database comparison main task. The completion of the pre-file generation subtask is the trigger condition for executing the database connection subtask, and the completion of the database connection subtask is the trigger condition for executing the database comparison main task.

5. The method according to claim 3, characterized in that, After creating a comparison task for the databases in the database list based on the attribute information, the process further includes: Add the database comparison main task to the task queue; The token bucket algorithm is used to configure the maximum number of concurrent tasks for the task queue.

6. The method according to claim 1, characterized in that, After establishing a connection with the target database based on the comparison task and executing the comparison task, the process further includes: Real-time acquisition of database comparison execution logs; The database comparison execution log is parsed to obtain comparison indicator data, which is then displayed through a visual interface.

7. The method according to claim 1, characterized in that, After obtaining the comparison results, the following is also included: If the comparison result is abnormal, the operation and maintenance terminal configuration list is invoked. The target receiving address is extracted from the operation and maintenance terminal configuration list based on the database identifier of the target database; An alarm message is generated for any abnormal comparison results, and the alarm message is sent to the target receiving address.

8. A database comparison device, characterized in that, include: An acquisition unit is used to determine the database to be compared, acquire the attribute information of the database, and construct a database list based on the database and the attribute information of the database. A creation unit is used to create a comparison task for the databases in the database list based on the attribute information; An execution unit is used to establish a connection with the target database based on the comparison task and execute the comparison task to obtain the comparison result; A storage unit is used to store the alignment results in a structured manner and to configure an index for the stored alignment results.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the database comparison method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the database comparison method according to any one of claims 1 to 7.

Images