Data comparison method and device, equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of data processing and the field of financial science and technology, and relates to a data comparison method which comprises the following steps: receiving a data comparison instruction carrying a command line parameter combination; determining a target object set according to the command line parameter combination; determining a to-be-compared target data table according to the target object set; executing a preset data comparison process according to the target data table to obtain difference information; and generating a data comparison result corresponding to the data comparison instruction according to the difference information. The invention further provides a data comparison device, computer equipment and a storage medium. Through the received parameters, the target object needing data comparison can be automatically selected, the corresponding data comparison process is executed, when the database is developed and maintained, the task of data comparison does not need to be completed through manual operation or tools, and therefore the accuracy of data comparison is guaranteed, and the data comparison efficiency is improved. And the data comparison efficiency is improved.

Description

Technical Field

[0001] The present application relates to the fields of data processing and financial technology, and is applied to scenarios of data comparison when developing and maintaining self-developed databases, and in particular to a data comparison method, device, equipment and storage medium. Background Art

[0002] With the development of computer technology, financial technology has been increasingly used in finance. For financial-related industries, databases, as core components of information systems, are of self-evident importance in terms of data security and privacy protection.

[0003] In the daily development and maintenance of self-developed databases, data verification is generally involved before and after product upgrades, data verification during backup and recovery, data verification during data migration, and data verification after data synchronization in the development and test environment. In these work scenarios, in order to ensure data consistency and avoid data loss or inconsistency, data comparison is an important and inevitable task.

[0004] Currently, the relevant operation and maintenance personnel can complete the task of data comparison by manually operating or using tools to compare the data of the two tables line by line. However, this method is time-consuming and labor-intensive, and the accuracy of the results cannot be guaranteed. When the scale of data to be compared is large, it faces serious performance bottlenecks, which greatly reduces the efficiency of data comparison. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a data comparison method, device, equipment and storage medium to solve the technical problem of low data comparison efficiency when developing and maintaining a self-developed database.

[0006] In order to solve the above technical problems, the present application embodiment provides a data comparison method, which adopts the following technical solution:

[0007] A data comparison method comprises the following steps:

[0008] Receiving a data comparison instruction carrying a command line parameter combination;

[0009] Determine a target object set according to the command line parameter combination;

[0010] Determining a target data table to be compared according to the target object set;

[0011] Execute a preset data comparison process according to the target data table to obtain difference information;

[0012] A data comparison result corresponding to the data comparison instruction is generated according to the difference information.

[0013] Further, the difference information includes structure difference information and data difference information, and the step of executing a preset data comparison process according to the target data table to obtain the difference information specifically includes:

[0014] Connect to the database instance corresponding to the target data table;

[0015] Acquire table structure information corresponding to the database instance, perform data comparison based on the table structure information, and obtain the structure difference information;

[0016] The data corresponding to the target data table is obtained from the database instance, and data comparison is performed based on the data to obtain the data difference information.

[0017] Furthermore, the step of obtaining the table structure information corresponding to the database instance, performing data comparison according to the table structure information, and obtaining the structure difference information specifically includes:

[0018] Querying the system table in the database instance to obtain definition information of the table in the database instance;

[0019] According to the definition information, a corresponding DDL statement is generated as the table structure information;

[0020] The DDL statements are compared to obtain the structural difference information.

[0021] Furthermore, the data corresponding to the target data table includes data sets of various preset sizes, and the step of obtaining the data corresponding to the target data table from the database instance, performing data comparison based on the data, and obtaining the data difference information specifically includes:

[0022] Reading each of the data sets from the database instance in batches, and comparing them in memory to obtain comparison results corresponding to each of the data sets;

[0023] The data difference information is generated according to the comparison result.

[0024] Furthermore, after the step of determining the target data table to be compared according to the target object set, the method further includes:

[0025] When executing the data comparison process, the process status information is recorded in real time;

[0026] When the data comparison process is monitored to be interrupted and reconnected, the remaining data that has not been compared is determined according to the process state information at the time of the interruption;

[0027] The data comparison process is completed based on the remaining data.

[0028] Furthermore, the step of generating a data comparison result corresponding to the data comparison instruction according to the difference information specifically includes:

[0029] Filling the difference information into a preset data comparison chart template to obtain a filled data comparison chart;

[0030] Data enhancement is performed on the data comparison chart to obtain the data comparison result.

[0031] Further, the target object set includes database objects, or database objects and schema objects, or database objects, schema objects and data table objects, and the step of determining the target data table to be compared based on the target object set specifically includes:

[0032] If the target object set includes a database object, determining a target database according to the database object, and using all data tables in the target database as the target data tables;

[0033] If the target object set includes database objects and schema objects, a target database is determined according to the database objects, a target schema is determined in the target database according to the schema objects, and all data tables in the target schema are used as the target data tables;

[0034] If the target object set includes database objects, schema objects and data table objects, the target database is determined according to the database objects, the target schema is determined in the target database according to the schema objects, and the target data table is determined in the target schema according to the data table objects.

[0035] In order to solve the above technical problems, the embodiment of the present application further provides a data comparison device, which adopts the following technical solution:

[0036] A data comparison device, comprising:

[0037] A receiving module, used for receiving a data comparison instruction carrying a command line parameter combination;

[0038] A first determination module, used to determine a target object set according to the command line parameter combination;

[0039] A second determination module, used to determine a target data table to be compared according to the target object set;

[0040] An execution module, used for executing a preset data comparison process according to the target data table to obtain difference information;

[0041] A generating module is used to generate a data comparison result corresponding to the data comparison instruction according to the difference information.

[0042] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0043] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data comparison method described above when executing the computer-readable instructions.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0045] A computer-readable storage medium stores computer-readable instructions, which implement the steps of the data comparison method described above when executed by a processor.

[0046] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0047] The data comparison method disclosed in the present application receives a data comparison instruction carrying a command line parameter combination; and determines a target object set based on the command line parameter combination; then determines a target data table to be compared based on the target object set; and then executes a preset data comparison process based on the target data table to obtain difference information; thereby generating a data comparison result corresponding to the data comparison instruction based on the difference information. The present application can automatically select the target object that needs to be compared with the data and execute the corresponding data comparison process through the received parameters. When developing and maintaining a self-developed database, there is no need for manual operation or the use of tools to complete the data comparison task, thereby ensuring the accuracy of the data comparison and improving the efficiency of the data comparison. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0050] Figure 2 is a flow chart of an embodiment of a data comparison method according to the present application;

[0051] Figure 3 is a structural schematic diagram of an embodiment of a data comparison device according to the present application;

[0052] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0054] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0055] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0056] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0057] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0058] The terminal devices 101, 102, and 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, and the like.

[0059] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0060] It should be noted that the data comparison method provided in the embodiment of the present application is generally executed by a terminal device, and accordingly, the data comparison device is generally arranged in the terminal device.

[0061] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0062] Continue to refer Figure 2 , shows a flow chart of an embodiment of a data comparison method according to the present application. The data comparison method comprises the following steps:

[0063] Step S201, receiving a data comparison instruction carrying a command line parameter combination.

[0064] In this embodiment, the electronic device (eg, Figure 1 The terminal device shown in the figure can send or receive data through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, Wi-Fi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (ultrawi deband) connection, and other wireless connection methods currently known or to be developed in the future.

[0065] It should be noted that with the acceleration of the digital transformation of the financial industry, the operating environment of related industries such as banking and insurance is undergoing tremendous changes. There are more and more new Internet business scenarios, and various online businesses pose a great challenge to the software and hardware capabilities of mid- and back-end databases. In this case, the self-developed databases independently developed and maintained by enterprises have advantages such as customization, performance optimization, and cost control, and have a wide range of application value in application scenarios such as large-scale data storage and processing, high-concurrency read and write operations, and real-time data analysis.

[0066] Specifically, in the process of developing and maintaining a database, when it comes to work scenarios that require data comparison, a data comparison instruction carrying a command line parameter combination can be received, the object that requires data comparison can be determined based on the command line parameter combination, and the subsequent data comparison process can be completed based on the data comparison instruction, wherein the data comparison instruction is an instruction sent by relevant R&D personnel or operation and maintenance personnel, and the command line parameter combination is a parameter input by the relevant R&D personnel or operation and maintenance personnel.

[0067] Step S202: determining a target object set according to the command line parameter combination.

[0068] Specifically, the target object set generally includes at least two target objects that need to be compared with data. Relevant R&D personnel or operation and maintenance personnel can specify objects of different dimensions as target objects for subsequent data comparison through the command line parameter combination carried by the data comparison instruction. After receiving the data comparison instruction, the command line parameter combination is parsed to determine the target object and form a target object set.

[0069] It can be understood that each database contains several schemas, and each schema contains several data tables. Databases, schemas, and data tables belong to objects of different dimensions. Therefore, the target object can be a database or a schema or a data table, that is, the target object set can include several databases, several schemas in several databases, or several data tables under several schemas in several databases.

[0070] Step S203: determining a target data table to be compared according to the target object set.

[0071] Specifically, since the target object set may include several databases, several schemas in several databases, or several data tables under several schemas in several databases, it can be understood that the target object set includes database objects, or database objects and schema objects, or database objects, schema objects and data table objects.

[0072] When the target object set includes database objects, the target database can be determined according to the database objects, and all data tables in the target database are used as target data tables to be compared;

[0073] When the target object set includes database objects and schema objects, the target database can be determined according to the database objects, the target schema can be determined in the target database according to the schema objects, and all data tables in the target schema can be used as target data tables to be compared;

[0074] When the target object set includes database objects, schema objects and data table objects, the target database can be determined according to the database objects, the target schema can be determined in the target database according to the schema objects, and the target data table to be compared can be determined in the target schema according to the data table objects.

[0075] Step S204, executing a preset data comparison process according to the target data table to obtain difference information.

[0076] Specifically, the difference information includes structural difference information and data difference information. After determining the target data table, the preset data comparison process can be executed. In the data comparison process, it is first necessary to connect to the database instance corresponding to the target data table, and then the table structure comparison subprocess and the table data comparison subprocess can be performed respectively, so as to obtain the structural difference information and the data difference information respectively. Among them, the table structure comparison subprocess includes: obtaining the table structure information corresponding to the database instance, performing data comparison according to the table structure information, and obtaining the structural difference information; the table data comparison subprocess includes: obtaining the data corresponding to the target data table from the database instance, performing data comparison according to the data corresponding to the target data table, and obtaining the data difference information. The specific implementation process of the above-mentioned table structure comparison subprocess and table data comparison subprocess will be further described in the subsequent specific embodiments, and will not be elaborated on here.

[0077] Step S205: generating a data comparison result corresponding to the data comparison instruction according to the difference information.

[0078] Specifically, generally speaking, in order to enable relevant R&D personnel or operation and maintenance personnel to intuitively determine the results of data comparison, they can be presented in the form of a chart. Therefore, a data comparison chart template is pre-set, and after obtaining the difference information, the difference information is filled into the preset data comparison chart template to obtain a filled-in data comparison chart. Finally, the data comparison chart is used as the data comparison result corresponding to the data comparison instruction.

[0079] This application can automatically select the target objects that need to be compared with the data and execute the corresponding data comparison process through the received parameters. When developing and maintaining the self-developed database, there is no need for manual operation or tools to complete the data comparison task, thereby ensuring the accuracy of the data comparison and improving the efficiency of the data comparison.

[0080] In some optional implementations of this embodiment, the difference information includes structure difference information and data difference information, and the step of executing a preset data comparison process according to the target data table to obtain the difference information includes:

[0081] Connect to the database instance corresponding to the target data table;

[0082] Acquire table structure information corresponding to the database instance, perform data comparison based on the table structure information, and obtain the structure difference information;

[0083] The data corresponding to the target data table is obtained from the database instance, and data comparison is performed based on the data to obtain the data difference information.

[0084] In this embodiment, after connecting to the database instance corresponding to the target data table, the structural difference information can be obtained through the table structure comparison sub-process, and the data difference information can be obtained through the table data comparison sub-process. Specifically, the specific implementation process of the table structure comparison sub-process includes: querying the system table in the database instance to obtain the definition information of the table in the database instance, and then generating the corresponding DDL statement as the table structure information according to the table definition information, so as to compare the DDL statement and obtain the structural difference information; the specific implementation process of the table data comparison sub-process includes: using a batch method to read each preset size of the data set from the database instance in turn, and compare them in the memory to obtain the comparison results corresponding to each data set, so as to generate the data difference information according to the comparison results corresponding to each data set.

[0085] This application automates the data comparison process. When data comparison is required during the development and maintenance of a self-developed database, the structure of the data table and the data can be compared at the same time, thereby improving the comprehensiveness of the data comparison and improving the efficiency of the data comparison.

[0086] In some optional implementations of this embodiment, the step of obtaining the table structure information corresponding to the database instance, performing data comparison according to the table structure information, and obtaining the structure difference information includes:

[0087] Querying the system table in the database instance to obtain definition information of the table in the database instance;

[0088] Generate a corresponding DDL statement according to the definition information as the table structure information;

[0089] The DDL statements are compared to obtain the structural difference information.

[0090] It should be noted that the system table is a special table in the database used to store database system information, usually including metadata information such as database structure, user permissions, and object definitions.

[0091] DDL statements, or Data Definition Language, are languages ​​used to describe real-world entities to be stored in a database. They are mainly used to define and manage the structure of a database, including creating, modifying, and deleting objects such as databases, tables, and views.

[0092] In this embodiment, different target data tables belong to different databases, so the table structures may be different. In the table structure comparison sub-process, by querying the system table in the database instance corresponding to the target data table, the definition information of the table in the database instance, such as the table name, columns, constraints, indexes and other information, can be obtained. Then, the corresponding DDL statement can be generated according to the table definition information as the table structure information corresponding to the database instance. By comparing the DDL statements of different database instances, the differences between the table structures can be obtained as the structural difference information, such as the addition and deletion of names, and the modification of columns, constraints, indexes, etc.

[0093] This application automates the data comparison process and can effectively compare the structure of data tables when data comparison is required during the development and maintenance of a self-developed database, thereby ensuring the accuracy of data comparison and improving the efficiency of data comparison.

[0094] In some optional implementations of this embodiment, the data corresponding to the target data table includes data sets of various preset sizes, and the step of obtaining the data corresponding to the target data table from the database instance, performing data comparison based on the data, and obtaining the data difference information includes:

[0095] Reading each of the data sets from the database instance in batches, and comparing them in memory to obtain comparison results corresponding to each of the data sets;

[0096] The data difference information is generated according to the comparison result.

[0097] It should be noted that database memory technology is an important research direction in the field of modern database management. With the development of hardware technology and the increase in data processing needs, traditional disk-based database systems can no longer meet the performance requirements in all scenarios, especially in application scenarios that require high-frequency data access and low-latency response. Storing data in memory can greatly increase the speed of data reading and writing, thereby improving the performance of the entire application system.

[0098] In this embodiment, the comparison sub-process of table data can be performed in a batch reading manner, that is, a data set of a preset size of the target data table is obtained from the database instance each time, and the comparison of the data set is completed in the memory to obtain the comparison result of the data set, and then data difference information is generated based on all the comparison results. Specifically, the data corresponding to the target data table includes data sets of various preset sizes, and the preset size can be determined by the number of rows in the data table. For example, if the number of rows in the target data table is 1 million and the preset size is 100,000 rows, then every 100,000 rows of data is regarded as a data set. In a batch manner, a data set is read from the database instance each time and stored in the memory for comparison, and the comparison result corresponding to the data set can be obtained, reflecting the difference in the comparison of the data set. Finally, data difference information can be generated based on the comparison results corresponding to each data set.

[0099] This application automates the data comparison process. When data comparison is required during the development and maintenance of a self-developed database, the data set can be read in batches and the comparison can be completed in memory to achieve data comparison of the data table, thereby reducing the number of database queries, network transmission overhead and memory overhead, and improving the efficiency of data comparison.

[0100] In some optional implementations of this embodiment, after the step of determining the target data table to be compared according to the target object set, the method further includes:

[0101] When executing the data comparison process, the process status information is recorded in real time;

[0102] When the data comparison process is monitored to be interrupted and reconnected, the remaining data that has not been compared is determined according to the process state information at the time of the interruption;

[0103] The data comparison process is completed based on the remaining data.

[0104] In this embodiment, when executing the preset data comparison process, the process status information can be recorded in real time, so that after the data comparison process is interrupted due to an exception, the data comparison process of the remaining data can be continued to be completed according to the process status information at the time of interruption when reconnecting, so as to realize the function of breakpoint resumption. Specifically, the process status information may include: the command line parameters of this call, the data and progress of the mode and table that have been compared, and the relevant information such as the mode, table and row being compared. When the data comparison process is reconnected, the process status information will be automatically detected, restored to the state before the interruption, and the data that has been compared will be skipped, and the data comparison process will continue to be completed for the remaining data without having to start from the beginning.

[0105] This application makes the data comparison process more flexible and enhances robustness by automatically implementing breakpoint resumption. In the process of large-scale data comparison, it can greatly save time costs and improve the efficiency of data comparison.

[0106] In some optional implementations of this embodiment, the step of generating a data comparison result corresponding to the data comparison instruction according to the difference information includes:

[0107] Filling the difference information into a preset data comparison chart template to obtain a filled data comparison chart;

[0108] Data enhancement is performed on the data comparison chart to obtain the data comparison result.

[0109] In this embodiment, the result of the data comparison is generally presented in the form of a chart. After executing the preset data comparison process according to the target data table and obtaining the difference information, the difference information can be filled into the preset data comparison chart template to obtain a filled-in data comparison chart, and then the data comparison chart is data enhanced, so that the enhanced data comparison chart is used as the data comparison result corresponding to the data comparison instruction. Specifically, the form of the chart used to present the data comparison results includes a bar chart, a bar chart, a line chart, an area chart, a pie chart, a radar chart, etc. The filled-in data comparison chart can belong to one or more of them. After obtaining the data comparison chart, it can be data enhanced, such as highlighting the data with large differences, marking with other colors, etc. Finally, the enhanced data comparison chart is used as the data comparison result corresponding to the data comparison instruction. The data comparison chart can be returned to the administrator terminal corresponding to the data comparison instruction for the administrator to view.

[0110] This application uses template filling and data enhancement to ultimately obtain a data comparison chart that represents the data comparison results, so that the data comparison results can be presented clearly and intuitively, thereby improving the efficiency and intelligence of data comparison.

[0111] In some optional implementations of this embodiment, the target object set includes database objects, or database objects and schema objects, or database objects, schema objects and data table objects, and the step of determining the target data table to be compared according to the target object set includes:

[0112] If the target object set includes a database object, determining a target database according to the database object, and using all data tables in the target database as the target data tables;

[0113] If the target object set includes database objects and schema objects, a target database is determined according to the database objects, a target schema is determined in the target database according to the schema objects, and all data tables in the target schema are used as the target data tables;

[0114] If the target object set includes database objects, schema objects and data table objects, the target database is determined according to the database objects, the target schema is determined in the target database according to the schema objects, and the target data table is determined in the target schema according to the data table objects.

[0115] In this embodiment, each database includes several modes, and there are several data tables under each mode. The database, mode, and data table are objects of different dimensions. When data comparison is required in the process of developing and maintaining the self-developed database, the relevant R&D personnel or operation and maintenance personnel can specify the target object that needs to be compared through the combination of command line parameters carried by the data comparison instruction to form a target object set, which can include target objects of different dimensions. For example, if the target object set includes two databases, all modes under the two databases and all data tables under each mode are compared one by one; if the target object set includes two databases and one mode in each of the two databases, all tables under the two modes are compared one by one; if the target object set includes two databases, one mode in each of the two databases, and one data table in each of the two modes, the two data tables are compared.

[0116] This application uses flexible control of command line parameter combinations. When data comparison is required during the development and maintenance of a self-developed database, the object for data comparison can be selected according to actual needs, thereby avoiding the waste of data processing resources and improving the efficiency of data comparison.

[0117] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0118] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0119] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0120] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0121] Further references Figure 3 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a data comparison device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0122] like Figure 3As shown, the data comparison device 300 described in this embodiment includes: a receiving module 301, a first determining module 302, a second determining module 303, an executing module 304 and a generating module 305. Among them:

[0123] A receiving module 301 is used to receive a data comparison instruction carrying a command line parameter combination;

[0124] A first determination module 302, configured to determine a target object set according to the command line parameter combination;

[0125] A second determination module 303 is used to determine a target data table to be compared according to the target object set;

[0126] An execution module 304 is used to execute a preset data comparison process according to the target data table to obtain difference information;

[0127] The generating module 305 is used to generate a data comparison result corresponding to the data comparison instruction according to the difference information.

[0128] The data comparison device provided in the present application can automatically select the target object that needs to be compared with the data and execute the corresponding data comparison process through the received parameters. When developing and maintaining a self-developed database, there is no need for manual operation or use of tools to complete the data comparison task, thereby ensuring the accuracy of the data comparison and improving the efficiency of the data comparison.

[0129] In some optional implementations of this embodiment, the difference information includes structure difference information and data difference information, and the execution module 304 is further used to:

[0130] Connect to the database instance corresponding to the target data table;

[0131] Acquire table structure information corresponding to the database instance, perform data comparison based on the table structure information, and obtain the structure difference information;

[0132] The data corresponding to the target data table is obtained from the database instance, and data comparison is performed based on the data to obtain the data difference information.

[0133] The data comparison device provided in the present application can realize the comparison of the structure of the data table and the data at the same time through an automated data comparison process when data comparison is required in the process of developing and maintaining a self-developed database, thereby improving the comprehensiveness of the data comparison and improving the efficiency of the data comparison.

[0134] In some optional implementations of this embodiment, the execution module 304 is further configured to:

[0135] Querying the system table in the database instance to obtain definition information of the table in the database instance;

[0136] Generate a corresponding DDL statement according to the definition information as the table structure information;

[0137] The DDL statements are compared to obtain the structural difference information.

[0138] The data comparison device provided in the present application can effectively compare the structure of data tables through an automated data comparison process when data comparison is required during the development and maintenance of a self-developed database, thereby ensuring the accuracy of data comparison and improving the efficiency of data comparison.

[0139] In some optional implementations of this embodiment, the data corresponding to the target data table includes data sets of various preset sizes, and the execution module 304 is further used to:

[0140] Reading each of the data sets from the database instance in batches, and comparing them in memory to obtain comparison results corresponding to each of the data sets;

[0141] The data difference information is generated according to the comparison result.

[0142] The data comparison device provided in the present application, through an automated data comparison process, can read the data set in batches and complete the comparison in memory when data comparison is needed in the process of developing and maintaining a self-developed database, so as to realize the comparison of data in the data table, thereby reducing the number of database queries, network transmission overhead and memory overhead, and improving the efficiency of data comparison.

[0143] In some optional implementations of this embodiment, the data comparison device 300 is further used for:

[0144] When executing the data comparison process, the process status information is recorded in real time;

[0145] When the data comparison process is monitored to be interrupted and reconnected, the remaining data that has not been compared is determined according to the process state information at the time of the interruption;

[0146] The data comparison process is completed based on the remaining data.

[0147] The data comparison device provided in the present application makes the data comparison process more flexible and enhances robustness by automatically implementing breakpoint resumption. In the process of large-scale data comparison, it can greatly save time costs and improve the efficiency of data comparison.

[0148] In some optional implementations of this embodiment, the generating module 305 is further configured to:

[0149] Filling the difference information into a preset data comparison chart template to obtain a filled data comparison chart;

[0150] Data enhancement is performed on the data comparison chart to obtain the data comparison result.

[0151] The data comparison device provided in the present application finally obtains a data comparison chart representing the data comparison result through template filling and data enhancement, so that the data comparison result can be presented clearly and intuitively, thereby improving the efficiency and intelligence of data comparison.

[0152] In some optional implementations of this embodiment, the target object set includes database objects, or database objects and schema objects, or database objects, schema objects and data table objects, and the second determining module 303 is further used to:

[0153] If the target object set includes a database object, determining a target database according to the database object, and using all data tables in the target database as the target data tables;

[0154] If the target object set includes database objects and schema objects, a target database is determined according to the database objects, a target schema is determined in the target database according to the schema objects, and all data tables in the target schema are used as the target data tables;

[0155] If the target object set includes database objects, schema objects and data table objects, the target database is determined according to the database objects, the target schema is determined in the target database according to the schema objects, and the target data table is determined in the target schema according to the data table objects.

[0156] The data comparison device provided in the present application can flexibly control the combination of command line parameters. When data comparison is required during the development and maintenance of a self-developed database, the object for data comparison can be selected according to actual needs, thereby avoiding the waste of data processing resources and improving the efficiency of data comparison.

[0157] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0158] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.

[0159] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0160] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the data comparison method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0161] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the data comparison method.

[0162] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0163] The computer device provided in the present application can automatically select the target object that needs to be compared with the data and execute the corresponding data comparison process through the received parameters. When developing and maintaining a self-developed database, there is no need for manual operation or use of tools to complete the data comparison task, thereby ensuring the accuracy of the data comparison and improving the efficiency of the data comparison.

[0164] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data comparison method as described above.

[0165] The computer-readable storage medium provided in the present application can automatically select the target object that needs to be compared with the data and execute the corresponding data comparison process through the received parameters. When developing and maintaining a self-developed database, there is no need for manual operation or the use of tools to complete the data comparison task, thereby ensuring the accuracy of the data comparison and improving the efficiency of the data comparison.

[0166] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0167] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A data comparison method, characterized in that: The steps include: Receiving a data comparison instruction carrying a command line parameter combination; Determine a target object set according to the command line parameter combination; Determining a target data table to be compared according to the target object set; Execute a preset data comparison process according to the target data table to obtain difference information; A data comparison result corresponding to the data comparison instruction is generated according to the difference information.

2. The data comparison method according to claim 1, characterized in that: The difference information includes structural difference information and data difference information. The step of executing a preset data comparison process according to the target data table to obtain the difference information specifically includes: Connect to the database instance corresponding to the target data table; Acquire table structure information corresponding to the database instance, perform data comparison based on the table structure information, and obtain the structure difference information; The data corresponding to the target data table is obtained from the database instance, and data comparison is performed based on the data to obtain the data difference information.

3. The data comparison method according to claim 2, characterized in that: The step of obtaining the table structure information corresponding to the database instance, performing data comparison according to the table structure information, and obtaining the structure difference information specifically includes: Querying the system table in the database instance to obtain definition information of the table in the database instance; According to the definition information, a corresponding DDL statement is generated as the table structure information; The DDL statements are compared to obtain the structural difference information.

4. The data comparison method according to claim 2, characterized in that: The data corresponding to the target data table includes data sets of various preset sizes. The step of obtaining the data corresponding to the target data table from the database instance, performing data comparison based on the data, and obtaining the data difference information specifically includes: Reading each of the data sets from the database instance in batches, and comparing them in memory to obtain comparison results corresponding to each of the data sets; The data difference information is generated according to the comparison result.

5. The data comparison method according to claim 1, characterized in that: After the step of determining the target data table to be compared according to the target object set, the method further includes: When executing the data comparison process, the process status information is recorded in real time; When the data comparison process is monitored to be interrupted and reconnected, the remaining data that has not been compared is determined according to the process state information at the time of the interruption; The data comparison process is completed based on the remaining data.

6. The data comparison method according to claim 1, characterized in that: The step of generating the data comparison result corresponding to the data comparison instruction according to the difference information specifically includes: Filling the difference information into a preset data comparison chart template to obtain a filled data comparison chart; Data enhancement is performed on the data comparison chart to obtain the data comparison result.

7. The data comparison method according to any one of claims 1 to 6, characterized in that: The target object set includes database objects, or database objects and schema objects, or database objects, schema objects and data table objects. The step of determining the target data table to be compared based on the target object set specifically includes: If the target object set includes a database object, determining a target database according to the database object, and using all data tables in the target database as the target data tables; If the target object set includes database objects and schema objects, a target database is determined according to the database objects, a target schema is determined in the target database according to the schema objects, and all data tables in the target schema are used as the target data tables; If the target object set includes database objects, schema objects and data table objects, the target database is determined according to the database objects, the target schema is determined in the target database according to the schema objects, and the target data table is determined in the target schema according to the data table objects.

8. A data comparison device, characterized in that: include: A receiving module, used for receiving a data comparison instruction carrying a command line parameter combination; A first determination module, used to determine a target object set according to the command line parameter combination; A second determination module, used to determine a target data table to be compared according to the target object set; An execution module, used for executing a preset data comparison process according to the target data table to obtain difference information; A generating module is used to generate a data comparison result corresponding to the data comparison instruction according to the difference information.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data comparison method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data comparison method according to any one of claims 1 to 7 are implemented.

Images