Data comparison method and device, equipment and storage medium

Abstract

The invention relates to the technical field of big data, can be applied to the field of financial science and technology, and discloses a data comparison method, device and equipment and a storage medium, and the method comprises the steps: obtaining the number of first records of a source data table, and obtaining the number of second records of a target data table; when the first record number is not the second record number, performing hash calculation on metadata in the source data table to generate a first hash value, and performing hash calculation on metadata in the target data table to generate a second hash value; when the first hash value is not the second hash value, obtaining a first field in the source data table, and obtaining a second field in the target data table; when the numerical value of the first field is not the numerical value of the second field, obtaining the accuracy rate of the target data table; and generating a data comparison result between the source data table and the target data table according to the difference value, the accuracy rate, the numerical value of the first field and the numerical value of the second field. The data comparison efficiency of the source data table and the target data table is improved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology and can be applied to the field of financial technology. In particular, the present invention relates to data comparison methods, devices, equipment and storage media. Background Art

[0002] As data becomes an indispensable factor in business operations and management, companies are building their own data centers to effectively utilize this data. To increase data access speed and reduce the risk of data loss, data centers migrate data to more efficient and secure locations.

[0003] Because the data migration process can be disrupted by factors such as network outages and human error, the risk of data migration errors always exists. Therefore, it is necessary to correct these errors by comparing the data between the source and target tables. Therefore, obtaining the data comparison results between the source and target tables is an urgent problem that needs to be solved. Summary of the Invention

[0004] The present invention provides a data comparison method, apparatus, computer equipment and storage medium to solve the technical problem of how to obtain data comparison results between a source data table and a target data table.

[0005] In a first aspect, a data comparison method is provided, comprising:

[0006] In the source cluster, select the data table before migration as the source data table, and in the target cluster, select the data table after migration as the target data table;

[0007] Obtaining a first number of records in the source data table and a second number of records in the target data table;

[0008] When the first number of records is not the second number of records, performing a hash calculation on the metadata in the source data table to generate a first hash value, and performing a hash calculation on the metadata in the target data table to generate a second hash value;

[0009] When the first hash value is not the second hash value, obtaining the first field in the source data table and obtaining the second field in the target data table;

[0010] When the value of the first field is not the same as the value of the second field, obtaining the difference between the source data table and the target data table according to the first number of records, the second number of records, and a predefined acquisition method, and obtaining the accuracy of the target data table;

[0011] A data comparison result between the source data table and the target data table is generated according to the difference, the accuracy, the value of the first field, and the value of the second field.

[0012] Furthermore, the obtaining of the first number of records in the source data table and the obtaining of the second number of records in the target data table include:

[0013] Sending a first query request to a first database corresponding to the source data table, receiving a first query result returned by the first database according to the first query request, and obtaining a first number of records in the source data table from the first query result;

[0014] A second query request is sent to a second database corresponding to the target data table, a second query result returned by the second database according to the second query request is received, and a second number of records of the target data table is obtained from the second query result.

[0015] Furthermore, when the first number of records is not the second number of records, performing a hash calculation on the metadata in the source data table to generate a first hash value, and performing a hash calculation on the metadata in the target data table to generate a second hash value, includes:

[0016] When the first number of records is not the second number of records, obtaining a preset hash algorithm;

[0017] The metadata in the source data table is hashed using the hash algorithm to generate a first hash value, and the metadata in the target data table is hashed to generate a second hash value. The hash algorithm includes one of the MD5 algorithm and the SHA algorithm, or a combination thereof.

[0018] Furthermore, when the first hash value is not the second hash value, obtaining the first field in the source data table and obtaining the second field in the target data table include:

[0019] When the first hash value is not the second hash value, obtaining a field name;

[0020] The first field corresponding to the field name in the source data table is obtained, and the second field corresponding to the field name in the target data table is obtained.

[0021] Furthermore, when the value of the first field is not the value of the second field, obtaining the difference between the source data table and the target data table based on the first number of records, the second number of records, and a predefined acquisition method, and obtaining the accuracy of the target data table includes:

[0022] When the value of the first field is not the same as the value of the second field, subtracting the second number of records from the first number of records to generate a difference between the source data table and the target data table;

[0023] The accuracy of the target data table is generated according to the first number of records, the second number of records, and a preset accuracy generation model.

[0024] Furthermore, generating a data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field, and the value of the second field includes:

[0025] selecting the difference, the accuracy, the value of the first field, and the value of the second field as target data;

[0026] The target data is imported into a preset comparison template to generate the data comparison result between the source data table and the target data table.

[0027] Furthermore, after generating the data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field, and the value of the second field, the data comparison method includes:

[0028] An upstream table of the target data table is obtained, and the data comparison result and the upstream table are displayed on a preset interactive interface.

[0029] In a second aspect, a data comparison device is provided, comprising:

[0030] A selection module is used to select a data table before migration as a source data table in a source cluster, and select the data table after migration as a target data table in a target cluster;

[0031] A first acquisition module is used to acquire a first number of records in the source data table and a second number of records in the target data table;

[0032] a generation module configured to, when the first number of records is not the second number of records, perform a hash calculation on the metadata in the source data table to generate a first hash value, and perform a hash calculation on the metadata in the target data table to generate a second hash value;

[0033] a second acquisition module, configured to, when the first hash value is not the second hash value, acquire the first field in the source data table and acquire the second field in the target data table;

[0034] a third acquisition module, configured to, when the value of the first field is not the same as the value of the second field, acquire the difference between the source data table and the target data table based on the first number of records, the second number of records, and a predefined acquisition method, and acquire the accuracy of the target data table;

[0035] A data comparison module is used to generate a data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field and the value of the second field.

[0036] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned data comparison method when executing the computer program.

[0037] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned data comparison method are implemented.

[0038] The present application provides a data comparison method, apparatus, computer equipment and storage medium, which comprises the following steps: in a source cluster, selecting a data table before migration as a source data table, and in a target cluster, selecting the data table after migration as a target data table; obtaining a first number of records in the source data table, and obtaining a second number of records in the target data table; when the first number of records is not the second number of records, performing a hash calculation on the metadata in the source data table to generate a first hash value, and performing a hash calculation on the metadata in the target data table to generate a second hash value; when the first hash value is not the second hash value, obtaining a first field in the source data table, and obtaining a second field in the target data table; when the value of the first field is not the value of the second field, obtaining a value between the source data table and the target data table according to the first number of records, the second number of records and a predefined acquisition method. The difference between the two is used to obtain the accuracy of the target data table; the data comparison result between the source data table and the target data table is generated according to the difference, the accuracy, the value of the first field and the value of the second field. The beneficial effects are in two aspects. On the one hand, the data comparison result between the source data table and the target data table is generated according to the difference, the accuracy, the value of the first field and the value of the second field, which solves the problem of how to obtain the data comparison result between the source data table and the target data table. Since the data comparison result can be automatically obtained, the acquisition time of the data comparison result is reduced, which is conducive to improving the acquisition efficiency of the data comparison result. On the other hand, by generating the data comparison result between the source data table and the target data table, the data comparison time can be significantly shortened, which is conducive to improving the data comparison efficiency between the source data table and the target data table. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 This is a schematic diagram of an application environment of a data comparison method according to an embodiment of the present invention;

[0041] Figure 2 A schematic flow chart of a data comparison method provided by one embodiment of the present invention;

[0042] Figure 3 yes Figure 1 A schematic flow chart of a specific implementation of step S23;

[0043] Figure 4 yes Figure 1 A schematic flow chart of a specific implementation of step S25;

[0044] Figure 5 yes Figure 1 A schematic flow chart of a specific implementation of step S26;

[0045] Figure 6 is a structural diagram of a data comparison device in one embodiment of the present invention;

[0046] Figure 7 is a structural diagram of a computer device in one embodiment of the present invention;

[0047] Figure 8 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0049] See also Figure 1 , Figure 1 This is a schematic diagram of an application environment of a data comparison method according to an embodiment of the present invention. The data comparison method provided by the embodiment of the present invention can be applied in the following situations: Figure 1 In an application environment, a client communicates with a server through a network.

[0050] The server selects the data table before migration as the source data table in the source cluster through the client, and selects the data table after migration as the target data table in the target cluster;

[0051] Obtaining a first number of records in the source data table and a second number of records in the target data table;

[0052] When the first number of records is not the second number of records, performing a hash calculation on the metadata in the source data table to generate a first hash value, and performing a hash calculation on the metadata in the target data table to generate a second hash value;

[0053] When the first hash value is not the second hash value, obtaining the first field in the source data table and obtaining the second field in the target data table;

[0054] When the value of the first field is not the same as the value of the second field, obtaining the difference between the source data table and the target data table according to the first number of records, the second number of records, and a predefined acquisition method, and obtaining the accuracy of the target data table;

[0055] A data comparison result between the source data table and the target data table is generated according to the difference, the accuracy, the value of the first field, and the value of the second field.

[0056] In the scheme implemented by the above-mentioned data comparison method, device, equipment and medium, the beneficial effects are in two aspects. On the one hand, the data comparison result between the source data table and the target data table is generated according to the difference, the accuracy, the value of the first field and the value of the second field, which solves the problem of how to obtain the data comparison result between the source data table and the target data table. Since the data comparison result can be obtained automatically, the time for obtaining the data comparison result is reduced, which is conducive to improving the efficiency of obtaining the data comparison result; on the other hand, by generating the data comparison result between the source data table and the target data table, the time for data comparison can be significantly shortened, which is conducive to improving the efficiency of data comparison between the source data table and the target data table.

[0057] The device running the client is referred to as a client device.

[0058] Among them, the device running the server is referred to as: server device.

[0059] The client devices may include but are not limited to various personal computers, laptops, smart phones, tablet computers, and portable wearable devices.

[0060] The server device can be implemented with an independent task database or a task database cluster composed of multiple task databases. The present invention will be described in detail below through specific embodiments.

[0061] See also Figure 2 , Figure 2 A flow chart of a data comparison method provided in one embodiment of the present invention includes the following steps:

[0062] S21, in the source cluster, selecting a data table before migration as a source data table, and in the target cluster, selecting the data table after migration as a target data table;

[0063] Exemplarily, selecting a data table before migration as a source data table in the source cluster and selecting the data table after migration as a target data table in the target cluster includes:

[0064] In the source cluster of the first data center, the data table before migration is selected as the source data table, and in the target cluster of the second data center, the data table after migration is selected as the target data table.

[0065] The first data center and the second data center are data centers with different target systems.

[0066] The data stored in the data center includes one or a combination of financial data, technology data, and insurance data.

[0067] Financial data refers to digital information directly related to a financial institution's core activities, such as business operations, market transactions, customer management, and risk management. Financial data is crucial for both individuals and businesses, so data centers must implement strict security measures to protect this data from unauthorized access, leakage, or tampering.

[0068] Among them, scientific and technological data includes experimental data, observation data, simulation data, user behavior data, code base, patent information and research results. Scientific and technological data plays a key role in promoting scientific and technological progress, promoting knowledge sharing and accelerating product innovation.

[0069] Insurance data refers to the vast amount of digital information directly related to insurance business accumulated during insurance operations. This data covers a wide range of dimensions, from basic customer information, policy details, and claims records to risk assessments and market trend analysis. Insurance data is not only fundamental for insurance companies to understand customer needs, assess risks, and price products and services, but also crucial for precision marketing, improving the customer experience, and optimizing operational processes.

[0070] S22, obtaining a first number of records in the source data table and a second number of records in the target data table;

[0071] The obtaining of the first number of records in the source data table and the obtaining of the second number of records in the target data table include:

[0072] Sending a first query request to a first database corresponding to the source data table, receiving a first query result returned by the first database according to the first query request, and obtaining a first number of records in the source data table from the first query result;

[0073] A second query request is sent to a second database corresponding to the target data table, a second query result returned by the second database according to the second query request is received, and a second number of records of the target data table is obtained from the second query result.

[0074] S23, when the first number of records is not the second number of records, performing a hash calculation on the metadata in the source data table to generate a first hash value, and performing a hash calculation on the metadata in the target data table to generate a second hash value;

[0075] S24, when the first hash value is not the second hash value, obtaining the first field in the source data table and obtaining the second field in the target data table;

[0076] The hash algorithm can map data of any length to a unique hash value of fixed length, so the first hash value can be used as a unique identifier for metadata in the source data table, and the second hash value can be used as a unique identifier for metadata in the target data table.

[0077] The characteristics of the hash algorithm determine that even a slight change in the metadata of the source data table will result in a completely different hash value. Therefore, when the first hash value is different from the second hash value, it can be confirmed that the metadata of the source data table and the metadata of the target data table are different.

[0078] Wherein, when the first hash value is not the second hash value, obtaining the first field in the source data table and obtaining the second field in the target data table include:

[0079] When the first hash value is not the second hash value, obtaining a field name;

[0080] The first field corresponding to the field name in the source data table is obtained, and the second field corresponding to the field name in the target data table is obtained.

[0081] S25, when the value of the first field is not the value of the second field, obtaining the difference between the source data table and the target data table according to the first number of records, the second number of records, and a predefined acquisition method, and obtaining the accuracy of the target data table;

[0082] S26 , generating a data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field, and the value of the second field.

[0083] After generating the data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field, and the value of the second field, the data comparison method includes:

[0084] An upstream table of the target data table is obtained, and the data comparison result and the upstream table are displayed on a preset interactive interface.

[0085] In an embodiment of the present invention, the beneficial effects lie in two aspects. On the one hand, the data comparison result between the source data table and the target data table is generated based on the difference, the accuracy, the value of the first field and the value of the second field, which solves the problem of how to obtain the data comparison result between the source data table and the target data table. Since the data comparison result can be obtained automatically, the time for obtaining the data comparison result is reduced, which is conducive to improving the efficiency of obtaining the data comparison result. On the other hand, by generating the data comparison result between the source data table and the target data table, the time for data comparison can be significantly shortened, which is conducive to improving the efficiency of data comparison between the source data table and the target data table.

[0086] See also Figure 3 , Figure 3 yes Figure 1 A specific implementation flow diagram of step S23 is described in detail as follows:

[0087] S31: When the first number of records is not the second number of records, obtaining a preset hash algorithm;

[0088] S32, using the hash algorithm, performing hash calculation on the metadata in the source data table to generate a first hash value, and performing hash calculation on the metadata in the target data table to generate a second hash value, wherein the hash algorithm includes one of the MD5 algorithm and the SHA algorithm or a combination thereof.

[0089] Among them, the Chinese name of the MD5 algorithm is: Message Digest Algorithm Version 5, and the English name of the MD5 algorithm is: Message-Digest Algorithm 5.

[0090] Among them, the Chinese name of the SHA algorithm is: Secure Hash Algorithm, and the English name of the SHA algorithm is: Secure Hash Algorithm.

[0091] Among them, the MD5 algorithm and the SHA algorithm can convert input data of any length into a hash value of fixed length, and both are one-way, that is, the original data cannot be reversed through the hash value.

[0092] In the embodiment of the present invention, the first hash value and the second hash value are saved, and the integrity and authenticity of the metadata in the target data table can be quickly verified by comparing the first hash value and the second hash value.

[0093] See also Figure 4 , Figure 4 yes Figure 1 A specific implementation flow diagram of step S25 is described in detail as follows:

[0094] S41, when the value of the first field is not the same as the value of the second field, subtract the second number of records from the first number of records to generate a difference between the source data table and the target data table;

[0095] S42: Generate a model based on the first number of records, the second number of records, and a preset accuracy rate to generate an accuracy rate of the target data table.

[0096] Exemplarily, the accuracy generation model is:

[0097]

[0098] Wherein, R is the accuracy of the target data table, CNT1 is the number of the first records, and CNT2 is the number of the second records.

[0099] The accuracy rate is used to measure the accuracy of the target data table. The higher the accuracy rate, the closer the target data table is to the source data table, which means the quality of data migration is higher. The lower the accuracy rate, the farther the target data table is from the source data table, which means the quality of data migration is lower.

[0100] In an embodiment of the present invention, the consistency of data migration is measured by the accuracy of the target data table. This consistency verification helps enterprises to promptly discover and resolve potential problems in data migration, reduce the risk of data migration, and improve the efficiency and quality of data migration.

[0101] See also Figure 5 , Figure 5 yes Figure 1 A specific implementation flow diagram of step S26 is described in detail as follows:

[0102] S51, selecting the difference, the accuracy, the value of the first field, and the value of the second field as target data;

[0103] S52: Import the target data into a preset comparison template to generate the data comparison result between the source data table and the target data table.

[0104] The target data is imported into a preset comparison template, and the comparison template into which the target data is imported is selected as the data comparison result between the source data table and the target data table.

[0105] Among them, through the data comparison result, the difference, the accuracy, the value of the first field and the value of the second field can be checked at one time, which is fast and convenient.

[0106] In the embodiment of the present invention, the problem of how to obtain the data comparison results between the source data table and the target data table is solved. Since the data comparison results can be obtained automatically, the time for obtaining the data comparison results is reduced, which is conducive to improving the efficiency of obtaining the data comparison results.

[0107] See also Figure 6 , Figure 6 FIG. 1 is a structural diagram of a data comparison device according to an embodiment of the present invention. Figure 6 As shown, the data comparison device includes a selection module 101, a first acquisition module 102, a generation module 103, a second acquisition module 104, a third acquisition module 105, and a data comparison module 106. The functional modules are described in detail as follows:

[0108] A selection module 101 is configured to select a data table before migration as a source data table in a source cluster, and select the data table after migration as a target data table in a target cluster;

[0109] A first acquisition module 102 is configured to acquire a first number of records in the source data table and a second number of records in the target data table;

[0110] A generating module 103 is configured to, when the first number of records is not the second number of records, perform a hash calculation on the metadata in the source data table to generate a first hash value, and perform a hash calculation on the metadata in the target data table to generate a second hash value;

[0111] A second acquisition module 104 is configured to acquire the first field in the source data table and the second field in the target data table when the first hash value is not the second hash value;

[0112] A third acquisition module 105 is configured to, when the value of the first field is not the same as the value of the second field, acquire the difference between the source data table and the target data table based on the first number of records, the second number of records, and a predefined acquisition method, and acquire the accuracy of the target data table;

[0113] The data comparison module 106 is configured to generate a data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field, and the value of the second field.

[0114] In one embodiment, the first acquisition module 102 includes:

[0115] a first acquiring subunit, configured to send a first query request to a first database corresponding to the source data table, receive a first query result returned by the first database according to the first query request, and acquire a first number of records in the source data table from the first query result;

[0116] The second acquisition subunit is used to send a second query request to the second database corresponding to the target data table, receive a second query result returned by the second database according to the second query request, and obtain a second number of records of the target data table from the second query result.

[0117] In one embodiment, the generating module 103 includes:

[0118] a third acquiring subunit, configured to acquire a preset hash algorithm when the first number of records is not the second number of records;

[0119] A generation subunit is used to perform hash calculation on the metadata in the source data table through the hash algorithm to generate a first hash value, and to perform hash calculation on the metadata in the target data table to generate a second hash value. The hash algorithm includes one of the MD5 algorithm and the SHA algorithm or a combination thereof.

[0120] In one embodiment, the second acquisition module 104 includes:

[0121] a fourth acquiring subunit, configured to acquire a field name when the first hash value is not the second hash value;

[0122] The fifth acquisition subunit is configured to acquire the first field corresponding to the field name in the source data table, and acquire the second field corresponding to the field name in the target data table.

[0123] In one embodiment, the third acquisition module 105 includes:

[0124] a first generating subunit, configured to, when the value of the first field is not the same as the value of the second field, subtract the second number of records from the first number of records to generate a difference between the source data table and the target data table;

[0125] The second generating subunit is configured to generate a model according to the first number of records, the second number of records, and a preset accuracy rate, and to generate the accuracy rate of the target data table.

[0126] In one embodiment, the data comparison module 106 includes:

[0127] a selection subunit, configured to select the difference, the accuracy, the value of the first field, and the value of the second field as target data;

[0128] The data comparison subunit is used to import the target data into a preset comparison template to generate the data comparison result between the source data table and the target data table.

[0129] In one embodiment, the data comparison device further includes:

[0130] The display module is used to obtain the upstream table of the target data table and display the data comparison result and the upstream table on a preset interactive interface.

[0131] In an embodiment of the present invention, the beneficial effects lie in two aspects. On the one hand, the data comparison result between the source data table and the target data table is generated based on the difference, the accuracy, the value of the first field and the value of the second field, which solves the problem of how to obtain the data comparison result between the source data table and the target data table. Since the data comparison result can be obtained automatically, the time for obtaining the data comparison result is reduced, which is conducive to improving the efficiency of obtaining the data comparison result. On the other hand, by generating the data comparison result between the source data table and the target data table, the time for data comparison can be significantly shortened, which is conducive to improving the efficiency of data comparison between the source data table and the target data table.

[0132] For the specific definition of the data comparison device, please refer to the definition of the data comparison method above, which will not be repeated here.

[0133] Each module in the above-mentioned data comparison device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0134] See also Figure 7 , Figure 7 This is a schematic diagram of a computer device in one embodiment of the present invention. In one embodiment, a computer device is provided. The computer device may be a server device, and its internal structure diagram may be as shown in FIG. Figure 7 The computer device includes a processor, a memory, a network interface and a database connected via a system bus.

[0135] The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When executed by the processor, the computer program can implement the functions or steps of a data comparison method on the server device.

[0136] See also Figure 8 , Figure 8 This is another structural diagram of a computer device in one embodiment of the present invention. In one embodiment, a computer device is provided. The computer device may be a client device, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external task database via a network connection. When the computer program is executed by the processor, the functions or steps of a data comparison method can be implemented on the client device.

[0137] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0138] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant description in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0139] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0140] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Parts and subsamples of some embodiments may be included in or replace parts and subsamples of other embodiments. Moreover, the terms used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to also include the plural forms. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of a stated subsample, whole, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other subsamples, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the process, method or device that includes the element. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments can be referenced to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be found in the description of the method part.

[0141] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0142] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some sub-samples can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to implement this embodiment. In addition, the functional units in the embodiments of the present disclosure can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0143] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A data comparison method, characterized in that: include: In the source cluster, the data table before migration is selected as the source data table, and in the target cluster, the data table after migration is selected as the target data table; Obtain a first number of records in the source data table, and obtain a second number of records in the target data table; When the first number of records is not the second number of records, performing hash calculation on the metadata in the source data table to generate a first hash value, and performing hash calculation on the metadata in the target data table to generate a second hash value; When the first Hash value is not the second Hash value, obtaining the first field in the source data table, and obtaining the second field in the target data table; When the value of the first field is not the value of the second field, obtaining the difference between the source data table and the target data table according to the first number of records, the second number of records and a predefined acquisition method, and obtaining the accuracy of the target data table; A data comparison result between the source data table and the target data table is generated according to the difference, the accuracy, the value of the first field, and the value of the second field.

2. The data comparison method according to claim 1, characterized in that: The obtaining of the first number of records in the source data table and the obtaining of the second number of records in the target data table include: Sending a first query request to a first database corresponding to the source data table, receiving a first query result returned by the first database according to the first query request, and obtaining a first number of records in the source data table from the first query result; A second query request is sent to a second database corresponding to the target data table, a second query result returned by the second database according to the second query request is received, and a second number of records of the target data table is obtained from the second query result.

3. The data comparison method according to claim 1, characterized in that: When the first number of records is not the second number of records, performing hash calculation on the metadata in the source data table to generate a first hash value, and performing hash calculation on the metadata in the target data table to generate a second hash value, including: When the first number of records is not the second number of records, obtaining a preset hash algorithm; Through the hash algorithm, the metadata in the source data table is hashed to generate a first hash value, and the metadata in the target data table is hashed to generate a second hash value. The hash algorithm includes one of the MD5 algorithm and the SHA algorithm or a combination thereof.

4. The data comparison method according to claim 1, characterized in that: When the first Hash value is not the second Hash value, obtaining the first field in the source data table and obtaining the second field in the target data table includes: When the first hash value is not the second hash value, obtaining a field name; The first field corresponding to the field name in the source data table is obtained, and the second field corresponding to the field name in the target data table is obtained.

5. The data comparison method according to claim 1, characterized in that: When the value of the first field is not the value of the second field, obtaining the difference between the source data table and the target data table according to the first number of records, the second number of records and a predefined acquisition method, and obtaining the accuracy of the target data table, includes: When the value of the first field is not the value of the second field, subtract the second number of records from the first number of records to generate a difference between the source data table and the target data table; The accuracy of the target data table is generated according to the first number of records, the second number of records and a preset accuracy generation model.

6. The data comparison method according to claim 1, characterized in that: The step of generating a data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field, and the value of the second field includes: Selecting the difference, the accuracy, the value of the first field, and the value of the second field as target data; The target data is imported into a preset comparison template to generate the data comparison result between the source data table and the target data table.

7. The data comparison method according to any one of claims 1 to 6, characterized in that: After generating the data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field and the value of the second field, the data comparison method includes: An upstream table of the target data table is obtained, and the data comparison result and the upstream table are displayed on a preset interactive interface.

8. A data comparison device, characterized in that: include: A selection module, used to select a data table before migration as a source data table in a source cluster, and select the data table after migration as a target data table in a target cluster; A first acquisition module, used to acquire a first number of records in the source data table and a second number of records in the target data table; a generating module, configured to, when the first number of records is not the second number of records, perform a hash calculation on the metadata in the source data table to generate a first hash value, and perform a hash calculation on the metadata in the target data table to generate a second hash value; A second acquisition module, configured to acquire the first field in the source data table and the second field in the target data table when the first hash value is not the second hash value; a third acquisition module, configured to acquire, when the value of the first field is not the value of the second field, the difference between the source data table and the target data table according to the first number of records, the second number of records and a predefined acquisition method, and acquire the accuracy of the target data table; A data comparison module is used to generate a data comparison result between the source data table and the target data table according to the difference, the accuracy, the value of the first field and the value of the second field.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the data comparison method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the data comparison method according to any one of claims 1 to 7 are implemented.

Images