Data verification method, device and equipment for storage system

Abstract

The data verification method, device and equipment of the storage system provided by the present application include: obtaining first data of a first database and second data of a second database in the storage system, wherein the storage system includes: multiple databases; if it is determined that the second data is different from the first data, then determining the remaining database set as a set of databases including multiple databases other than the first database and the second database; repeating the following steps until a preset stop condition is reached: determining a target database in the remaining database set, and performing data verification on the target data in the target database with the first data to determine the verification result; deleting the target database in the remaining database set. Through the above method, the problem of not being able to ensure whether the data in the target database is accurate when verifying the data in the target database with the data in the remaining databases can be avoided, and compared with the manual data verification method of the database, the data verification efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of data, and in particular to a data verification method, device and equipment for a storage system. Background Art

[0002] At present, in order to facilitate users to query data, data can be stored in multiple databases, and then multiple databases form a data storage system, so that different users can access different databases to query data. In addition, in order to ensure the accuracy of data when multiple databases are synchronized, data verification is usually required.

[0003] In the related art, manual sampling can be used to check the data in multiple databases in the data storage system. However, the above method is inefficient and may easily lead to users being unable to obtain accurate data.

[0004] Therefore, a data verification method for a storage system is urgently needed to avoid the above technical problems. Summary of the invention

[0005] The present application provides a data verification method, device and equipment for a storage system, which are used to solve the problem of low efficiency in performing data verification on a database in a storage system in the related art.

[0006] In a first aspect, the present application provides a data verification method for a storage system, comprising:

[0007] Acquire first data of a first database and second data of a second database in a storage system, wherein the storage system comprises: a plurality of databases; the plurality of databases are connected in sequence; the database is used to synchronize data of a previous database connected thereto; the first database is the first database among the plurality of databases; the second database is the last database among the plurality of databases;

[0008] If it is determined that the second data is different from the first data, determining the remaining database set as a set including databases of the plurality of databases other than the first database and the second database;

[0009] Repeat the following steps until a preset stop condition is reached: determine a target database in the remaining database set, and perform data verification on the target data in the target database and the first data to determine a verification result; and delete the target database in the remaining database set.

[0010] In a possible implementation manner, determining a target database in the remaining database set includes:

[0011] If it is determined that there is a previous target database, and the verification result representation corresponding to the previous target database is consistent, then in the remaining database set, a database located after the connection order of the previous target database is determined as the target database; wherein the previous target database is the target database determined in the previous repetition process of this repetition process.

[0012] In a possible implementation, performing data verification on the target data in the target database and the first data to determine the verification result includes:

[0013] If it is determined that the target database is used to store the incremental data of the first database, then determining the change rate information of the incremental data in the first database; wherein the change rate information is used to indicate the amount of incremental data generated per unit time;

[0014] If it is determined that the change rate information is greater than a preset threshold, the incremental data of the first data with the maximum storage time is determined as the first comparison data; and the target data with the maximum storage time is determined as the second comparison data;

[0015] A verification result is determined according to the first comparison data and the second comparison data.

[0016] In a possible implementation manner, determining the incremental data of the first data having the maximum storage time as the first comparison data includes:

[0017] Generate a data query statement based on a preset query statement corresponding to the first database and the maximum storage time;

[0018] Based on the data query statement, incremental data of the first data having the maximum storage time is determined in the first database as first comparison data.

[0019] In a possible implementation, the method further includes:

[0020] If it is determined that the change rate information is less than or equal to a preset threshold, determining the incremental data of the first data stored in the preset period as the third comparison data; determining the target data stored in the preset period as the fourth comparison data;

[0021] A verification result is determined according to the third comparison data and the fourth comparison data.

[0022] In a possible implementation manner, determining a verification result according to the third comparison data and the fourth comparison data includes:

[0023] Determining a first total number of the third comparison data and a second total number of the fourth comparison data;

[0024] If it is determined that the first total number and the second total number are different, then it is determined that the verification result indicates that the third comparison data and the fourth comparison data are different.

[0025] In a possible implementation, the method further includes:

[0026] If it is determined that the target database is used to store the basic data of the first database, determining at least one data generation time;

[0027] Determine the base data of the first data having the data generation time as the fifth comparison data; determine the target data having the data generation time as the sixth comparison data;

[0028] A verification result is determined according to the fifth comparison data and the sixth comparison data.

[0029] In a possible implementation, if the database types of the target database and the first database are different, determining a verification result according to the fifth comparison data and the sixth comparison data includes:

[0030] Determining a first database type of the target database and a second database type of the first database;

[0031] Determine conversion information according to the first database type and the second database type, wherein the conversion information is used to convert a data format in the first database type into a data format in the second database type;

[0032] The conversion information is called to perform data conversion processing on the sixth comparison data to obtain converted sixth data; and a verification result is determined according to the converted sixth data and the fifth comparison data.

[0033] In a possible implementation, the preset stop condition is that, in two adjacent databases connected in sequence, the verification result representation of the former database is consistent, and the verification result representation of the latter database is inconsistent.

[0034] In a second aspect, the present application provides a data checking device for a storage system, comprising:

[0035] an acquisition unit, configured to acquire first data from a first database and second data from a second database in a storage system, wherein the storage system comprises: a plurality of databases; the plurality of databases are sequentially connected; the database is configured to synchronize data from a previous database connected thereto; the first database is the first database in the plurality of databases; the second

[0036] The second database is the last database among the multiple databases;

[0037] 5 A first determining unit, configured to determine the remaining database set if it is determined that the second data is different from the first data

[0038] into a set including databases other than the first database and the second database among the plurality of databases;

[0039] Repeat the following second determination unit, checking unit and deletion unit until a preset stop condition is reached:

[0040] A second determining unit, configured to determine a target database from the remaining database set;

[0041] A checking unit, used for checking the target data in the target database with the first data to determine a check result;

[0042] A deleting unit is used to delete the target database from the remaining database set.

[0043] In a possible implementation manner, the second determining unit is specifically configured to:

[0044] If it is determined that there is a previous target database, and the verification result corresponding to the previous target database is consistent, then

[0045] In the remaining database set, a database located after the previous target database in the connection order is determined as the target database 5; wherein the previous target database is the target database determined in the previous repetition process of the current repetition process.

[0046] In a possible implementation, the checking unit includes:

[0047] The first determination module is used to determine the change rate information of the incremental data in the first database if it is determined that the target database is used to store the incremental data of the first database; wherein the change rate information is used to indicate the change rate per unit time.

[0048] The amount of incremental data generated within;

[0049] 0 A second determination module, configured to determine, if it is determined that the change rate information is greater than a preset threshold, the incremental data of the first data having the maximum storage time as the first comparison data;

[0050] A third determination module, used to determine the target data with the maximum storage time as the second comparison data;

[0051] The first checking module is used to determine a checking result according to the first comparison data and the second comparison data.

[0052] In a possible implementation manner, the second determining module is specifically configured to:

[0053] Generate a data query statement based on a preset query statement corresponding to the first database and the maximum storage time;

[0054] Based on the data query statement, incremental data of the first data having the maximum storage time is determined in the first database as first comparison data.

[0055] In a possible implementation manner, the device further includes:

[0056] a fourth determination module, configured to determine, if it is determined that the change rate information is less than or equal to a preset threshold, the incremental data of the first data stored within a preset period of time as the third comparison data;

[0057] A fifth determination module, used to determine that the target data stored in the preset time period is fourth comparison data;

[0058] The second checking module is used to determine a checking result according to the third comparison data and the fourth comparison data.

[0059] In a possible implementation, the second checking module is specifically configured to:

[0060] Determining a first total number of the third comparison data and a second total number of the fourth comparison data;

[0061] If it is determined that the first total number and the second total number are different, then it is determined that the verification result indicates that the third comparison data and the fourth comparison data are different.

[0062] In a possible implementation manner, the device further includes:

[0063] a sixth determination module, configured to determine at least one data generation time if it is determined that the target database is used to store the basic data of the first database;

[0064] A seventh determination module, configured to determine the basic quantity data of the first data having the data generation time as the fifth comparison data;

[0065] An eighth determination module, configured to determine the target data having the data generation time as the sixth comparison data;

[0066] The third checking module is used to determine a checking result according to the fifth comparison data and the sixth comparison data.

[0067] In a possible implementation, if the target database and the first database are of different database types, the third checking module is specifically configured to:

[0068] Determining a first database type of the target database and a second database type of the first database;

[0069] Determining conversion information according to the first database type and the second database type, wherein the conversion information is used to convert a data format in the first database type into a data format in the second database type;

[0070] Calling the conversion information, performing data conversion processing on the sixth comparison data, and obtaining converted sixth data;

[0071] A verification result is determined according to the converted sixth data and the fifth comparison data.

[0072] In a possible implementation, the preset stop condition is that, in two adjacent databases connected in sequence, the verification result representation of the former database is consistent, and the verification result representation of the latter database is inconsistent.

[0073] In a third aspect, the present application provides an electronic device, including: a memory, a processor;

[0074] Memory; Memory for storing instructions executable by the processor;

[0075] Wherein, the processor is used to execute the method as described in any one of the first aspects according to the executable instructions.

[0076] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in any one of the first aspects.

[0077] In a fifth aspect, the present application provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.

[0078] The data verification method, device and equipment of the storage system provided by the present application include: obtaining first data of a first database and second data of a second database in the storage system, wherein the storage system includes: multiple databases; the multiple databases are connected in sequence; the database is used to synchronize the data of the previous database connected thereto; the first database is the first database among the multiple databases; the second database is the last database among the multiple databases; if it is determined that the second data is different from the first data, then the remaining database set is determined to be a set of databases including the multiple databases except the first database and the second database; repeat the following steps until a preset stop condition is reached: determine a target database in the remaining database set, and perform data verification on the target data in the target database with the first data to determine the verification result; delete the target database in the remaining database set. Through the above method, the problem of not being able to ensure whether the data in the target database is accurate when verifying the data in the target database with the data in the remaining databases can be avoided, and the data verification efficiency can be improved compared to the manual data verification method of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0080] Figure 1 A schematic diagram of a data verification method provided in an embodiment of the present application;

[0081] Figure 2 A schematic diagram of a storage system provided in an embodiment of the present application;

[0082] Figure 3 A schematic diagram of a second data verification method provided in an embodiment of the present application;

[0083] Figure 4 A schematic diagram of a third data verification method provided in an embodiment of the present application;

[0084] Figure 5 A schematic diagram of the structure of another storage system provided in an embodiment of the present application;

[0085] Figure 6 A schematic diagram of the structure of a data checking device of a storage system is provided for an embodiment of the present application;

[0086] Figure 7 A schematic diagram of the structure of a data checking device for a second storage system provided in an embodiment of the present application;

[0087] Figure 8A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0088] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0089] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0090] At present, with the continuous development of information technology, more and more users are storing data in databases for subsequent data processing. In addition, in order to facilitate users to query data, data can also be stored in multiple databases, and then a data storage system composed of multiple databases can be formed so that different users can choose to access different databases for data query. In a data storage system composed of multiple databases, the types of databases included can be the same or different to meet the user's access requirements. In order to facilitate multiple databases to ensure the accuracy of data storage in multiple databases, it is usually necessary to perform data verification processing on the data synchronized between different databases.

[0091] In the related art, manual sampling can be used to check the data in multiple databases in the data storage system. However, the above method is inefficient and may cause users to fail to obtain accurate data.

[0092] The data verification method of the storage system provided in this application is used to solve the above technical problems.

[0093] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0094] Figure 1 A flow chart of a data verification method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method comprises the following steps:

[0095] S101. Obtain first data of a first database and second data of a second database in a storage system, wherein the storage system includes: multiple databases; the multiple databases are connected in sequence; a database is used to synchronize data of a previous database connected thereto; the first database is the first database among the multiple databases; and the second database is the last database among the multiple databases.

[0096] For example, Figure 2 A schematic diagram of a storage system provided in an embodiment of the present application. Figure 2 The storage system shown includes multiple databases connected in sequence, for example, databases 1-n in the figure, where n is a positive integer; wherein the data stored in the first database is the initial data, and the data in the multiple databases except the first database are obtained by synchronizing the data in the previous database connected thereto, that is, in the figure, the data in the second database is obtained from the first database, the data in the third database is obtained from the second database, and so on, to complete the data synchronization in multiple databases. Furthermore, it can be understood that the data stored in each database exists in the first database among the multiple databases.

[0097] The first database in this embodiment is the first database among the multiple databases, and the second database in this embodiment is the last database among the multiple databases. The data stored in the first database is called the first data, and the data stored in the second database is called the second data.

[0098] Therefore, in this embodiment, in order to verify the data in the database in the storage system, the first data in the first database is first compared with the second data in the second database to determine whether the second data is the same as the first data.

[0099] If it is determined that the second data is the same as the first data, it means that when multiple sequentially connected databases perform data synchronization in sequence, the synchronized data is accurate, and there is no need to perform data verification processing between other databases at this time.

[0100] It should be noted that, in this embodiment, when comparing the first data and the second data, reference may be made to the data verification method in the related art, and no specific limitation is made in this embodiment.

[0101] S102: If it is determined that the second data is different from the first data, determine that the remaining database set is a set of databases including multiple databases other than the first database and the second database.

[0102] For example, in this embodiment, if it is determined that the second data is different from the first data, it indicates that when multiple databases perform data synchronization between databases in sequence, there is a synchronization error, and it is necessary to continue to check the data of the remaining databases to determine the remaining databases where the data synchronization error occurs. That is, the remaining databases in the multiple databases except the first database and the second database are taken as the databases included in the remaining database set.

[0103] Repeat the following steps S103 and S104 until the preset stop condition is reached:

[0104] S103: Determine a target database in the remaining database set, and perform data verification on the target data in the target database and the first data to determine a verification result.

[0105] For example, in this embodiment, when data is checked on the databases in the remaining database set, a database may be selected from the remaining databases as the target database. When the target database is selected from the remaining database set for the first time, a database may be randomly selected as the target database, or a database previously connected to the second database may be selected as the target database. In this embodiment, there is no specific limitation on the selection of the target database.

[0106] Moreover, after determining the target database, since the data stored in the first database is the original data in the storage system, the target data in the target database can be compared with the first data in the first database to determine whether there is a synchronization error in the data in the target database.

[0107] S104: Delete the target database from the remaining database set.

[0108] Exemplarily, in this embodiment, if it is determined that the data verification result between the first data in the first database and the target data indicates that the first data and the target data are the same, then it indicates that the data stored in the target database is accurate; if it is determined that the data verification result between the first data in the first database and the target data indicates that the first data and the target data are not the same, then it indicates that the data stored in the target database is inaccurate.

[0109] Furthermore, after the verification result is determined, the target database may be deleted from the remaining database set.

[0110] At this time, it can be determined whether the preset stop condition is met. If it is determined that the preset stop condition is met, the data verification process is stopped at this time, otherwise, step S103 and step S104 are repeated.

[0111] In one example, the preset stop condition may be that the number of databases in the remaining database set is 0, that is, by continuously repeating the above data verification process, the data in each of the multiple databases is verified with the first data in the first database, thereby determining the accuracy of data synchronization in each database.

[0112] It can be understood that in this embodiment, the data of the last database (i.e., the second database) and the first database (i.e., the first database) in the multiple databases can be compared first. If the data of the two databases are determined to be the same, it indicates that the data storage between the databases in the storage system is accurate. If they are inconsistent, the target database can be selected from the remaining database set to perform data verification with the data in the first database to avoid the problem of not being able to ensure whether the data in the target database is accurate when verifying the target database with the data in the remaining databases, and compared with the manual data verification method of the database, the data verification efficiency can be improved.

[0113] Figure 3 A flow chart of a second data verification method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the method comprises the following steps:

[0114] S301. Obtain first data from a first database and second data from a second database in a storage system, wherein the storage system includes: multiple databases; multiple databases are connected in sequence; a database is used to synchronize data from a previous database connected thereto; the first database is the first database among the multiple databases; and the second database is the last database among the multiple databases.

[0115] S302: If it is determined that the second data is different from the first data, determine that the remaining database set is a set of databases including multiple databases other than the first database and the second database.

[0116] Exemplarily, the specific principles of step S301 and step S302 can refer to step S101 and step S102, which will not be repeated here.

[0117] Repeat the following steps until the preset stop condition is reached:

[0118] S303. If it is determined that there is a previous target database, and the verification result corresponding to the previous target database is consistent, then in the remaining database set, a database located after the connection order of the previous target database is determined as the target database; wherein the previous target database is the target database determined in the previous repetition process of this repetition process.

[0119] For example, in this embodiment, when determining a target database in the remaining database set, if step S303 and step S304 are steps executed in the repetition process, when determining the target database, the currently selected target database can be determined according to the data verification result obtained in the previous repetition process (i.e., the target database determined in the previous repetition process). Specifically, if the verification result of the previous target database indicates that the data verification is consistent, it means that the stored data of the database located before the connection sequence of the previous target database are all accurate, and therefore, the database located after the previous target database can be selected from the remaining databases as the target database in this repetition process.

[0120] In one example, when there is no previous target database, any database may be randomly selected as the target database.

[0121] It can be understood that in this embodiment, by selecting the target database as described above, the complex step of comparing the data in each database with the first database during the database data verification process can be avoided, thereby improving the data verification efficiency of the storage system.

[0122] In one example, if the verification result of the current target database indicates data verification inconsistency, it means that there is an error in the storage data of the database located before the previous target database in the connection order. Therefore, the database located before the previous target database can be selected from the remaining databases as the target database in this repetition process.

[0123] S304: If it is determined that the target database is used to store the incremental data of the first database, determine the change rate information of the incremental data in the first database; wherein the change rate information is used to indicate the amount of incremental data generated per unit time.

[0124] Exemplarily, in this embodiment, when comparing the target data of the target database and the first data of the first database, it is first necessary to determine the data type of the data stored in the target database, that is, whether the data stored in the target database is the base data in the first database or the incremental data in the first database.

[0125] If the target database is determined to be a database for storing incremental data in the first database, then at this time, different data verification methods are selected according to the change rate information of the incremental data in the first data. The change rate information of the incremental data can be understood as the update efficiency of the incremental data in the first database, that is, the amount of incremental data newly stored in the first database per unit time.

[0126] In one example, the first data in the first database includes incremental data and base data, and the incremental data and base data have different identification information. It is then possible to determine whether the target data is incremental data or base data based on the identification information carried by the data.

[0127] In one example, the change rate information may be used to determine the change rate information in the current period based on the change rate information in the historical period.

[0128] S305: If it is determined that the change rate information is greater than a preset threshold, determine the incremental data of the first data with the maximum storage time as the first comparison data; and determine the target data with the maximum storage time as the second comparison data.

[0129] Exemplarily, in this embodiment, when the change rate information is greater than a preset threshold, it indicates that the amount of incremental data generated in the current time period is large. At this time, the incremental data in the first data with the latest storage time (i.e., the maximum storage time) can be selected as the first comparison data, and the data with the maximum storage time in the target data can be determined as the second comparison data.

[0130] In one example, "determining the incremental data of the first data with the maximum storage time as the first comparison data" in step S305 includes the following steps: generating a data query statement based on a preset query statement and the maximum storage time corresponding to the first database; based on the data query statement, determining in the first database the incremental data of the first data with the maximum storage time as the first comparison data.

[0131] Exemplarily, in this embodiment, when searching for storage data with a maximum storage time in the incremental data in the first database, the maximum storage time in the incremental data in the first database can be determined, wherein the maximum storage time can be determined based on the operation log of the first database. In addition, in this embodiment, a preset query statement corresponding to the first database is also pre-set, which is used to search for data in the incremental data in the first data. When searching for the first comparison data, the maximum storage time can be written to the preset position of the preset query statement, thereby obtaining a data query statement, so that the first comparison data can be found by calling the data query statement.

[0132] It can be understood that, in this embodiment, by using a pre-query statement, the incremental data with the maximum storage time in the first data can be automatically queried as the first comparison data, so as to improve the data query efficiency and the subsequent data verification efficiency.

[0133] S306: Determine a verification result according to the first comparison data and the second comparison data.

[0134] Exemplarily, after the first comparison data and the second comparison data are determined, the comparison result corresponding to the first comparison data and the second comparison data may be used as the verification result of the first database and the target database.

[0135] In one example, when comparing the first comparison data and the second comparison data, the data corresponding to the key indicators in the first comparison data and the second comparison data can be extracted respectively, thereby reducing the complexity of the data verification process and improving the data comparison efficiency.

[0136] It can be understood that in this embodiment, when determining the target database for incremental data in the first database, at this time, the change rate information of the incremental data in the first database can be determined first. If the change rate information is greater than a preset threshold, a direct comparison is performed by comparing the data with the largest storage in the two databases, thereby improving data verification efficiency.

[0137] S307: If it is determined that the change rate information is less than or equal to the preset threshold, determine the incremental data of the first data stored in the preset period as the third comparison data; and determine the target data stored in the preset period as the fourth comparison data.

[0138] Exemplarily, in this embodiment, when it is determined that the change rate information of the incremental data in the first database is less than or equal to a preset threshold, it indicates that the update rate of the incremental data in the current first database is low. At this time, the data stored corresponding to the preset time can be selected from the incremental data of the first data and the target data in the target database, respectively, as the third comparison data and the fourth comparison data.

[0139] S308: Determine a verification result according to the third comparison data and the fourth comparison data.

[0140] Illustratively, in this embodiment, when the third comparison data and the fourth comparison data are obtained, the third comparison data and the fourth comparison data corresponding to the same generation time may be compared one by one to determine the data comparison result of the first database and the target database.

[0141] It can be understood that in this embodiment, when the change rate information of the incremental data in the first database is less than or equal to the preset threshold, at this time, since the change rate of the incremental data is small, in order to ensure the accuracy of data verification, data within a preset time period can be selected for data verification, thereby improving the accuracy of data verification.

[0142] In one example, when comparing the third comparison data and the fourth comparison data, the comparison may also be performed by extracting values ​​corresponding to preset keywords, thereby determining the comparison result.

[0143] In one example, step S308 includes the following steps: "determining a first total number of the third comparison data and a second total number of the fourth comparison data; if it is determined that the first total number and the second total number are different, determining that the verification result indicates that the third comparison data are different from the fourth comparison data."

[0144] Exemplarily, in the actual comparison process, when there are multiple third comparison data within a preset time period, and there are also multiple fourth comparison data, when comparing the third comparison data and the fourth comparison data, the total number of the third comparison data, i.e., the first total number, can be determined first, and the total number of the fourth comparison data, i.e., the second total number, can be determined. Afterwards, the first total number and the second total number are compared. If it is determined that the first total number and the second total number are different, then it indicates that an error occurred during the data synchronization process, i.e., the third comparison data and the fourth comparison data are different. Furthermore, by comparing the numbers described above, the efficiency of data verification can be improved.

[0145] In one example, when it is determined that the first total number and the second total number are the same, at this time, the keywords in the multiple third data can be compared one by one with the keywords in the multiple fourth data, or some keywords can be extracted for comparison to improve the accuracy of data verification.

[0146] S309: Delete the target database from the remaining database set.

[0147] Exemplarily, the technical principle of step S309 can be found in step S104 and will not be described in detail here.

[0148] In one example, the preset stop condition is that, in two adjacent databases connected in sequence, the verification result representation of the former database is consistent, and the verification result representation of the latter database is inconsistent.

[0149] Exemplarily, in this embodiment, when performing a repeated data verification process, at this time, the preset stop condition set can be that a pair of adjacent databases are found among multiple sequentially connected databases, and in the pair of databases, the verification result corresponding to the previous database (that is, the database that is earlier in the connection order between the two adjacent databases) (that is, the data verification result between the previous database and the first database) indicates that the data verification results of the previous database and the first database are consistent, and the data verification result of the latter database of the two adjacent databases (that is, the remaining databases of the two adjacent databases except the above-mentioned previous database) indicates that the data verification results between the database and the first database are inconsistent, then it can be determined that the latter database is the first database where a data synchronization error occurs, and the data verification process can be stopped at this time.

[0150] It is understandable that by setting the above preset stop condition, the first database having data synchronization problem can be found, and there is no need to perform data verification processing between each database and the first database, thereby improving the efficiency of database verification.

[0151] In some embodiments, in the above data verification process, after verifying the data in the target database with the data in the first database, there is no need to delete the target database from the remaining database set, that is, the remaining database set always includes all the remaining databases except the first database and the second database. Furthermore, when the number of databases in the remaining database set remains unchanged, when determining the target database, a binary search method can be used for searching. For example, first, a database in the middle of the connection order in the remaining database is selected as the target database. Then, according to the verification result corresponding to the target database determined this time, if the verification result is consistent, the binary search method is continued for all the remaining databases after the connection order of the target database this time to determine the target database; if the verification result corresponding to the target database determined this time is inconsistent, the binary search method is continued for all the remaining databases after the connection order of the target database this time to determine the target database, so as to quickly find the first database with data synchronization problems.

[0152] Figure 4 A schematic diagram of the process flow of the third data verification method provided in the embodiment of the present application is as follows: Figure 4 As shown, the method comprises the following steps:

[0153] S401. Obtain first data of a first database and second data of a second database in a storage system, wherein the storage system includes: multiple databases; the multiple databases are connected in sequence; a database is used to synchronize data of a previous database connected thereto; the first database is the first database among the multiple databases; and the second database is the last database among the multiple databases.

[0154] S402: If it is determined that the second data is different from the first data, determine that the remaining database set is a set of databases including multiple databases other than the first database and the second database.

[0155] Exemplarily, the specific principles of step S401 and step S402 can refer to step S101 and step S102, which will not be repeated here.

[0156] Repeat the following steps S403, S404, S405 and S406 until the preset stop condition is reached:

[0157] S403: Determine a target database in the remaining database set, and if it is determined that the target database is used to store basic data of the first database, determine at least one data generation time.

[0158] For example, in this embodiment, when the target database is determined, if it is further determined that the target database is used to store the basic data in the first database, since the basic data will not be updated within a period of time, at this time, at least one data generation time can be determined first. The data generation time can be understood as the time when a certain piece of data is generated.

[0159] S404, determining the base data of the first data having the data generation time as the fifth comparison data; and determining the target data having the data generation time as the sixth comparison data.

[0160] Exemplarily, after the data generation time is determined, fifth comparison data and sixth comparison data corresponding to the data generation time may be determined in the base data in the first database and the base data in the target data, respectively.

[0161] S405: Determine a verification result according to the fifth comparison data and the sixth comparison data.

[0162] Exemplarily, after the fifth comparison data and the sixth comparison data are determined, the data verification results corresponding to the fifth comparison data and the sixth comparison data may be used as the data verification results between the first database and the target database.

[0163] It can be understood that in this embodiment, if the target database is used to store the baseline data in the first database, then at this time, since the baseline data usually has a large amount of data and will not change within a certain period of time, at this time, at least one data generation time can be randomly determined to determine the fifth comparison data and the sixth comparison data corresponding to the data generation time for data comparison, so as to improve the data verification results.

[0164] In one example, if the database type of the target database is different from that of the first database, step S405 includes the following steps: determining the first database type of the target database and the second database type of the first database; determining conversion information based on the first database type and the second database type, the conversion information being used to convert the data format in the first database type into the data format of the second database type; calling the conversion information to perform data conversion processing on the sixth comparison data to obtain the converted sixth data; and determining the verification result based on the converted sixth data and the fifth comparison data.

[0165] Exemplarily, in the present embodiment, there are multiple databases with different database types in multiple databases connected in sequence. When the database types are different, the corresponding data storage formats will also be different. Therefore, when performing data verification, at this time, it can also be determined whether the target database and the first database are databases of the same type. If they are databases of the same type, the fifth comparison data and the sixth comparison data can be directly compared. If the target database and the first database are not databases of the same type, at this time, when comparing the fifth comparison data and the sixth comparison data, data conversion processing is first required. Specifically, the conversion information corresponding to the database type of the target database can be first searched, so that after determining the conversion information, the sixth comparison data can be converted into data with the same format as the fifth comparison data. Afterwards, the converted sixth comparison data obtained is compared with the fifth comparison data, and then the data verification result between the fifth comparison data and the sixth comparison data is obtained, and the data verification result is used as the verification result between the two databases. It should be noted that when comparing incremental data, the comparison can also be performed after data conversion processing. Here, only the basic data verification is used as an example to illustrate the data verification.

[0166] S406: Delete the target database from the remaining database set.

[0167] Exemplarily, the technical principle of step S309 can be found in step S104 and will not be described in detail here.

[0168] It can be understood that, in this embodiment, after the formats of data of different database types are converted, data comparison is performed to ensure the accuracy of subsequent data verification.

[0169] Figure 5 A structural diagram of another storage system provided for an embodiment of the present application. As shown in the figure, the storage system includes: 5 databases connected in sequence, wherein the data type of the first database is an Informix database, wherein the Informix database, as the first database among multiple databases connected in sequence, can be used to store the base data that needs to be stored and the incremental data updated in real time. The first Kafka database connected to the Informix database is used to synchronize the incremental data in the Informix database, that is, the first Kafka database is used to store the incremental data in the Informix database. During the data synchronization process, a table in the Informix corresponds to a topic in the first Kafka database. The data synchronization process here can refer to the principle description in the relevant technology, which will not be repeated here.

[0170] Afterwards, the data in the first Kafka database is stored in the HBase database connected to the first Kafka database in a column-based storage manner, and the Informix database can also be connected to the HBase database so that the basic data in the Informix database can also be stored in the HBase database in a column-based storage manner.

[0171] In addition, the incremental data in the hbase database will be converted into json format and synchronized to the second kafka database connected to it, and the data contained in the second kafka database can also be synchronized to the Elasticsearch database connected to the second kafka through the second kafka database. In addition, the Elasticsearch (ES for short) database is also connected to the informix database to store the basic data synchronized from the informix database.

[0172] Furthermore, when checking data in the above storage system, the base data and incremental data in the Elasticsearch database and the informix database can be checked separately, and then the check results of the two databases. Afterwards, if the check results are inconsistent, the target database can be found in the database before the ES database, and the data check can be continued to find the remaining databases with data synchronization problems.

[0173] Figure 6 A schematic diagram of a data checking device for a storage system is provided in an embodiment of the present application. Figure 6 As shown, the device comprises:

[0174] An acquisition unit 601 is used to acquire first data of a first database and second data of a second database in a storage system, wherein the storage system includes: a plurality of databases; the plurality of databases are connected in sequence; a database is used to synchronize data of a previous database connected thereto; the first database is the first database among the plurality of databases; and the second database is the last database among the plurality of databases;

[0175] A first determining unit 602 is configured to determine, if it is determined that the second data is different from the first data, that the remaining database set is a set of databases including the first database and the second database in the plurality of databases;

[0176] Repeat the following second determination unit 603, checking unit 604 and deletion unit 605 until a preset stop condition is reached:

[0177] A second determining unit 603 is used to determine a target database in the remaining database set;

[0178] A checking unit 604, configured to check the target data in the target database with the first data and determine a checking result;

[0179] The deleting unit 605 is used to delete the target database from the remaining database set.

[0180] The device provided in this embodiment is used to implement the technical solution provided by the above method. Its implementation principle and technical effect are similar and will not be described in detail.

[0181] Figure 7 A schematic diagram of the structure of a data verification device of a second storage system provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, in Figure 6 Based on the device structure shown, the second determining unit 603 in this embodiment is specifically used to:

[0182] If it is determined that there is a previous target database, and the verification result corresponding to the previous target database is consistent, then in the remaining database set, the database located after the connection order of the previous target database is determined as the target database; wherein the previous target database is the target database determined in the previous repetition process of this repetition process.

[0183] In a possible implementation, the checking unit 604 includes:

[0184] A first determination module 6041 is used to determine the change rate information of the incremental data in the first database if it is determined that the target database is used to store the incremental data of the first database; wherein the change rate information is used to indicate the amount of incremental data generated per unit time;

[0185] The second determination module 6042 is configured to determine the incremental data of the first data having the maximum storage time as the first comparison data if it is determined that the change rate information is greater than a preset threshold;

[0186] A third determination module 6043, configured to determine the target data having the maximum storage time as the second comparison data;

[0187] The first checking module 6044 is used to determine a checking result according to the first comparison data and the second comparison data.

[0188] In a possible implementation, the second determining module 6042 is specifically configured to:

[0189] Generate a data query statement based on a preset query statement and a maximum storage time corresponding to the first database;

[0190] Based on the data query statement, incremental data of the first data having the maximum storage time is determined in the first database as the first comparison data.

[0191] In a possible implementation, the device further includes:

[0192] The fourth determination module 6045 is configured to determine that the incremental data of the first data stored in the preset period is the third comparison data if it is determined that the change rate information is less than or equal to the preset threshold value;

[0193] A fifth determination module 6046, configured to determine that the target data stored in a preset period is fourth comparison data;

[0194] The second checking module 6047 is used to determine a checking result according to the third comparison data and the fourth comparison data.

[0195] In a possible implementation, the second checking module 6047 is specifically configured to:

[0196] Determining a first total number of the third comparison data and a second total number of the fourth comparison data;

[0197] If it is determined that the first total quantity and the second total quantity are different, then it is determined that the verification result indicates that the third comparison data and the fourth comparison data are different.

[0198] In a possible implementation, the device further includes:

[0199] A sixth determination module 6048 is configured to determine at least one data generation time if it is determined that the target database is used to store the basic data of the first database;

[0200] A seventh determination module 6049, configured to determine the basic quantity data of the first data having the data generation time as the fifth comparison data;

[0201] An eighth determination module 60410, configured to determine the target data having the data generation time as the sixth comparison data;

[0202] The third checking module 60411 is used to determine a checking result according to the fifth comparison data and the sixth comparison data.

[0203] In a possible implementation, if the database type of the target database is different from that of the first database, the third checking module 60411 is specifically used to:

[0204] Determine a first database type of the target database and a second database type of the first database;

[0205] Determining conversion information according to the first database type and the second database type, where the conversion information is used to convert a data format in the first database type into a data format in the second database type;

[0206] Calling the conversion information, performing data conversion processing on the sixth comparison data, and obtaining converted sixth data;

[0207] A verification result is determined based on the converted sixth data and the fifth comparison data.

[0208] In a possible implementation, the preset stop condition is that, in two adjacent databases connected in sequence, the verification result representation of the former database is consistent, and the verification result representation of the latter database is inconsistent.

[0209] The device provided in this embodiment is used to implement the technical solution provided by the above method. Its implementation principle and technical effect are similar and will not be described in detail.

[0210] The present application provides an electronic device, comprising: a memory, a processor;

[0211] Memory; Memory for storing processor executable instructions;

[0212] Wherein, the processor is used to execute the method according to the executable instructions.

[0213] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the electronic device includes:

[0214] The electronic device includes a processor 291 and a memory 292; it may also include a communication interface 293 and a bus 294. The processor 291, the memory 292, and the communication interface 293 may communicate with each other through the bus 294. The communication interface 293 may be used for information transmission. The processor 291 may call the logic instructions in the memory 294 to execute the method of the above embodiment.

[0215] In addition, the logic instructions in the above-mentioned memory 292 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0216] 0 Memory 292 is a computer-readable storage medium that can be used to store software programs, computer executable

[0217] Programs, such as program instructions / modules corresponding to the methods in the embodiments of the present application. The processor 291 executes the software programs, instructions and modules stored in the memory 292 to perform functional applications and data processing, that is, to implement the methods in the above method embodiments.

[0218] The memory 292 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the terminal device.

[0219] Etc. In addition, the memory 292 may include a high-speed random access memory and may also include a non-volatile memory.

[0220] The present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement any one of the methods.

[0221] The present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, any one of the methods is implemented.

[0222] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or other methods in the art that are not disclosed in this application.

[0223] The specification and examples are to be considered as exemplary only, with the true scope and spirit of the present application being indicated by the appended claims.

[0224] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data verification method for a storage system, characterized in that: include: Acquire first data of a first database and second data of a second database in a storage system, wherein the storage system comprises: a plurality of databases; the plurality of databases are connected in sequence; the database is used to synchronize data of a previous database connected thereto; the first database is the first database among the plurality of databases; the second database is the last database among the plurality of databases; If it is determined that the second data is different from the first data, determining the remaining database set as a set including databases of the plurality of databases other than the first database and the second database; Repeat the following steps until a preset stop condition is reached: determine a target database in the remaining database set, and perform data verification on the target data in the target database and the first data to determine a verification result; delete the target database in the remaining database set; Determining a target database from the remaining database set includes: If it is determined that there is a previous target database, and the verification result representation corresponding to the previous target database is consistent, then in the remaining database set, a database located after the connection order of the previous target database is determined as the target database; wherein the previous target database is the target database determined in the previous repetition process of this repetition process.

2. The method according to claim 1, characterized in that: Performing data verification on the target data in the target database and the first data to determine the verification result includes: If it is determined that the target database is used to store the incremental data of the first database, then determining the change rate information of the incremental data in the first database; wherein the change rate information is used to indicate the amount of incremental data generated per unit time; If it is determined that the change rate information is greater than a preset threshold, the incremental data of the first data with the maximum storage time is determined as the first comparison data; and the target data with the maximum storage time is determined as the second comparison data; A verification result is determined according to the first comparison data and the second comparison data.

3. The method according to claim 2, characterized in that The step of determining the incremental data of the first data having the maximum storage time as the first comparison data includes: Generate a data query statement based on a preset query statement corresponding to the first database and the maximum storage time; Based on the data query statement, incremental data of the first data having the maximum storage time is determined in the first database as first comparison data.

4. The method according to claim 2, characterized in that: The method further comprises: If it is determined that the change rate information is less than or equal to a preset threshold, determining the incremental data of the first data stored in the preset period as the third comparison data; determining the target data stored in the preset period as the fourth comparison data; A verification result is determined according to the third comparison data and the fourth comparison data.

5. The method according to claim 4, characterized in that Determining a verification result according to the third comparison data and the fourth comparison data includes: Determining a first total number of the third comparison data and a second total number of the fourth comparison data; If it is determined that the first total number and the second total number are different, then it is determined that the verification result indicates that the third comparison data and the fourth comparison data are different.

6. The method according to claim 2, characterized in that The method further comprises: If it is determined that the target database is used to store the basic data of the first database, determining at least one data generation time; Determine the base data of the first data having the data generation time as the fifth comparison data; determine the target data having the data generation time as the sixth comparison data; A verification result is determined according to the fifth comparison data and the sixth comparison data.

7. The method according to claim 6, characterized in that If the database types of the target database and the first database are different, determining a verification result according to the fifth comparison data and the sixth comparison data includes: Determining a first database type of the target database and a second database type of the first database; Determining conversion information according to the first database type and the second database type, wherein the conversion information is used to convert a data format in the first database type into a data format in the second database type; Calling the conversion information, performing data conversion processing on the sixth comparison data, and obtaining converted sixth data; A verification result is determined according to the converted sixth data and the fifth comparison data.

8. The method according to any one of claims 1 to 7, characterized in that The preset stop condition is that, in two adjacent databases connected in sequence, the check result representations of the former database are consistent, and the check result representations of the latter database are inconsistent.

9. A data checking device for a storage system, comprising: An acquisition unit is used to acquire first data of a first database and second data of a second database in a storage system, wherein the storage system comprises: a plurality of databases; the plurality of databases are connected in sequence; the database is used to synchronize data of a previous database connected thereto; the first database is the first database among the plurality of databases; the second database is the last database among the plurality of databases; a first determining unit, configured to determine, if it is determined that the second data is different from the first data, that the remaining database set is a set including databases other than the first database and the second database among the plurality of databases; Repeat the following second determination unit, checking unit and deletion unit until a preset stop condition is reached: A second determining unit, configured to determine a target database from the remaining database set; A checking unit, used for checking the target data in the target database with the first data to determine a checking result; A deleting unit, configured to delete the target database from the remaining database set; The first determination unit is specifically used to determine, if it is determined that there is a previous target database and the verification result corresponding to the previous target database is consistent, a database located after the connection order of the previous target database in the remaining database set as the target database; wherein the previous target database is the target database determined in the previous repetition process of this repetition process.

10. An electronic device comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images