Data management method and device, storage medium and electronic device

Abstract

The application discloses a data management method and device, a storage medium and an electronic equipment, and can also be applied to the technical field of databases, and particularly relates to data synchronization in a database. The method comprises the following steps: acquiring a data set in a source-end database and a data set in a target-end database respectively; sequentially comparing a plurality of first data subsets in the source-end database with a plurality of second data subsets in the target-end database to obtain a target comparison result; determining at least one data subset pair having a data difference relationship by using the target comparison result; performing N times of comparison on data in two data subsets in the at least one data subset pair until a convergence condition is reached; and displaying a target prompt information in the case that the comparison result of the N times of comparison indicates that the data in the two data subsets in the at least one data subset pair reaches an abnormal condition. The application solves the technical problem of low data management efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, in particular to a data management method and device, storage medium and electronic equipment. BACKGROUND

[0002] In the scenario of data synchronization, the data in the source database and the data in the target database are usually real-time changes, and the data synchronization in the related art is for static data, and is not applicable to the above-mentioned real-time changes of dynamic data, thereby leading to inaccurate management of data. Therefore, there is a problem of low accuracy of data management.

[0003] For the above problems, no effective solution has been proposed so far. SUMMARY

[0004] The embodiments of the present application provide a data management method, device, storage medium and electronic equipment to at least solve the technical problem of low accuracy of data management.

[0005] According to an aspect of the embodiments of the present application, a data management method is provided, comprising: acquiring a data set in a source database and a data set in a target database respectively, wherein the data set in the source database comprises a plurality of first data subsets, and the data set in the target database comprises a plurality of second data subsets; sequentially comparing the plurality of first data subsets in the source database and the plurality of second data subsets in the target database to obtain a target comparison result; determining at least one data subset pair having a data difference relationship by using the target comparison result, wherein the data difference relationship is used to represent that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to a difference threshold; performing N times of comparison on the data in the two data subsets in the at least one data subset pair until a convergence condition is reached, wherein N is a natural number; and displaying a target prompt information in the case that the comparison result of the N times of comparison indicates that the data in the two data subsets in the at least one data subset pair reaches an abnormal condition, wherein the target prompt information is used to prompt that the data synchronization in the two data subsets in the at least one data subset pair is abnormal.

[0006] According to another aspect of the embodiments of the present application, a data management apparatus is also provided, comprising: a first obtaining unit, configured to obtain a data set in a source database and a data set in a target database respectively, wherein the data set in the source database comprises a plurality of first data subsets, and the data set in the target database comprises a plurality of second data subsets; a first comparing unit, configured to compare the plurality of first data subsets in the source database and the plurality of second data subsets in the target database in sequence to obtain a target comparison result; a determining unit, configured to determine at least one pair of data subsets having a data difference relationship by using the target comparison result, wherein the data difference relationship is used to indicate that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to a difference threshold; a second comparing unit, configured to compare the data in the two data subsets in the at least one pair of data subsets for N times until a convergence condition is reached, wherein N is a natural number; and a display unit, configured to display a target prompt information in a case where the comparison result of the N times of comparison indicates that the data in the two data subsets in the at least one pair of data subsets reaches an abnormal condition, wherein the target prompt information is used to prompt that the data synchronization in the two data subsets in the at least one pair of data subsets is abnormal.

[0007] As an optional solution, the second comparing unit comprises at least one of the following: a first comparing module, configured to compare the data in the two data sets in the at least one pair of data subsets for the N times until the similarity between the data in the two data sets in the at least one pair of data subsets is lower than the difference threshold; and a second comparing module, configured to compare the data in the two data sets in the at least one pair of data subsets for the N times until the number of times of comparison of the data in the two data sets in the at least one pair of data subsets is greater than or equal to a convergence threshold, wherein N is a natural number less than or equal to the convergence threshold.

[0008] As an optional solution, the first comparing module comprises a first repeating submodule, configured to repeatedly execute the following steps until the similarity between the data in the two data sets in the at least one data subset pair is lower than the difference threshold: acquiring the current data in the first target data subset from the source-end database according to the first set identifier of the at least one data subset pair, and acquiring the current data in the second target data subset from the target-end database according to the second set identifier of the at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset; comparing the current data in the first target data subset and the current data in the second target data subset to obtain a current comparison result; in the case where the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is lower than the difference threshold, ending the N times of comparison; in the case where the current comparison result indicates that the similarity between the current data in the first data subset and the current data in the second data subset is greater than or equal to the difference threshold, acquiring the data in the first target data subset from the source-end database as the current data in the first target data subset according to the first set identifier of the at least one data subset pair, and acquiring the data in the second target data subset from the target-end database as the current data in the second target data subset according to the second set identifier of the at least one data subset pair.

[0009] As an optional solution, the second comparison module comprises a second repeating submodule, configured to repeatedly perform the following steps until the number of comparisons of data in the two data sets in the at least one data subset pair reaches the convergence threshold: obtaining, from the source-end database, current data in a first target data subset according to a first set identifier of the at least one data subset pair, and obtaining, from the target-end database, current data in a second target data subset according to a second set identifier of the at least one data subset pair, wherein the first set identifier is an identifier of the first target data subset, and the second set identifier is an identifier of the second target data subset; comparing the current data in the first target data subset and the current data in the second target data subset to obtain a current comparison result, and determining a current comparison number; in a case where the current comparison number is greater than or equal to the convergence threshold, ending the N times of comparison; in a case where the current comparison number is less than the convergence threshold, obtaining, from the source-end database, data in the first target data subset according to the first set identifier of the at least one data subset pair as the current data in the first target data subset, and obtaining, from the target-end database, data in the second target data subset according to the second set identifier of the at least one data subset pair as the current data in the second target data subset.

[0010] As an optional solution, the device further comprises a first obtaining submodule, configured to, after the N times of comparison of data in the two data sets in the at least one data subset pair is performed until the convergence condition is reached, obtain N comparison results of the N times of comparison; and perform integration processing on the N comparison results to obtain a comparison result to be judged; and the display unit comprises a display module, configured to, in a case where the comparison result to be judged meets the abnormal condition, display the target prompt information.

[0011] As an optional solution, the second comparison unit includes: a repetition module, used to repeatedly execute the following steps until the convergence condition is met: obtaining current data within a first target data subset from the source database based on a first set identifier of the at least one data subset pair, and obtaining current data within a second target data subset from the target database based on a second set identifier of the at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset; comparing the current data within the first target data subset and the current data within the second target data subset to obtain a current comparison result, and determining the current comparison count; and when the current comparison result indicates that the current data within the first target data subset is compared with... If the similarity between the current data in the second target data subset is less than the above-mentioned anomaly threshold, or the current number of comparisons is greater than or equal to the convergence threshold, then the above-mentioned convergence condition is determined to have been met. If the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is greater than or equal to the above-mentioned anomaly threshold, and the current number of comparisons has not reached the above-mentioned convergence threshold, then data in the first target data subset is obtained from the source database based on the first set identifier of the at least one data subset pair as the current data in the first target data subset, and data in the second target data subset is obtained from the target database based on the second set identifier of the at least one data subset pair as the current data in the second target data subset.

[0012] As an optional solution, the above-mentioned apparatus further includes: a processing unit, configured to convert data in each data subset of the source database and the target database into strings and establish an association relationship between each string and each data subset before acquiring the data sets in the source database and the target database respectively; the first comparison unit includes: a third comparison module, configured to sequentially compare the plurality of first data subsets in the source database with the plurality of second data subsets in the target database using the strings and the association relationships to obtain the target comparison result.

[0013] As an optional solution, the third comparison module includes: a second acquisition submodule, used to acquire M data subset pairs of the plurality of first data subsets in the source database and the plurality of second data subsets in the target database, and to acquire string pairs of each of the M data subset pairs using the association relationship, where M is a natural number; and a comparison submodule, used to sequentially compare the string pairs of each data subset pair to obtain the target comparison result. As an optional solution, the device further includes: a second acquisition unit, used to acquire the first moment when the at least one data subset pair with a data difference relationship is determined using the target comparison result; and the second comparison unit includes: a fourth comparison module, used to perform the N comparisons on the data in the two data sets of the at least one data subset pair when the current moment is the second moment, until the convergence condition is reached, where the time difference between the first moment and the second moment is a preset duration.

[0014] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data management method described above.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described data management method through the computer program.

[0016] In this embodiment, a data set from a source database and a data set from a target database are obtained, wherein the data set in the source database includes multiple first data subsets, and the data set in the target database includes multiple second data subsets. The multiple first data subsets in the source database and the multiple second data subsets in the target database are sequentially compared to obtain a target comparison result. At least one data subset pair with a data difference relationship is determined using the target comparison result, wherein the data difference relationship indicates that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to a difference threshold. The data in the two data subsets of the at least one data subset pair are compared N times until a convergence condition is reached, where N is a natural number. The comparison result of the N comparisons indicates that the at least one data subset pair... When data within two subsets of a dataset reaches an abnormal condition, a target prompt message is displayed. This prompt message indicates a synchronization anomaly in the data between two subsets of the at least one data subset pair. Using data subsets as the unit of data comparison improves comparison efficiency. Furthermore, considering that the data during synchronization is dynamic and delays are normal, when a comparison result showing discrepancies is obtained, it is not directly identified as an anomaly. Instead, multiple secondary comparisons are performed until a convergence condition is met. The more accurate comparison result obtained upon reaching the convergence condition is used to determine whether a synchronization anomaly exists, thereby achieving the goal of obtaining more accurate data comparison results. This improves the accuracy of data management and solves the technical problem of low data management accuracy. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a schematic diagram of an application environment for an optional data management method according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the flow of an optional data management method according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of an optional data management method according to an embodiment of this application;

[0021] Figure 4This is a schematic diagram of another optional data management method according to an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of another optional data management method according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of another optional data management method according to an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of another optional data management method according to an embodiment of this application;

[0025] Figure 8 This is a schematic diagram of another optional data management method according to an embodiment of this application;

[0026] Figure 9 This is a schematic diagram of another optional data management method according to an embodiment of this application;

[0027] Figure 10 This is a schematic diagram of another optional data management method according to an embodiment of this application;

[0028] Figure 11 This is a schematic diagram of an optional data management device according to an embodiment of this application;

[0029] Figure 12 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] For ease of understanding, the following is an explanation of the terms used in this application:

[0033] Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computing, storage, processing, and sharing.

[0034] Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

[0035] A database, simply put, can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, with minimal redundancy, and independent of application programs.

[0036] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as SQL (Structured Query Language) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages ​​simultaneously.

[0037] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be connected directly or indirectly through wired or wireless communication methods, which is not limited herein.

[0038] According to one aspect of the embodiments of this application, a data management method is provided. Optionally, as an optional implementation, the above-described data management method may be applied to, but is not limited to, [examples of other methods]. Figure 1 The environment shown may include, but is not limited to, user equipment 102, which may include, but is not limited to, a display 108, a processor 106, and a memory 104.

[0039] According to one aspect of the embodiments of this application, a data management method is provided. Optionally, as an optional implementation, the above-described data management method may be applied to, but is not limited to, [examples of other methods]. Figure 1 The environment shown may include, but is not limited to, user equipment 102 and server 112. User equipment 102 may include, but is not limited to, a display 108, a processor 106 and a memory 1004. Server 112 includes a database 114 and a processing engine 116.

[0040] The specific process can be summarized in the following steps:

[0041] In step S102, user equipment 102 obtains the data set in source database 1002 and the data set in target database 1004 respectively. The data set in source database 1002 includes multiple first data subsets (such as the data subsets of primary key 1, primary key 2 and primary key 3 in source database 1002), and the data set in target database 1004 includes multiple second data subsets (such as the data subsets of primary key 1, primary key 2 and primary key 3 in target database 1004).

[0042] Steps S104-S106: The data set in the source database 1002 and the data set in the target database 1004 are sent to the server 112 via network 110;

[0043] In step S108, server 112 uses a processing engine to sequentially compare multiple first data subsets in source database 1002 with multiple second data subsets in target database 1004 to obtain target comparison results; it uses the target comparison results to determine at least one pair of data subsets with data difference relationships; it performs N comparisons on the data in the two data subsets of the at least one data subset pair until a convergence condition is reached; if the comparison results of the N comparisons indicate that the data in the two data subsets of the at least one data subset pair has reached an abnormal condition, it generates target prompt information, wherein the target prompt information is used to indicate that the data synchronization in the two data subsets of the at least one data subset pair is abnormal;

[0044] In steps S110-S112, the target prompt information is sent to the user equipment 102 via the network 110. The user equipment 102 displays the target prompt information on the display 108 via the processor 106 and stores the target prompt information in the memory 104.

[0045] remove Figure 1 Beyond the examples shown, the above steps can be completed with server assistance. Specifically, the server can perform steps such as acquiring the target comparison results, comparing data within two subsets of at least one data subset pair, and generating target prompt information, thereby reducing the server's processing load. The user equipment 102 includes, but is not limited to, handheld devices (such as mobile phones), laptops, desktop computers, and in-vehicle devices. This application does not limit the specific implementation of the user equipment 102.

[0046] Alternatively, as an alternative implementation method, such as Figure 2 As shown, data management methods include:

[0047] S202, respectively obtain the data set in the source database and the data set in the target database, wherein the data set in the source database includes multiple first data subsets and the data set in the target database includes multiple second data subsets;

[0048] S204, compare multiple first data subsets in the source database with multiple second data subsets in the target database in turn to obtain the target comparison result;

[0049] S206, using the target comparison results to determine at least one data subset pair with a data difference relationship, wherein the data difference relationship is used to indicate that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to the difference threshold;

[0050] S208, compare the data in the two data subsets of at least one data subset pair N times until the convergence condition is met, where N is a natural number;

[0051] S210, if the comparison results of N comparisons indicate that the data in two data subsets of at least one data subset pair has reached an abnormal condition, a target prompt message is displayed, wherein the target prompt message is used to indicate that the data synchronization in two data subsets of at least one data subset pair is abnormal.

[0052] Optionally, in this embodiment, the above data management method can be applied, but is not limited to, in database migration scenarios. After the database migration is completed using an online migration product, the source database and the target database will be synchronized in real time for a period of time using data synchronization software. The data in the source database and the target database are changing in real time. Traditional static data management methods are not suitable for comparing data in such dynamically changing databases. However, in this embodiment, by using efficient comparison of data subset dimensions and combining it with data convergence comparison logic, the differences between the source data and the target database can be accurately located, thereby completing the comparison and verification of the source data and the target database.

[0053] Optionally, in this embodiment, the relationship between the source database and the target database can be understood, but is not limited to, synchronizing data from the source database to the target database, such as... Figure 3As shown, data in the source database 302 is synchronized to the target database 304. For example, data A in the source database 302 has been synchronized to the target database 304, while data B in the source database 302 is being synchronized to the target database 304, and data C in the source database 302 has not yet been synchronized to the target database 304. The above data management method can be applied after data synchronization, such as after data A, data B, and data C in the source database 302 have all been synchronized to the target database 304. However, the data in the target database 302 is usually updated in real time, which leads to frequent data synchronization. Therefore, to improve the data synchronization efficiency, the above data management method can be applied during the data synchronization process, such as during the process of synchronizing data A, data B, and data C in the source database 302 to the target database 304 in sequence.

[0054] Optionally, in this embodiment, the dataset may, but is not limited to, be divided into multiple data subsets, wherein each data subset may, but is not limited to, be named with a different data subset identifier, such as... Figure 4 As shown, the data in dataset 402 is organized into 6 data subsets 404 according to the data columns, namely the data subset corresponding to primary key 1, the data subset corresponding to primary key 2, the data subset corresponding to primary key 3, the data subset corresponding to primary key 4, the data subset corresponding to primary key 5, and the data subset corresponding to primary key 6.

[0055] Optionally, in this embodiment, to improve the efficiency of data comparison, the data within each data subset can be organized into a code, such as a CRC code, MD5 code, etc., for example based on... Figure 4 The scenario shown continues, for example... Figure 5 As shown, the data in each data subset 404 is calculated as a CRC code value, thus obtaining the CRC code value corresponding to each data subset 404.

[0056] Optionally, in this embodiment, the target comparison result can be understood as, but is not limited to, the first comparison result. The first comparison result does not take into account factors such as data synchronization delay, which makes it difficult to guarantee the accuracy of the first comparison result. On the other hand, the N comparisons performed on the data in two data subsets of at least one data subset pair can be understood as multiple secondary comparisons. The secondary comparisons take into account factors such as data synchronization delay to make up for the lower accuracy of the first comparison result.

[0057] It should be noted that the process involves acquiring data sets from both the source and target databases. The source database contains multiple first data subsets, and the target database contains multiple second data subsets. The multiple first data subsets in the source database are sequentially compared with the multiple second data subsets in the target database to obtain target comparison results. These results are used to identify at least one pair of data subsets with a data difference relationship, where the data difference relationship indicates that the similarity between data in the first and second data subsets is greater than or equal to a difference threshold. The data in the two data subsets of the at least one data subset pair are compared N times until a convergence condition is met, where N is a natural number. If the comparison results of the N comparisons indicate that the data in the two data subsets of the at least one data subset pair has reached an abnormal condition, a target prompt message is displayed, indicating an anomaly in the data synchronization of the two data subsets in the at least one data subset pair.

[0058] To further illustrate, optional examples include... Figure 6 As shown, data sets in source database 602 and target database 604 are obtained respectively. The data set in source database 602 includes multiple first data subsets (e.g., first data subset A, first data subset B, and first data subset C), and the data set in target database 604 includes multiple second data subsets (e.g., second data subset A, second data subset B, and second data subset C). The multiple first data subsets in source database 602 and the multiple second data subsets in target database 604 are compared sequentially to obtain a target comparison result 606. Using the target comparison result 606, at least one pair of data subsets with a data difference relationship is determined. This data difference relationship indicates that the similarity between data in the first data subset and data in the second data subset is greater than or equal to a difference threshold. For at least one... The data within two data subsets (such as the first data subset C and the second data subset C) of a data subset pair are compared N times until a convergence condition is met. For example, if the first comparison is performed and the comparison result 608-1 is obtained, and the comparison result 608-1 does not meet the convergence condition, the second comparison is performed to obtain the comparison result 608-2. If the comparison result 608-2 meets the convergence condition, the N comparisons are terminated. If the comparison result of the N comparisons (such as the comparison result 608-2) indicates that the data within at least two data subsets (such as the first data subset C and the second data subset C) of at least one data subset pair has reached an abnormal condition, a target prompt message 610 is displayed. The target prompt message is used to indicate that the data synchronization within at least two data subsets (such as the first data subset C and the second data subset C) of at least one data subset pair is abnormal.

[0059] The embodiments provided in this application involve obtaining a data set from a source database and a data set from a target database, wherein the data set in the source database includes multiple first data subsets, and the data set in the target database includes multiple second data subsets; the multiple first data subsets in the source database and the multiple second data subsets in the target database are sequentially compared to obtain target comparison results; at least one pair of data subsets with a data difference relationship is determined using the target comparison results, wherein the data difference relationship indicates that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to a difference threshold; the data in the two data subsets of the at least one pair of data subsets are compared N times until a convergence condition is reached, wherein N is a natural number; the comparison results of the N comparisons indicate at least one data subset. If data within two subsets of a data pair reaches an anomaly condition, a target prompt message is displayed. This prompt message indicates a synchronization anomaly in at least one subset pair between two subsets. Using data subsets as the unit of data comparison improves comparison efficiency. Furthermore, considering that the data during synchronization is dynamic and delays are normal, when a comparison result showing discrepancies is obtained, it is not directly identified as an anomaly. Instead, multiple secondary comparisons are performed until a convergence condition is met. The more accurate comparison result obtained upon reaching the convergence condition is used to determine if a synchronization anomaly exists, thereby achieving the goal of obtaining more accurate data comparison results and improving the technical effect of data management accuracy.

[0060] As an optional approach, the data within the two subsets of at least one data subset pair are compared N times until a convergence condition is met, including at least one of the following:

[0061] S1, compare the data in the two data sets of at least one data subset pair N times until the similarity between the data in the two data sets of at least one data subset pair is lower than the difference threshold;

[0062] S2, perform N comparisons on the data in the two data sets of at least one data subset pair, until the number of comparisons on the data in the two data sets of at least one data subset pair is greater than or equal to the convergence threshold, where N is a natural number less than or equal to the convergence threshold.

[0063] Optionally, in this embodiment, the data in the two data sets of at least one data subset pair are compared N times until the similarity between the data in the two data sets of at least one data subset pair is lower than the difference threshold.

[0064] To further illustrate, the following steps may be optionally repeated until the similarity between the data in the two data sets of at least one subset pair is below a difference threshold:

[0065] S1, obtain the current data in the first target data subset from the source database according to the first set identifier of at least one data subset pair, and obtain the current data in the second target data subset from the target database according to the second set identifier of at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset;

[0066] S2, compare the current data in the first target data subset with the current data in the second target data subset to obtain the current comparison result;

[0067] S3, if the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is lower than the difference threshold, end the N comparisons;

[0068] S4, if the current comparison result indicates that the similarity between the current data in the first data subset and the current data in the second data subset is greater than or equal to the difference threshold, then the data in the first target data subset is obtained from the source database based on the first set identifier of at least one data subset pair as the current data in the first target data subset, and the data in the second target data subset is obtained from the target database based on the second set identifier of at least one data subset pair as the current data in the second target data subset.

[0069] Optionally, in this embodiment, the plurality of first data subsets include a first target data subset, and the plurality of second data subsets include a second target data subset; the first target data subset and the second target data subset are any pair of data subsets in at least one data subset pair.

[0070] Optionally, in this embodiment, the data in the two data sets of at least one data subset pair are compared N times until the number of comparisons of the data in the two data sets of at least one data subset pair is greater than or equal to the convergence threshold, where N is a natural number less than or equal to the convergence threshold.

[0071] To further illustrate, the following steps may be repeated, for example, until the number of comparisons of data from the two datasets in at least one subset pair reaches a convergence threshold:

[0072] S1, obtain the current data in the first target data subset from the source database according to the first set identifier of at least one data subset pair, and obtain the current data in the second target data subset from the target database according to the second set identifier of at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset;

[0073] S2, compare the current data in the first target data subset with the current data in the second target data subset to obtain the current comparison result and determine the current comparison count;

[0074] S3, if the current number of alignments is greater than or equal to the convergence threshold, terminate the N alignments;

[0075] S4, if the current number of comparisons is less than the convergence threshold, retrieve data from the source database for the first target data subset based on the first set identifier of at least one data subset pair as the current data in the first target data subset, and retrieve data from the target database for the second target data subset based on the second set identifier of at least one data subset pair as the current data in the second target data subset.

[0076] Optionally, in this embodiment, the plurality of first data subsets include a first target data subset, and the plurality of second data subsets include a second target data subset; the first target data subset and the second target data subset are any pair of data subsets in at least one data subset pair.

[0077] As an optional approach, after performing N comparisons on the data in two data sets within at least one data subset pair until the convergence condition is met, the process includes: obtaining N comparison results from the N comparisons; and integrating the N comparison results to obtain the comparison result to be judged.

[0078] As an optional approach, if the comparison results of N comparisons meet the abnormal conditions, a target prompt message is displayed, including: if the comparison result to be judged meets the abnormal conditions, a target prompt message is displayed.

[0079] Optionally, in this embodiment, to improve the accuracy of the comparison results, the N comparison results obtained from multiple secondary comparisons can be integrated, but are not limited to. For example, in the first comparison, the data in the two datasets are "QWERT" in dataset A and "AWERT" in dataset B, respectively. The current comparison result is "01111", where "0" indicates that the similarity between the data is less than the difference threshold, and "1" indicates that the similarity between the data is greater than the difference threshold. Then, in the second comparison, the data in the two datasets are "QWERT" in dataset A and "AWERT" in dataset B, respectively. If we combine "QWERA" in set B, the current comparison result is "11110". Furthermore, in the third comparison, the data in the two sets are "QWERT" in set A and "QWERT" in set B, respectively, and the current comparison result is "11111". Although the final comparison result indicates that there are no abnormalities in the data in the two sets, the comparison result to be judged obtained by integrating the three comparison results can be found that the data synchronization abnormality that occurred during the data synchronization process is not within the normal range, or in other words, the comparison result to be judged meets the abnormal conditions, and the target prompt information will also be displayed.

[0080] The embodiments provided in this application obtain N comparison results from N comparisons; integrate the N comparison results to obtain the comparison result to be judged; and display target prompt information when the comparison result to be judged meets the abnormal conditions, thereby achieving the technical effect of improving the accuracy of the comparison results.

[0081] Optionally, in this embodiment, the data in the two data sets of at least one data subset pair are compared N times until the similarity between the data in the two data sets of at least one data subset pair is lower than the difference threshold, or the number of comparisons of the data in the two data sets of at least one data subset pair is greater than or equal to the convergence threshold.

[0082] To further illustrate, the following steps may be repeated, for example, until the convergence condition is met:

[0083] S1, obtain the current data in the first target data subset from the source database according to the first set identifier of at least one data subset pair, and obtain the current data in the second target data subset from the target database according to the second set identifier of at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset;

[0084] S2, compare the current data in the first target data subset with the current data in the second target data subset to obtain the current comparison result and determine the current comparison count;

[0085] S3, if the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is less than the anomaly threshold, or the current number of comparisons is greater than or equal to the convergence threshold, then the convergence condition is determined to be met.

[0086] S4, if the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is greater than or equal to the anomaly threshold, and the current number of comparisons has not reached the convergence threshold, then the data in the first target data subset is obtained from the source database based on the first set identifier of at least one data subset pair as the current data in the first target data subset, and the data in the second target data subset is obtained from the target database based on the second set identifier of at least one data subset pair as the current data in the second target data subset.

[0087] Optionally, in this embodiment, the plurality of first data subsets include a first target data subset, and the plurality of second data subsets include a second target data subset; the first target data subset and the second target data subset are any pair of data subsets in at least one data subset pair.

[0088] It should be noted that the following steps are repeated until the convergence condition is met: Retrieve the current data within the first target data subset from the source database based on the first set identifier of at least one data subset pair, and retrieve the current data within the second target data subset from the target database based on the second set identifier of at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset; compare the current data within the first target data subset and the current data within the second target data subset to obtain the current comparison result, and determine the current comparison count; The current comparison result indicates the comparison between the current data within the first target data subset and the current data within the second target data subset. If the similarity between the current data in the subset is less than the anomaly threshold, or the current number of comparisons is greater than or equal to the convergence threshold, the convergence condition is determined to have been met. If the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is greater than or equal to the anomaly threshold, and the current number of comparisons has not reached the convergence threshold, data in the first target data subset is obtained from the source database based on the first set identifier of at least one data subset pair as the current data in the first target data subset, and data in the second target data subset is obtained from the target database based on the second set identifier of at least one data subset pair as the current data in the second target data subset.

[0089] As an optional approach, before obtaining the data sets in the source database and the target database respectively, the following steps are taken: converting the data in each data subset of the source database and the target database into strings, and establishing the association between each string and each data subset;

[0090] As an optional approach, multiple first data subsets in the source database are sequentially compared with multiple second data subsets in the target database to obtain the target comparison result. This includes: using strings and association relationships to sequentially compare multiple first data subsets in the source database with multiple second data subsets in the target database to obtain the target comparison result.

[0091] Optionally, in this embodiment, to improve the efficiency of data comparison, the comparison of multiple data pairs can be transformed into a comparison of one string to another, such as... Figure 5 As shown, the data “ABC”, “XYZ” and “FGH” in the data subset 404 corresponding to primary key 1 are converted into the CRC code value “04940304”.

[0092] It should be noted that the data in each data subset of the source database and the target database are converted into strings, and the association between each string and each data subset is established. The strings and the association are then used to compare the multiple first data subsets in the source database with the multiple second data subsets in the target database in turn to obtain the target comparison result.

[0093] To further illustrate, optional examples include... Figure 7 As shown, the data in each data subset in the source database 702 and the target database 704 is converted into strings (CRC codes), and an association relationship is established between each string and each data subset. For example, the CRC code "04940304" is associated with the primary key 1 in the first data subset 706. The strings and the association relationships are used to compare the multiple first data subsets 706 in the source database 702 with the multiple second data subsets 708 in the target database 704 in turn, to obtain the target comparison result 710. If the similarity between the CRC code corresponding to the primary key 6 of the first data subset 706 and the CRC code corresponding to the primary key 6 of the second data subset 708 is less than the abnormal threshold, then the target comparison result 710 used to represent the CRC code corresponding to the primary key 6 of the second data subset 708 is obtained.

[0094] The embodiments provided in this application convert the data in each data subset of the source database and the target database into strings and establish the association between each string and each data subset; the strings and the association are used to compare the multiple first data subsets in the source database with the multiple second data subsets in the target database in turn to obtain the target comparison result, thereby achieving the technical effect of improving the efficiency of obtaining the target comparison result.

[0095] As an optional approach, multiple first data subsets in the source database and multiple second data subsets in the target database are sequentially compared using strings and association relationships to obtain the target comparison results, including:

[0096] S1, obtain M data subset pairs from multiple first data subsets in the source database and multiple second data subsets in the target database, and use the association relationship to obtain the string pair of each data subset pair in the M data subset pairs, where M is a natural number;

[0097] S2, compare the string pairs of each data subset in turn to obtain the target comparison result.

[0098] It should be noted that the process involves obtaining M data subset pairs from multiple first data subsets in the source database and multiple second data subsets in the target database, and using the association relationship to obtain the string pair of each of the M data subset pairs, where M is a natural number; the string pairs of each data subset pair are compared sequentially to obtain the target comparison result.

[0099] To further illustrate, optional examples include... Figure 7 As shown, M data subset pairs are obtained from multiple first data subsets 706 in the source database 702 and multiple second data subsets 708 in the target database 704. For example, there are 6 data subset pairs, namely the data subset pair corresponding to primary key 1, the data subset pair corresponding to primary key 2, the data subset pair corresponding to primary key 3, the data subset pair corresponding to primary key 4, the data subset pair corresponding to primary key 5, and the data subset pair corresponding to primary key 6. Then, the string pair of each data subset pair is obtained using the association relationship. For example, the string pair corresponding to primary key 6 is the CRC code "04940309" and the CRC code "049403TT". The string pairs of each data subset pair are compared in turn to obtain the target comparison result 710.

[0100] The embodiments provided in this application obtain M data subset pairs of multiple first data subsets in the source database and multiple second data subsets in the target database, and use the association relationship to obtain the string pair of each data subset pair, where M is a natural number; the string pair of each data subset pair is compared sequentially to obtain the target comparison result, thereby achieving the technical effect of improving the efficiency of obtaining the target comparison result.

[0101] As an optional approach, after determining at least one data subset pair with a data difference relationship using the target comparison results, the method includes: obtaining the first moment when at least one data subset pair is determined;

[0102] As an optional approach, the data in the two data subsets of at least one data subset pair are compared N times until the convergence condition is met. This includes: when the current time is the second time, comparing the data in the two data sets of at least one data subset pair N times until the convergence condition is met, wherein the time difference between the first time and the second time is a preset duration.

[0103] Optionally, in this embodiment, in order to reduce the impact of data synchronization delay on data comparison, a preset time can be set to leave a delay space of a preset time, or in other words, after the first comparison, wait for the preset time before performing the second comparison.

[0104] It should be noted that at least one data subset is obtained at the first time point when it is determined; when the current time point is the second time point, the data in the two data sets of the at least one data subset are compared N times until the convergence condition is met, wherein the time difference between the first time point and the second time point is a preset duration.

[0105] To further illustrate, optional examples include... Figure 8 As shown, assuming that during the process of synchronously migrating data from the source database 802 to the target database 804, at least two data comparisons are performed respectively, where data comparison 1 is the first comparison, data comparison 2... data comparison N is the second comparison. After data comparison 1 is completed, a second comparison can be performed after a preset time period, but is not limited to. In addition, if there are multiple second comparisons, a second comparison can be performed after the first second comparison (such as data comparison 2) is completed, but is not limited to waiting for a target time period, to reduce the impact of data synchronization delay on data comparison.

[0106] Through the embodiments provided in this application, at least one data subset pair is determined at a first moment; when the current moment is the second moment, the data in the two data sets of the at least one data subset pair are compared N times until the convergence condition is reached. The time difference between the first moment and the second moment is a preset duration, thereby achieving the technical objective of reducing the impact of data synchronization delay on data comparison, and thus realizing the technical effect of improving the accuracy of data management.

[0107] As an optional solution, for ease of understanding, the above data management method is applied to a database migration scenario. After the database migration is completed using an online migration product, the source and target databases will synchronize in real time for a period of time using data synchronization software, resulting in real-time changes in the data of the source and target databases. Optionally, in the face of the above situation, this embodiment can, but is not limited to, use a primary key combined with CRC code for efficient data comparison, plus data convergence comparison logic, to accurately locate the difference data between the source and target, thereby completing the comparison and verification of the source and target data.

[0108] Optionally, in this embodiment, data comparison is performed for multiple database types, such as... Figure 9 As shown, for databases of MySQL, Oracle, Tdsql, Tbase, and DB2 types, the first step, S1.1-S1.2, uses a multi-source data collector to extract data to be compared from the collection thread pools of the source and target databases; the second step, S1.3, organizes the data in the source and target databases according to Map<primary key, md5>.

[0109] Further, optionally in steps S2.1-S2.2, the data in the above Map<primary key, md5> format is compared using a data comparator, and the comparison results are stored in the comparison thread pool; in step S2.3, the data in the above Map<primary key, md5> format is compared a second time according to the preset convergence condition, and the comparison results are stored in the persistent thread pool; in steps S3.1-S3.2, the final result set after the comparison is completed is stored in the database.

[0110] It should be noted that the method of using primary key combined with CRC code for efficient data comparison, along with data convergence comparison logic, accurately locates the differences between the source and target data, thereby completing the comparison and verification of the source and target data.

[0111] To further illustrate, alternatively, for example... Figure 10As shown, step 1 involves retrieving data from source database 1002 and target database 1004, calculating CRC values ​​for the data according to the primary key of the specified table and the attributes of the columns, and exporting the data through the client. Step 2 involves storing the CRC (MD5) values ​​calculated from source database 1002 PK+ on the file system to form corresponding files, and storing the CRC (MD5) values ​​calculated from target database 1004 PK+ on the file system to form corresponding files. Step 3 involves comparing the generated files from source database 1002 and target database 1006. If the primary keys of source database 1002 and target database 1006 are the same, subsequent data rows are counted... Step 4: Based on the primary key value of the generated difference result set, re-query and calculate the data in the source database 1002 and the target database 1006, and store the CRC code values ​​calculated from the primary key and data row of these difference result sets in the corresponding file system; Step 5: Re-compare the CRC code values ​​calculated from the primary key extracted from the difference result set a second time, as well as the CRC code values ​​of other data columns; Steps 3, 4, and 5 can be repeated according to the number of convergence times of the result set set set set set by external settings; Step 6: Store the final result set set after comparison in the metadata database 1006.

[0112] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0113] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0114] According to another aspect of the embodiments of this application, a data management apparatus for implementing the above-described data management method is also provided. For example... Figure 11 As shown, the device includes:

[0115] The first acquisition unit 1102 is used to acquire a data set in the source database and a data set in the target database respectively, wherein the data set in the source database includes multiple first data subsets and the data set in the target database includes multiple second data subsets;

[0116] The first comparison unit 1104 is used to sequentially compare multiple first data subsets in the source database with multiple second data subsets in the target database to obtain the target comparison result.

[0117] The determining unit 1106 is used to determine at least one data subset pair with a data difference relationship using the target comparison result, wherein the data difference relationship is used to indicate that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to the difference threshold.

[0118] The second comparison unit 1108 is used to perform N comparisons on the data in the two data subsets of at least one data subset pair until the convergence condition is met, where N is a natural number.

[0119] Display unit 1110 is used to display target prompt information when the comparison results of N comparisons indicate that the data in two data subsets of at least one data subset pair has reached an abnormal condition. The target prompt information is used to indicate that the data synchronization in two data subsets of at least one data subset pair is abnormal.

[0120] Optionally, in this embodiment, the aforementioned data management device can be applied, but is not limited to, in database migration scenarios. After the database migration is completed using an online migration product, the source database and the target database will be synchronized in real time for a period of time using data synchronization software. The data in the source database and the target database are changing in real time. Traditional static data management devices are not suitable for comparing data in such dynamically changing databases. However, in this embodiment, by using efficient comparison at the data subset dimension and combining it with data convergence comparison logic, the differences between the source data and the target database can be accurately located, thereby completing the comparison and verification of the source data and the target database.

[0121] Optionally, in this embodiment, the target comparison result can be understood as, but is not limited to, the first comparison result. The first comparison result does not take into account factors such as data synchronization delay, which makes it difficult to guarantee the accuracy of the first comparison result. On the other hand, the N comparisons performed on the data in two data subsets of at least one data subset pair can be understood as multiple secondary comparisons. The secondary comparisons take into account factors such as data synchronization delay to make up for the lower accuracy of the first comparison result.

[0122] It should be noted that the process involves acquiring data sets from both the source and target databases. The source database contains multiple first data subsets, and the target database contains multiple second data subsets. The multiple first data subsets in the source database are sequentially compared with the multiple second data subsets in the target database to obtain target comparison results. These results are used to identify at least one pair of data subsets with a data difference relationship, where the data difference relationship indicates that the similarity between data in the first and second data subsets is greater than or equal to a difference threshold. The data in the two data subsets of the at least one data subset pair are compared N times until a convergence condition is met, where N is a natural number. If the comparison results of the N comparisons indicate that the data in the two data subsets of the at least one data subset pair has reached an abnormal condition, a target prompt message is displayed, indicating an anomaly in the data synchronization of the two data subsets in the at least one data subset pair.

[0123] For specific embodiments, please refer to the examples shown in the data management device above, which will not be repeated here.

[0124] The embodiments provided in this application involve obtaining a data set from a source database and a data set from a target database, wherein the data set in the source database includes multiple first data subsets, and the data set in the target database includes multiple second data subsets; the multiple first data subsets in the source database and the multiple second data subsets in the target database are sequentially compared to obtain target comparison results; at least one pair of data subsets with a data difference relationship is determined using the target comparison results, wherein the data difference relationship indicates that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to a difference threshold; the data in the two data subsets of the at least one pair of data subsets are compared N times until a convergence condition is reached, wherein N is a natural number; the comparison results of the N comparisons indicate at least one data subset. If data within two subsets of a data pair reaches an anomaly condition, a target prompt message is displayed. This prompt message indicates a synchronization anomaly in at least one subset pair between two subsets. Using data subsets as the unit of data comparison improves comparison efficiency. Furthermore, considering that the data during synchronization is dynamic and delays are normal, when a comparison result showing discrepancies is obtained, it is not directly identified as an anomaly. Instead, multiple secondary comparisons are performed until a convergence condition is met. The more accurate comparison result obtained upon reaching the convergence condition is used to determine if a synchronization anomaly exists, thereby achieving the goal of obtaining more accurate data comparison results and improving the technical effect of data management accuracy.

[0125] As an optional solution, the second comparison unit 1108 includes at least one of the following:

[0126] The first comparison module is used to compare the data in the two data sets of at least one data subset pair N times until the similarity between the data in the two data sets of at least one data subset pair is lower than the difference threshold.

[0127] The second comparison module is used to perform N comparisons on the data in the two data sets of at least one data subset pair until the number of comparisons on the data in the two data sets of at least one data subset pair is greater than or equal to the convergence threshold, where N is a natural number less than or equal to the convergence threshold.

[0128] For specific implementation examples, please refer to the examples shown in the data management method above. These examples will not be repeated here.

[0129] As an optional solution, the first comparison module includes:

[0130] The first repeating submodule is used to repeatedly perform the following steps until the similarity between the data in the two data sets of at least one data subset pair is lower than the difference threshold:

[0131] The system retrieves current data within a first target data subset from the source database based on a first set identifier of at least one data subset pair, and retrieves current data within a second target data subset from the target database based on a second set identifier of at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset.

[0132] The current data within the first target data subset and the current data within the second target data subset are compared to obtain the current comparison result;

[0133] If the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is lower than the difference threshold, then the Nth comparison is terminated.

[0134] If the current comparison result indicates that the similarity between the current data in the first data subset and the current data in the second data subset is greater than or equal to the difference threshold, the data in the first target data subset is obtained from the source database based on the first set identifier of at least one data subset pair as the current data in the first target data subset, and the data in the second target data subset is obtained from the target database based on the second set identifier of at least one data subset pair as the current data in the second target data subset.

[0135] For specific implementation examples, please refer to the examples shown in the data management method above. These examples will not be repeated here.

[0136] As an optional solution, the second comparison module includes:

[0137] The second repeating submodule is used to repeatedly execute the following steps until the number of comparisons of data from the two datasets in at least one subset pair reaches a convergence threshold:

[0138] The system retrieves current data within a first target data subset from the source database based on a first set identifier of at least one data subset pair, and retrieves current data within a second target data subset from the target database based on a second set identifier of at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset.

[0139] The current data in the first target data subset and the current data in the second target data subset are compared to obtain the current comparison result and determine the current comparison count;

[0140] If the current number of alignments is greater than or equal to the convergence threshold, terminate the N alignments.

[0141] If the current number of comparisons is less than the convergence threshold, data within the first target data subset is obtained from the source database based on the first set identifier of at least one data subset pair as the current data within the first target data subset, and data within the second target data subset is obtained from the target database based on the second set identifier of at least one data subset pair as the current data within the second target data subset.

[0142] For specific implementation examples, please refer to the examples shown in the data management method above. These examples will not be repeated here.

[0143] As an optional solution, the apparatus further includes: a first acquisition submodule, configured to perform N comparisons on data in two data sets of at least one data subset pair until a convergence condition is met, and acquire N comparison results from the N comparisons; and to integrate the N comparison results to obtain the comparison result to be judged.

[0144] Display unit 1110 includes: display module, used to display target prompt information when the comparison result to be judged meets the abnormal conditions.

[0145] For specific implementation examples, please refer to the examples shown in the data management method above. These examples will not be repeated here.

[0146] As an optional solution, the second comparison unit 1108 includes:

[0147] The repeat module is used to repeatedly execute the following steps until the convergence condition is met:

[0148] The system retrieves current data within a first target data subset from the source database based on a first set identifier of at least one data subset pair, and retrieves current data within a second target data subset from the target database based on a second set identifier of at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset.

[0149] The current data in the first target data subset and the current data in the second target data subset are compared to obtain the current comparison result and determine the current comparison count;

[0150] If the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is less than the anomaly threshold, or the current number of comparisons is greater than or equal to the convergence threshold, then the convergence condition is determined to have been met.

[0151] If the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is greater than or equal to the anomaly threshold, and the current number of comparisons has not reached the convergence threshold, then the data in the first target data subset is obtained from the source database based on the first set identifier of at least one data subset pair as the current data in the first target data subset, and the data in the second target data subset is obtained from the target database based on the second set identifier of at least one data subset pair as the current data in the second target data subset.

[0152] For specific implementation examples, please refer to the examples shown in the data management method above. These examples will not be repeated here.

[0153] As an optional solution, the apparatus further includes: a processing unit, used to convert the data in each data subset of the source database and the target database into strings and establish the association between each string and each data subset before acquiring the data set in the source database and the data set in the target database respectively;

[0154] The first comparison unit 1104 includes: a third comparison module, used to sequentially compare multiple first data subsets in the source database with multiple second data subsets in the target database using strings and association relationships, to obtain the target comparison result.

[0155] For specific implementation examples, please refer to the examples shown in the data management method above. These examples will not be repeated here.

[0156] As an optional solution, the third comparison module includes:

[0157] The second acquisition submodule is used to acquire M data subset pairs between multiple first data subsets in the source database and multiple second data subsets in the target database, and to acquire string pairs of each data subset pair in the M data subset pairs using the association relationship, where M is a natural number;

[0158] The comparison submodule is used to compare the string pairs of each data subset in turn to obtain the target comparison result.

[0159] For specific implementation examples, please refer to the examples shown in the data management method above. These examples will not be repeated here.

[0160] As an optional solution, the apparatus further includes: a second acquisition unit, configured to acquire the first moment when at least one data subset pair is determined after determining at least one data subset pair with a data difference relationship using the target comparison result;

[0161] The second comparison unit 1108 includes: a fourth comparison module, used to perform N comparisons on the data in two data sets of at least one data subset pair when the current time is the second time, until the convergence condition is reached, wherein the time difference between the first time and the second time is a preset duration.

[0162] For specific implementation examples, please refer to the examples shown in the data management method above. These examples will not be repeated here.

[0163] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described data management method is also provided, such as... Figure 12 As shown, the electronic device includes a memory 1202 and a processor 1204. The memory 1202 stores a computer program, and the processor 1204 is configured to execute the steps of any of the above method embodiments through the computer program.

[0164] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0165] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0166] S1, obtain the data set in the source database and the data set in the target database respectively, wherein the data set in the source database includes multiple first data subsets and the data set in the target database includes multiple second data subsets;

[0167] S2, compare multiple first data subsets in the source database with multiple second data subsets in the target database in turn to obtain the target comparison result;

[0168] S3, using the target comparison results to determine at least one data subset pair with a data difference relationship, wherein the data difference relationship is used to indicate that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to the difference threshold;

[0169] S4, compare the data in the two data subsets of at least one data subset pair N times until the convergence condition is met, where N is a natural number;

[0170] S5, if the comparison results of N comparisons indicate that the data in two data subsets of at least one data subset pair has reached an abnormal condition, display the target prompt information, wherein the target prompt information is used to indicate that the data synchronization in two data subsets of at least one data subset pair is abnormal.

[0171] Alternatively, as those skilled in the art will understand, Figure 12The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 12 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 12 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 12 The different configurations shown.

[0172] The memory 1202 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data management method and apparatus in this embodiment. The processor 1204 executes various functional applications and data processing by running the software programs and modules stored in the memory 1202, thereby realizing the aforementioned data management method. The memory 1202 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1202 may further include memory remotely located relative to the processor 1204, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1202 may be used, but is not limited to, to store information such as a first data subset, a second data subset, target comparison results, and target prompt information. As an example, such as... Figure 12 As shown, the memory 1202 may include, but is not limited to, the first acquisition unit 1102, the first comparison unit 1104, the determination unit 1106, the second comparison unit 1108, and the display unit 1110 of the data management device. Furthermore, it may include, but is not limited to, other module units of the data management device, which will not be elaborated upon in this example.

[0173] Optionally, the transmission device 1206 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1206 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1206 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0174] In addition, the aforementioned electronic device also includes: a display 1208 for displaying the first data subset, the second data subset, the target comparison results, and target prompt information; and a connection bus 1210 for connecting the various module components in the aforementioned electronic device.

[0175] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0176] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in embodiments of this application.

[0177] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0178] It should be noted that the computer system of the electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0179] A computer system includes a Central Processing Unit (CPU), which performs various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) or loaded from RAM. ROM also stores various programs and data required for system operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output interfaces (I / O interfaces) are also connected to the bus.

[0180] The following components are connected to the input / output interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard drives; and communication sections including network interface cards such as LAN cards and modems. The communication section performs communication processing via a network such as the Internet. Drives are also connected to the input / output interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0181] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions defined in the system of this application.

[0182] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of a computer device reads computer instructions from the computer-readable storage medium, and executes the computer instructions to cause the computer device to perform the methods provided in the various alternative implementations described above.

[0183] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0184] S1, obtain the data set in the source database and the data set in the target database respectively, wherein the data set in the source database includes multiple first data subsets and the data set in the target database includes multiple second data subsets;

[0185] S2, compare multiple first data subsets in the source database with multiple second data subsets in the target database in turn to obtain the target comparison result;

[0186] S3, using the target comparison results to determine at least one data subset pair with a data difference relationship, wherein the data difference relationship is used to indicate that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to the difference threshold;

[0187] S4, compare the data in the two data subsets of at least one data subset pair N times until the convergence condition is met, where N is a natural number;

[0188] S5, if the comparison results of N comparisons indicate that the data in two data subsets of at least one data subset pair has reached an abnormal condition, display the target prompt information, wherein the target prompt information is used to indicate that the data synchronization in two data subsets of at least one data subset pair is abnormal.

[0189] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0190] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0191] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0192] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0193] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0194] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0195] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0196] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data management method, characterized in that, include: The data sets in the source database and the target database are obtained respectively, wherein the data set in the source database includes multiple first data subsets and the data set in the target database includes multiple second data subsets; The plurality of first data subsets in the source database are compared sequentially with the plurality of second data subsets in the target database to obtain the target comparison result; Using the target comparison results, at least one data subset pair with a data difference relationship is determined, wherein the data difference relationship is used to indicate that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to a difference threshold; The data within the two data subsets in the at least one data subset pair are compared N times until the convergence condition is met, where N is a natural number; If the comparison results of the N comparisons indicate that the data in two data subsets of the at least one data subset pair has reached an abnormal condition, a target prompt message is displayed, wherein the target prompt message is used to indicate that the data synchronization in two data subsets of the at least one data subset pair is abnormal; The step of performing N comparisons on the data within two data subsets of the at least one data subset pair until a convergence condition is met includes at least one of the following: performing the N comparisons on the data within two data sets of the at least one data subset pair until the similarity between the data within the two data sets of the at least one data subset pair is lower than the difference threshold; or performing the N comparisons on the data within two data sets of the at least one data subset pair until the number of comparisons on the data within the two data sets of the at least one data subset pair is greater than or equal to a convergence threshold, where N is a natural number less than or equal to the convergence threshold.

2. The method according to claim 1, characterized in that, The step of performing the N comparisons on the data within the two data sets in the at least one data subset pair until the similarity between the data within the two data sets in the at least one data subset pair is lower than the difference threshold includes: Repeat the following steps until the similarity between the data in the two data sets of the at least one data subset pair is lower than the difference threshold: The system retrieves current data within a first target data subset from the source database based on a first set identifier of the at least one data subset pair, and retrieves current data within a second target data subset from the target database based on a second set identifier of the at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset. The current data within the first target data subset and the current data within the second target data subset are compared to obtain the current comparison result; If the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is lower than the difference threshold, the N comparisons are terminated. If the current comparison result indicates that the similarity between the current data in the first data subset and the current data in the second data subset is greater than or equal to the difference threshold, the data in the first target data subset is obtained from the source database as the current data in the first target data subset based on the first set identifier of the at least one data subset pair, and the data in the second target data subset is obtained from the target database as the current data in the second target data subset based on the second set identifier of the at least one data subset pair.

3. The method according to claim 1, characterized in that, The step of performing the N comparisons on the data within the two data sets of the at least one data subset pair until the number of comparisons on the data within the two data sets of the at least one data subset pair is greater than or equal to the convergence threshold includes: Repeat the following steps until the number of comparisons of data within the two data sets in the at least one data subset pair reaches the convergence threshold: The system retrieves current data within a first target data subset from the source database based on a first set identifier of the at least one data subset pair, and retrieves current data within a second target data subset from the target database based on a second set identifier of the at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset. The current data in the first target data subset and the current data in the second target data subset are compared to obtain the current comparison result and determine the current comparison count; If the current number of alignments is greater than or equal to the convergence threshold, the N alignments are terminated. If the current number of comparisons is less than the convergence threshold, data within the first target data subset is obtained from the source database based on the first set identifier of the at least one data subset pair as the current data within the first target data subset, and data within the second target data subset is obtained from the target database based on the second set identifier of the at least one data subset pair as the current data within the second target data subset.

4. The method according to claim 3, characterized in that, After performing N comparisons on the data within two data sets in the at least one data subset pair until a convergence condition is met, the process includes: obtaining N comparison results from the N comparisons; and integrating the N comparison results to obtain the comparison result to be judged. The step of displaying target prompt information when the comparison results of the N comparisons meet the abnormal conditions includes: displaying the target prompt information when the comparison result to be judged meets the abnormal conditions.

5. The method according to claim 1, characterized in that, The step of performing N comparisons on the data within the two data sets in the at least one data subset pair until a convergence condition is met includes: Repeat the following steps until the convergence condition is met: The system retrieves current data within a first target data subset from the source database based on a first set identifier of the at least one data subset pair, and retrieves current data within a second target data subset from the target database based on a second set identifier of the at least one data subset pair, wherein the first set identifier is the identifier of the first target data subset, and the second set identifier is the identifier of the second target data subset. The current data in the first target data subset and the current data in the second target data subset are compared to obtain the current comparison result and determine the current comparison count; If the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is less than the anomaly threshold, or the current number of comparisons is greater than or equal to the convergence threshold, then the convergence condition is determined to be met. If the current comparison result indicates that the similarity between the current data in the first target data subset and the current data in the second target data subset is greater than or equal to the anomaly threshold, and the current number of comparisons has not reached the convergence threshold, then the data in the first target data subset is obtained from the source database as the current data in the first target data subset based on the first set identifier of the at least one data subset pair, and the data in the second target data subset is obtained from the target database as the current data in the second target data subset based on the second set identifier of the at least one data subset pair.

6. The method according to claim 1, characterized in that, Before obtaining the data sets in the source database and the target database respectively, the method includes: converting the data in each data subset of the source database and the target database into strings, and establishing the association between each string and each data subset; The step of sequentially comparing the plurality of first data subsets in the source database with the plurality of second data subsets in the target database to obtain the target comparison result includes: using the string and the association relationship to sequentially compare the plurality of first data subsets in the source database with the plurality of second data subsets in the target database to obtain the target comparison result.

7. The method according to claim 6, characterized in that, The step of sequentially comparing the plurality of first data subsets in the source database with the plurality of second data subsets in the target database using the string and the association relationship to obtain the target comparison result includes: Obtain M data subset pairs of the plurality of first data subsets in the source database and the plurality of second data subsets in the target database, and use the association relationship to obtain a string pair of each of the M data subset pairs, where M is a natural number; The string pairs of each data subset are compared sequentially to obtain the target comparison result.

8. The method according to any one of claims 1 to 7, characterized in that, After determining at least one data subset pair with a data difference relationship using the target comparison result, the method includes: obtaining the first moment when the at least one data subset pair is determined; The step of performing N comparisons on the data within the two data subsets of the at least one data subset pair until the convergence condition is met includes: when the current time is the second time, performing the N comparisons on the data within the two data sets of the at least one data subset pair until the convergence condition is met, wherein the time difference between the first time and the second time is a preset duration.

9. A data management device, characterized in that, include: The first acquisition unit is used to acquire a data set in the source database and a data set in the target database respectively, wherein the data set in the source database includes multiple first data subsets and the data set in the target database includes multiple second data subsets; The first comparison unit is used to sequentially compare the plurality of first data subsets in the source database with the plurality of second data subsets in the target database to obtain the target comparison result; The determining unit is used to determine at least one data subset pair with a data difference relationship using the target comparison result, wherein the data difference relationship is used to indicate that the similarity between the data in the first data subset and the data in the second data subset is greater than or equal to the difference threshold; The second comparison unit is used to perform N comparisons on the data in the two data subsets of the at least one data subset pair until the convergence condition is met, where N is a natural number. The display unit is configured to display target prompt information when the comparison results of the N comparisons indicate that the data in two data subsets of the at least one data subset pair has reached an abnormal condition, wherein the target prompt information is used to indicate that the data synchronization in two data subsets of the at least one data subset pair is abnormal. The second comparison unit includes at least one of the following: a first comparison module, configured to perform the N comparisons on the data within the two data sets of the at least one data subset pair until the similarity between the data within the two data sets of the at least one data subset pair is lower than the difference threshold; and a second comparison module, configured to perform the N comparisons on the data within the two data sets of the at least one data subset pair until the number of comparisons on the data within the two data sets of the at least one data subset pair is greater than or equal to a convergence threshold, wherein N is a natural number less than or equal to the convergence threshold.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program can be executed by a terminal device or computer at runtime as described in any one of claims 1 to 8.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 8.

12. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 8 through the computer program.

Images