Data verification method, device, equipment and storage medium

Abstract

The embodiments of the present application disclose a data verification method, apparatus, device and storage medium. The method includes: obtaining verification task information, the verification task information includes clue data of the data crawler corresponding to the target data and the data relationship spectrum of the target data, the clue data includes the identification and acquisition method information of the target data, the data relationship spectrum includes the correspondence information between the metadata and the data crawler, and the relationship information between the metadata; according to the correspondence information, determining the data crawler corresponding to the metadata; obtaining the target data through the data crawler according to the identification and acquisition method information of the target data, the target data including at least two metadata; verifying the relationship between at least two metadata according to the relationship information between the at least two metadata and the metadata, and obtaining a verification result. The method provided by the embodiments of the present application can ensure the accuracy of the data.

Description

Technical Field

[0001] The present application belongs to the field of data verification technology, and in particular relates to a data verification method, device, equipment and storage medium. Background Art

[0002] Data is generated and transmitted in Information Technology (IT) systems. Accurate data is a prerequisite for IT systems to provide accurate and efficient IT services, so it is very important to ensure the accuracy of data in the system.

[0003] Currently, because data in a system often has logical relationships, most solutions use a proactive approach to ensure data accuracy. During database design, they ensure that the database adheres to a certain paradigm to ensure the accuracy of relationships between tables. Database transactions are also introduced to guarantee data accuracy. Database transactions ensure database consistency even under abnormal conditions and provide isolation between multiple applications when accessing the database concurrently, preventing interference between their operations.

[0004] The current solution requires embedding applications into data processing processes and introducing database transactions, which reduces system performance and efficiency. Therefore, designers often only use database transactions to process critical data, which cannot guarantee the accuracy of all data. Summary of the Invention

[0005] The embodiments of the present application provide a data verification method, apparatus, device, and storage medium, which can ensure the accuracy of data by verifying the relationship between data.

[0006] In a first aspect, an embodiment of the present application provides a data verification method, the method comprising:

[0007] Obtain verification task information, which includes clue data of the data crawler corresponding to the target data and a data relationship spectrum of the target data. The clue data includes information about the identification and acquisition method of the target data. The data relationship spectrum includes information about the correspondence between metadata and the data crawler, as well as information about the relationship between metadata.

[0008] According to the corresponding relationship information, obtain the data crawler corresponding to the metadata;

[0009] Acquire target data using a data crawler according to the identifier and acquisition method information of the target data, where the target data includes at least two metadata;

[0010] The relationship between the at least two metadata is verified according to the at least two metadata and the relationship information between the metadata to obtain a verification result.

[0011] In one possible implementation, before obtaining the verification task information, the method further includes:

[0012] Obtaining the identifier of the data crawler, the identifier of the metadata, the correspondence information between the data crawler and the metadata, and the relationship information between the metadata;

[0013] A data relationship spectrum is generated based on the identifier of the data crawler, the identifier of the metadata, the correspondence information between the data crawler and the metadata, and the relationship information between the metadata.

[0014] In one possible implementation, obtaining verification task information includes:

[0015] When the target data changes, obtain the data relationship spectrum of the target data;

[0016] Generate clue data for the data crawler in the data relationship spectrum based on the identifier of the target data and the data relationship spectrum;

[0017] Generate verification task information based on clue data and data relationship spectrum.

[0018] In one possible implementation, based on the identifier of the target data and the data relationship spectrum, the clue data of the data crawler in the data relationship spectrum is generated, including:

[0019] When the target data changes, obtain the change record information when the target data changes;

[0020] According to the identifier of the target data, the data relationship spectrum and the change record information when the target data is changed, the clue data of the data crawler in the data relationship spectrum is generated.

[0021] In one possible implementation, the clue data includes a metadata identifier, a primary key of the metadata in a data table, and an index. A data crawler, based on the identifier and acquisition method information of the target data, obtains at least two metadata items, including:

[0022] By using a data crawler, at least two metadata are obtained from the data table according to the metadata identifier, the primary key of the metadata in the data table, and the index.

[0023] In a second aspect, an embodiment of the present application provides a data verification device, the device comprising:

[0024] An acquisition module is configured to acquire verification task information, the verification task information including clue data of a data crawler corresponding to the target data and a data relationship spectrum of the target data, the clue data including the identification and acquisition method information of the target data, and the data relationship spectrum including the correspondence information between metadata and the data crawler and the relationship information between the metadata; further configured to acquire the data crawler corresponding to the metadata based on the correspondence information; and further configured to acquire the target data through the data crawler based on the identification and acquisition method information of the target data, wherein the target data includes at least two metadata;

[0025] The verification module is used to verify the relationship between at least two metadata based on the at least two metadata and the relationship information between the metadata to obtain a verification result.

[0026] In one possible implementation, the acquisition module is used to obtain the identification of the data crawler, the identification of the metadata, the correspondence information between the data crawler and the metadata, and the relationship information between the metadata before obtaining the verification task information; and is also used to generate a data relationship spectrum based on the identification of the data crawler, the identification of the metadata, the correspondence information between the data crawler and the metadata, and the relationship information between the metadata.

[0027] In one possible implementation, the module is obtained, specifically for:

[0028] When the target data changes, obtain the data relationship spectrum of the target data;

[0029] Generate clue data for the data crawler in the data relationship spectrum based on the identifier of the target data and the data relationship spectrum;

[0030] Generate verification task information based on clue data and data relationship spectrum.

[0031] In a possible implementation, the method further includes:

[0032] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the method in the first aspect or any possible implementation of the first aspect.

[0033] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the method in the first aspect or any possible implementation of the first aspect is implemented.

[0034] The technical solutions provided by the embodiments of the present application bring at least the following beneficial effects:

[0035] In an embodiment of the present application, after obtaining the verification task information, the data crawler corresponding to the metadata is determined based on the correspondence information between the metadata and the data crawler in the verification task information, and then the crawler obtains at least two metadata based on the identification and acquisition method information of the target data. These metadata are data that need to verify the relationship between them. The verification task information includes the relationship information between the at least two metadata. The relationship between the at least two metadata is verified based on the relationship information to obtain a verification result. When the verification result includes that the relationship between the at least two metadata is accurate, it indicates that the at least two metadata are accurate; when the verification result includes that the relationship between the at least two metadata is inaccurate, it indicates that there is erroneous metadata in the at least two metadata, and the erroneous metadata is processed, thereby ensuring the accuracy of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] Figure 1 This is a flow chart of a data verification method provided in an embodiment of the present application;

[0038] Figure 2 This is a flow chart of another data verification method provided in an embodiment of the present application;

[0039] Figure 3 This is a schematic diagram of a data relationship spectrum provided in an embodiment of the present application;

[0040] Figure 4 This is a flow chart of another data verification method provided in an embodiment of the present application;

[0041] Figure 5 This is a structural diagram of a data verification device provided in an embodiment of the present application;

[0042] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0044] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0045] Data is generated and transmitted within IT systems. Accurate data is a prerequisite for IT systems to provide accurate and efficient IT services. Logical relationships often exist between data within a system, so a proactive approach is often adopted during database design. This ensures that the database adheres to certain paradigms to ensure the accuracy of relationships between tables within the database. Database transactions are also introduced to guarantee data accuracy. However, these solutions require embedding them within the application's data processing flow and introducing database transactions, which reduces system performance and efficiency. Consequently, designers often only use database transactions for critical data, failing to guarantee the accuracy of all data.

[0046] An embodiment of the present application provides a method for verifying data relationships. When data changes, a data crawler is used to capture data, and the relationship between the data is verified based on the relationship information between the data, thereby ensuring the accuracy of the data.

[0047] The execution subject of the method provided in the embodiment of the present application is a device such as a server or a computer with data transmission and data processing capabilities.

[0048] The following will be combined Figure 1 A method for verifying data relationships provided by an embodiment of the present application is described in detail. Figure 1 As shown, the method may include the following steps:

[0049] S110, obtaining verification task information.

[0050] According to the pre-set cycle for obtaining verification task information, verification task information is obtained once in each cycle. The verification task information may include clue data of the data crawler corresponding to the target data and the data relationship spectrum of the target data. The clue data includes the identification and acquisition method information of the target data. The data relationship spectrum includes the correspondence information between the metadata and the data crawler, as well as the relationship information between the metadata.

[0051] S120: Determine the data crawler corresponding to the metadata according to the corresponding relationship information.

[0052] The device stores a data crawler, and the corresponding relationship information includes the identifier of the metadata, the identifier of the data crawler corresponding to the metadata, and the corresponding relationship between the metadata and the data crawler. The data crawler corresponding to the metadata is determined based on the corresponding relationship.

[0053] S130, acquiring target data through a data crawler according to the identifier of the target data and the acquisition method information.

[0054] The target data may include at least two metadata.

[0055] The clue data used by the same data crawler within the same data relationship spectrum is the same. The clue data includes the identification of the target data and the acquisition method information, which can ensure that the data crawler captures the accurate metadata from the accurate storage location.

[0056] In some embodiments, the acquisition method information may include the database and data table storing the target data and the query conditions for querying the metadata. A connection is first established with the database, and a data crawler corresponding to the metadata is used to query and crawl the metadata corresponding to the data crawler from the database and data table where the metadata is located, based on the query conditions and the metadata identifier.

[0057] S140 , verifying the relationship between at least two metadata according to the at least two metadata and the relationship information between the metadata, and obtaining a verification result.

[0058] The relationship information may include a logical relationship between at least two metadata, and the logical relationship between at least two metadata is an object of verification.

[0059] In one example, the metadata is the expiration date of a product and the expiration date of a discount. Logically, the discount is subordinate to the product, so the discount's expiration date must be within the product's expiration date. The logical relationship can be described as: 1. The discount's start time is greater than or equal to the product's start time; 2. The discount's end time is less than or equal to the product's end time.

[0060] Based on the at least two metadata and the relationship information between the metadata, the relationship between the at least two metadata is verified. When the relationship between the at least two metadata is consistent with the relationship in the relationship information center, it indicates that the at least two metadata are accurate; when the relationship between the at least two metadata is inconsistent with the relationship in the relationship information center, it indicates that erroneous data exists in the at least two metadata, and corresponding processing measures can be taken for the erroneous data to ensure data accuracy.

[0061] In an embodiment of the present application, after obtaining the verification task information, the data crawler corresponding to the metadata is determined based on the correspondence information between the metadata and the data crawler in the verification task information, and then the crawler obtains at least two metadata based on the identification and acquisition method information of the target data. These metadata are data that need to verify the relationship between them. The verification task information includes the relationship information between the at least two metadata, so the relationship between the at least two metadata is verified based on the relationship information to obtain a verification result. When the relationship between the at least two metadata is consistent with the relationship of the relationship information center, it means that the at least two metadata are accurate; when the relationship between the at least two metadata is inconsistent with the relationship of the relationship information center, it means that there is erroneous data in the at least two metadata, and corresponding measures can be taken to process the erroneous data, thereby ensuring the accuracy of the data.

[0062] In one embodiment, Figure 2 As shown, before obtaining the verification task information, that is, before S110, the method further includes:

[0063] S150, obtaining the identifier of the data crawler, the identifier of the metadata, the correspondence information between the data crawler and the metadata, and the relationship information between the metadata.

[0064] The correspondence information between the data crawler and the metadata may include the correspondence between the data crawler and the metadata, and the data crawler corresponding to the metadata can crawl the metadata. The relationship information between the metadata may include the metadata's equal, greater than, less than, mutually exclusive or dependent relationships.

[0065] In one example, it is necessary to verify the relationship between the validity period of a product and the validity period of a discount, and the metadata includes the product's validity start time, the product's validity end time, the discount's validity start time, and the discount's validity end time. The relationship information between the metadata includes that the discount's validity start time is greater than or equal to the product's validity start time, and that the discount's validity end time is less than or equal to the product's validity end time.

[0066] Obtain the identifier of each of the above metadata, the identifier of the data crawler, the corresponding relationship information between the data crawler and the above metadata, and the relationship information between the above metadata.

[0067] S160 , generating a data relationship spectrum according to the identifier of the data crawler, the identifier of the metadata, the correspondence information between the data crawler and the metadata, and the relationship information between the metadata.

[0068] In some embodiments, the generated data relationship spectrum can be as follows: Figure 3 As shown, the data relationship spectrum may include n relationship spectra, where n is an integer greater than 0. In relationship spectrum 1, crawler A corresponds to metadata 1 and metadata 2, crawler B corresponds to metadata 1, and crawler C corresponds to metadata 1, metadata 2, and metadata 3. "Metadata 1 = metadata 1" indicates that metadata 1 and metadata 1 have the same relationship.

[0069] In practical applications, relationship spectra can be added or deleted in the data relationship spectrum as required.

[0070] The data relationship spectrum generated in the embodiment of the present application describes the relationship information between metadata, and also provides a basis for verifying the relationship between metadata. It can also dynamically add or delete the relationship between metadata in the data relationship spectrum according to the requirements of actual applications, thereby simply and conveniently and comprehensively ensuring the accuracy of the data.

[0071] In some embodiments, as Figure 4 As shown, obtaining verification task information, that is, S110, may specifically include:

[0072] S1101, when target data changes, obtain the data relationship spectrum of the target data.

[0073] When the target data undergoes changes such as addition, deletion or modification, a data relationship spectrum including the identifier of the target data is obtained.

[0074] In some embodiments, the target data is monitored by Oracle Golden Gate (OGG) synchronization technology, and when the target data changes, a data relationship spectrum including an identifier of the target data is obtained.

[0075] S1102: Generate clue data for the data crawler in the data relationship spectrum according to the identifier of the target data and the data relationship spectrum.

[0076] Obtain the identification and acquisition method information of the target data, and generate clue data corresponding to the data crawler represented by the identification of the data crawler in the data relationship spectrum based on the identification of the target data, the acquisition method information and the identification of the data crawler in the data relationship spectrum.

[0077] S1103: Generate verification task information based on the clue data and the data relationship spectrum.

[0078] The clue data of the target data is combined with the data relationship spectrum of the target data to generate verification task information, where the verification task information includes the clue data and the data relationship spectrum.

[0079] In the embodiment of the present application, after the target data changes, verification task information of the target data is generated based on the clue data and data relationship spectrum of the target data, so that the task of verifying the target data is triggered in time after the target data changes, and the target data is verified in time to ensure the accuracy of the data.

[0080] In some embodiments, generating verification task information based on clue data and data relationship spectrum, i.e., S1103, may specifically include:

[0081] First, when the target data changes, the change record information of the target data change is obtained.

[0082] In some embodiments, when the metadata in the product table and the offer table in the database changes, the target data is the validity start time and validity end time of the product, as well as the validity start time and validity end time of the offer. Through OGG synchronization technology, when the metadata in the product table and the offer table in the database are added, deleted, or modified, the change record information when the metadata in the product table and the offer table changes is obtained, for example, the number, product ID and order unique identifier in the relevant record information when the metadata in the product table and the offer table changes are obtained.

[0083] Secondly, based on the identifier of the target data, the data relationship spectrum and the change record information when the target data changes, the clue data of the data crawler in the data relationship spectrum is generated.

[0084] In some embodiments, to obtain the validity start time and validity end time of a product ordered by a user, it is necessary to obtain the validity start time and validity end time of the product from the product table based on the change record information such as the number, product ID and subscription unique identifier corresponding to the identifiers of the validity start time and validity end time of the product, and the primary key and index of the validity start time and validity end time of the product in the product table. Therefore, the clue data of the product data crawler includes the identifiers of the validity start time and validity end time of the product, the number, product ID, subscription unique identifier and other change record information, as well as the primary key and index of the validity start time and validity end time of the product in the product table.

[0085] To obtain the validity start time and validity end time of a discount corresponding to a product ordered by a user, it is necessary to obtain the validity start time and validity end time of the discount from the discount table based on the change record information such as the number, product ID, discount ID, discount subscription unique identifier corresponding to the identifier of the validity start time and validity end time of the product, and the primary key and index of the validity start time and validity end time of the discount in the product table. Therefore, the clue data of the discount data crawler includes the identifier of the validity start time and validity end time of the discount, the number, product ID, discount ID, discount subscription unique identifier and other change record information, as well as the primary key and index of the validity start time and validity end time of the discount in the product table.

[0086] Generally speaking, lead data does not change after it is generated.

[0087] The method provided in the embodiment of the present application generates clue data for a data crawler of metadata when target data changes, so that the data crawler can capture metadata based on the clue data.

[0088] In some embodiments, acquiring target data by a data crawler according to the identifier and acquisition method information of the target data, i.e., S130, may specifically include:

[0089] By using a data crawler, at least two metadata are obtained from the data table according to the metadata identifier, the primary key of the metadata in the data table, and the index.

[0090] In some embodiments, the target data is stored in a relational database, and the method for obtaining at least two metadata from a data table in the database includes the following steps:

[0091] First, establish a connection with the relational database based on the network access address, user name, password and other information of the relational database connection.

[0092] Next, the crawler parses the preconfigured acquisition method for the relational database. This method can be configured to retrieve metadata using Structured Query Language (SQL) statements. An executable SQL statement contains the specific database, data table, and query conditions for the statement execution. The query conditions can include the primary key and index of the data table. By executing the SQL statement, the data crawler retrieves at least two pieces of metadata from the data table based on the database, data table, primary key, and index contained in the SQL statement.

[0093] In an example, the data crawler is a product data crawler, and the clue data of the product data crawler is: 13836765391, pl_mgkytx, 39875635264, where 13836765391 is the primary key, pl_mgkytx is the product identifier, and 9875635264 is the index. The executable statement of the product data crawler is select startdate, enddate from subs_product where serv=13836765391and prodid=pl_mgkytx and oid=39875635264, where startdate and enddate are metadata identifiers, subs_product represents the product data table, serv represents the primary key, prodid represents the product identifier, and oid represents the index.

[0094] In some embodiments, the target data is stored in a non-relational database, and the method for obtaining metadata from the database includes the following steps:

[0095] First, establish a connection with the non-relational database based on the network access address, user name, password and other information of the non-relational database connection.

[0096] Then, the acquisition method pre-configured for the non-relational database is parsed, and metadata is acquired from the non-relational database through the pre-configured acquisition method.

[0097] In some embodiments, the target data is stored in multiple databases, and the method for obtaining metadata from the databases includes the following steps:

[0098] First, establish a connection with each database based on the network access address, user name, password and other information of each database connection.

[0099] Then, the acquisition method pre-configured for the database is parsed, and metadata is obtained from each database through the pre-configured acquisition method.

[0100] The embodiment of the present application provides a metadata acquisition mode, namely, first establishing a connection with the database, then parsing a pre-configured acquisition method, and executing the statement corresponding to the acquisition method, so that metadata can be acquired from the database through a data crawler. No modification is required when acquiring metadata from different databases, effectively reducing costs. After acquiring at least two metadata from a data table, it is convenient to verify the relationship between the at least two metadata.

[0101] The present application also provides a data verification device, such as Figure 5 As shown, the apparatus 500 may include an acquisition module 510 and a verification module 520 .

[0102] Acquisition module 510 is used to obtain verification task information, which includes clue data of the data crawler corresponding to the target data and a data relationship spectrum of the target data, the clue data includes the identification and acquisition method information of the target data, and the data relationship spectrum includes the correspondence information between the metadata and the data crawler and the relationship information between the metadata; it is also used to obtain the data crawler corresponding to the metadata based on the correspondence information; it is also used to obtain the target data through the data crawler based on the identification and acquisition method information of the target data, and the target data includes at least two metadata.

[0103] The verification module 520 is configured to verify the relationship between at least two metadata based on the at least two metadata and the relationship information between the metadata to obtain a verification result.

[0104] After obtaining the verification task information, the device provided in the embodiment of the present application determines the data crawler corresponding to the metadata based on the correspondence information between the metadata and the data crawler in the verification task information, and then the data crawler obtains at least two metadata based on the identification and acquisition method information of the target data. These metadata are data that need to verify the relationship between them. The verification task information includes the relationship information between the at least two metadata, so the relationship between the at least two metadata is verified based on the relationship information to obtain a verification result. When the relationship between the at least two metadata is consistent with the relationship of the relationship information center, it means that the at least two metadata are accurate; when the relationship between the at least two metadata is inconsistent with the relationship of the relationship information center, it means that there is erroneous data in the at least two metadata, and corresponding measures can be taken to process the erroneous data, thereby ensuring the accuracy of the data.

[0105] In some embodiments, the acquisition module 510 can also be used to obtain the identification of the data crawler, the identification of the metadata, the correspondence information between the data crawler and the metadata, and the relationship information between the metadata; it can also be used to generate a data relationship spectrum based on the identification of the data crawler, the identification of the metadata, the correspondence information between the data crawler and the metadata, and the relationship information between the metadata.

[0106] The data relationship spectrum generated by the device provided in the embodiment of the present application describes the relationship between metadata, provides a basis for verifying the relationship between metadata, and comprehensively ensures data accuracy in a simple and convenient manner.

[0107] In some embodiments, the acquisition module 510 may be specifically configured to:

[0108] When the target data changes, obtain the data relationship spectrum of the target data;

[0109] Generate clue data for the data crawler in the data relationship spectrum based on the identifier of the target data and the data relationship spectrum;

[0110] Generate verification task information based on clue data and data relationship spectrum.

[0111] In the embodiment of the present application, after the target data changes, verification task information of the target data is generated based on the clue data and data relationship spectrum of the target data, so that the verification of the target data is triggered in time after the target data changes, and the target data is verified in time to ensure the accuracy of the data.

[0112] In some embodiments, the acquisition module 510 may also be specifically configured to:

[0113] When the target data changes, obtain the change record information when the target data changes;

[0114] According to the identifier of the target data, the data relationship spectrum and the change record information when the target data is changed, the clue data of the data crawler in the data relationship spectrum is generated.

[0115] The device provided in the embodiment of the present application generates clue data for the data crawler of metadata when the target data changes, so that the data crawler can capture metadata based on the clue data.

[0116] In some embodiments, the clue data includes the metadata identifier, the primary key and index of the metadata in the data table. The acquisition module 510 may also be specifically used to:

[0117] By using a data crawler, at least two metadata are obtained from the data table according to the metadata identifier, the primary key of the metadata in the data table, and the index.

[0118] The embodiment of the present application provides a metadata acquisition mode for acquiring metadata from a database to facilitate verification of the relationship between the metadata.

[0119] The device for verifying data relationship provided in the embodiment of the present application performs Figure 1-2 and Figure 4 The various steps in the method shown can achieve the technical effect of ensuring the accuracy of the data. For the sake of brevity, they will not be described in detail here.

[0120] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0121] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0122] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0123] The memory 602 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 602 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory. In a specific embodiment, the memory 602 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0124] The processor 601 reads and executes computer program instructions stored in the memory 602 to implement any one of the methods for verifying data relationships in the embodiments shown in the figure.

[0125] In one example, the electronic device may further include a communication interface 603 and a bus 610. Figure 6 As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 610 and communicate with each other.

[0126] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0127] Bus 610 comprises hardware, software or both, couples the parts of electronic equipment to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus 610 can comprise one or more buses.Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0128] The electronic device can execute the method for verifying data relationship in the embodiment of the present application, thereby realizing the combination Figure 1-2 and Figure 4 Describes the method for verifying data relationships.

[0129] In addition, in conjunction with the method for verifying data relationships in the above embodiments, embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any of the methods for verifying data relationships in the above embodiments is implemented.

[0130] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0131] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0132] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0133] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A data verification method, characterized in that: The method comprises: When target data in the database changes, verification task information is obtained, the verification task information including clue data of the data crawler corresponding to the target data and a data relationship spectrum of the target data, the clue data including the identification and acquisition method information of the target data, the data relationship spectrum including the corresponding relationship information between metadata and the data crawler, and the relationship information between the metadata, the relationship information including the logical relationship between at least two metadata, the logical relationship including equal, greater than, less than, mutually exclusive or dependent; Determining the data crawler corresponding to the metadata according to the corresponding relationship information; Acquire the target data by the data crawler according to the identifier of the target data and the acquisition method information, wherein the target data includes at least two metadata; According to the relationship information between the at least two metadata and the metadata, the relationship between the at least two metadata is verified to obtain a verification result, and the verification object corresponding to the verification result is the logical relationship between the at least two metadata.

2. The method according to claim 1, characterized in that Before obtaining the verification task information, the method further includes: Obtaining an identifier of a data crawler, an identifier of metadata, corresponding relationship information between the data crawler and the metadata, and relationship information between the metadata; A data relationship spectrum is generated according to the identifier of the data crawler, the identifier of the metadata, the corresponding relationship information between the data crawler and the metadata, and the relationship information between the metadata.

3. The method according to claim 1, characterized in that The obtaining of verification task information includes: When target data changes, obtaining a data relationship spectrum of the target data; generating clue data for a data crawler in the data relationship spectrum according to the identifier of the target data and the data relationship spectrum; The verification task information is generated according to the clue data and the data relationship spectrum.

4. The method according to claim 3, characterized in that Generating clue data for a data crawler in the data relationship spectrum according to the identifier of the target data and the data relationship spectrum includes: When the target data changes, obtain the change record information when the target data changes; According to the identifier of the target data, the data relationship spectrum and the change record information when the target data is changed, clue data of the data crawler in the data relationship spectrum is generated.

5. The method according to claim 1, wherein The clue data includes an identifier of the metadata, a primary key of the metadata in a data table, and an index thereof. The acquiring, by the data crawler, at least two pieces of metadata according to the identifier of the target data and the acquisition method information includes: At least two metadata are obtained from the data table by the data crawler according to the identifier of the metadata, the primary key and the index of the metadata in the data table.

6. A data verification device, characterized in that: The device comprises: An acquisition module is configured to acquire verification task information when target data in a database changes, the verification task information including clue data of a data crawler corresponding to the target data and a data relationship spectrum of the target data, the clue data including an identifier and acquisition method information of the target data, the data relationship spectrum including corresponding relationship information between metadata and the data crawler and relationship information between the metadata, the relationship information including a logical relationship between at least two metadata, the logical relationship including being equal to, greater than, less than, mutually exclusive, or dependent; the module is further configured to acquire the data crawler corresponding to the metadata based on the corresponding relationship information; and the module is further configured to acquire the target data through the data crawler based on the identifier and acquisition method information of the target data, the target data including at least two metadata; The verification module is used to verify the relationship between the at least two metadata according to the relationship information between the at least two metadata and the metadata, and obtain a verification result, wherein the verification object corresponding to the verification result is the logical relationship between the at least two metadata.

7. The device according to claim 6, characterized in that The acquisition module is further configured to acquire an identifier of the data crawler, an identifier of the metadata, information on a corresponding relationship between the data crawler and the metadata, and information on a relationship between the metadata; It is also used to generate a data relationship spectrum based on the identifier of the data crawler, the identifier of the metadata, the corresponding relationship information between the data crawler and the metadata, and the relationship information between the metadata.

8. The device according to claim 6, characterized in that The acquisition module is specifically used to: When target data changes, obtaining a data relationship spectrum of the target data; generating clue data for a data crawler in the data relationship spectrum according to the identifier of the target data and the data relationship spectrum; The verification task information is generated according to the clue data and the data relationship spectrum.

9. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the data verification method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the data verification method according to any one of claims 1 to 5 is implemented.

Images