Data Migration Integrity Verification Method, Apparatus, Device, and Computer-Readable Medium

Abstract

The present application relates to a method, apparatus, device, and computer-readable medium for integrity verification of data migration. The method includes: obtaining a target data file sent by a distributed data collection device, where the target data file is encoded in accordance with a target encoding format when the distributed data collection device collects data; parsing the target data file and the encoding format of the target data file to obtain parsed data; and determining that the target data file passes the integrity verification when the parsed data indicates that the target data file meets the verification conditions. The present application solves the technical problem that data integrity cannot be guaranteed during the data migration process.

Description

Technical Field

[0001] This application relates to the technical field of data migration, and in particular, to a method, apparatus, device, and computer-readable medium for integrity verification of data migration. Background Art

[0002] In the face of business data with a large number of users and high traffic, if a service provider only uses one machine to collect and store data, it cannot meet the business requirements at all. Therefore, a distributed collection system needs to be adopted, with multiple collection machines, and the same collection program is deployed on each collection machine. All collection machines together undertake the data collection of high traffic. Each machine in this distributed collection system is independent of each other and has no dependence. Therefore, in order to perform the final business settlement, it is necessary to summarize the data collected by all machines. During the summarization process, it is difficult to ensure the integrity of the data.

[0003] Currently, in the related art, in order to ensure the integrity of data transfer, a method of periodically uploading data is usually adopted. However, periodically uploading data can only ensure that there is data uploaded during the target period, but errors that occur during the data transmission process cannot be detected. Moreover, if there is a system failure or a hardware failure, the periodically uploaded data is likely to be out of order, and the integrity of the data cannot be guaranteed even more.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] This application provides a method, apparatus, device, and computer-readable medium for integrity verification of data migration to solve the technical problem that the integrity of data cannot be guaranteed during the data migration process.

[0006] According to one aspect of the embodiments of this application, this application provides a method for integrity verification of data migration, including: obtaining a target data file sent by a distributed data collection device, where the target data file is encoded in accordance with a target encoding format when the distributed data collection device collects data; parsing the target data file and the encoding format of the target data file to obtain parsed data; and determining that the target data file passes the integrity verification when the parsed data indicates that the target data file meets the verification conditions.

[0007] Optionally, obtaining the target data file sent by the distributed data collection device includes: obtaining a log file and a verification file that matches the log file, where the log file is used to save the data collected by the distributed data collection device, and the verification file is used to calculate a first hash value for the log file through a cryptographic hash function, and the first hash value is used to perform integrity verification on the data saved in the log file before and after data migration.

[0008] Optionally, parsing the target data file and the encoding format of the target data file, the obtained parsed data includes at least one of the following methods: determining the first generation time of the log file according to the encoding format of the log file, and the parsed data includes the first generation time; determining the second generation time of the verification file according to the encoding format of the verification file, and the parsed data includes the second generation time; determining the first sending device identifier of the log file according to the encoding format of the log file, and the parsed data includes the first sending device identifier; determining the second sending device identifier of the verification file according to the encoding format of the verification file, and the parsed data includes the second sending device identifier; determining the first data volume information of the log file and determining the second data volume information from the encoding format of the log file, and the parsed data includes the first data volume information and the second data volume information; determining the third data volume information of the verification file and determining the fourth data volume information from the encoding format of the verification file, and the parsed data includes the third data volume information and the fourth data volume information.

[0009] Optionally, when the parsed data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity verification includes: when the first time chain composed of multiple first generation times is consistent with the target time chain, determining that the log file passes the integrity verification; when the second time chain composed of multiple second generation times is consistent with the target time chain, determining that the verification file passes the integrity verification; when both the log file and the verification file pass the integrity verification, determining that the target data file passes the integrity verification.

[0010] Optionally, when the parsed data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity verification further includes: when the first identifier set composed of multiple first sending device identifiers is consistent with the target identifier set, determining that the log file passes the integrity verification; when the second identifier set composed of multiple second sending device identifiers is consistent with the target identifier set, determining that the verification file passes the integrity verification; when both the log file and the verification file pass the integrity verification, determining that the target data file passes the integrity verification.

[0011] Optionally, when the parsed data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity verification further includes: when the first data volume is consistent with the second data volume, determining that the log file passes the integrity verification; when the third data volume is consistent with the fourth data volume, determining that the verification file passes the integrity verification; when both the log file and the verification file pass the integrity verification, determining that the target data file passes the integrity verification.

[0012] Optionally, when the parsed data indicates that the target data file meets the verification condition, determining that the target data file passes the integrity verification further includes: determining a second hash value of the log file by using a cryptographic hash function; and determining that the target data file passes the integrity verification when the second hash value is consistent with the first hash value in the verification file.

[0013] According to another aspect of the embodiments of the present application, the present application provides a data migration integrity verification device, including: a data file acquisition module, configured to acquire a target data file sent by a distributed data acquisition device, where the target data file is encoded in a target encoding format when the distributed data acquisition device acquires data; a file parsing module, configured to parse the target data file and the encoding format of the target data file to obtain parsed data; and a file verification module, configured to determine that the target data file passes the integrity verification when the parsed data indicates that the target data file meets the verification condition.

[0014] According to another aspect of the embodiments of the present application, the present application provides an electronic device, including a memory, a processor, a communication interface, and a communication bus. A computer program that can run on the processor is stored in the memory. The memory and the processor communicate through the communication bus and the communication interface. When the processor executes the computer program, the steps of the above method are implemented.

[0015] According to another aspect of the embodiments of the present application, the present application further provides a computer-readable medium having non-volatile program code executable by a processor, and the program code causes the processor to execute the above method.

[0016] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the related technologies:

[0017] The technical solution of the present application is to acquire a target data file sent by a distributed data acquisition device, where the target data file is encoded in a target encoding format when the distributed data acquisition device acquires data; parse the target data file and the encoding format of the target data file to obtain parsed data; and determine that the target data file passes the integrity verification when the parsed data indicates that the target data file meets the verification condition. The present application solves the technical problem that data integrity cannot be guaranteed during the data migration process. Description of the Drawings

[0018] The accompanying drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0019] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 Schematic diagram of the hardware environment of an optional data migration integrity verification method provided according to an embodiment of the present application;

[0021] Figure 2 Flowchart of an optional data migration integrity verification method provided according to an embodiment of the present application;

[0022] Figure 3 Block diagram of an optional data migration integrity verification device provided according to an embodiment of the present application;

[0023] Figure 4 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0025] In subsequent descriptions, the use of suffixes such as "module", "component", or "unit" to represent elements is only for the convenience of description of the present application, and they have no specific meaning in themselves. Therefore, "module" and "component" can be used interchangeably.

[0026] In related technologies, in order to ensure the integrity of data transfer, the method of periodically uploading data is usually adopted. However, periodically uploading data can only ensure that data is uploaded during the target period, and errors that occur during the data transmission process cannot be detected. Moreover, if there is a system failure or hardware failure, the periodically uploaded data is prone to being out of order, and the integrity of the data cannot be guaranteed even more.

[0027] To solve the problems mentioned in the background technology, according to one aspect of the embodiments of the present application, an embodiment of a data migration integrity verification method is provided.

[0028] Optionally, in the embodiments of the present application, the above data migration integrity verification method can be applied to, for example Figure 1In the hardware environment composed of the terminal 101 and the server 103 as shown. As Figure 1 shown, the server 103 is connected to the terminal 101 through a network and can be used to provide services for the terminal or the client installed on the terminal. A database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The above network includes but is not limited to: wide area network, metropolitan area network or local area network. The terminal 101 includes but is not limited to PC, mobile phone, tablet computer, etc.

[0029] A data migration integrity verification method in an embodiment of the present application can be executed by the server 103 or jointly executed by the server 103 and the terminal 101. As Figure 2 shown, the method may include the following steps:

[0030] Step S202, obtain a target data file sent by a distributed data acquisition device. The target data file is encoded according to a target encoding format when the distributed data acquisition device collects data.

[0031] The data migration integrity verification method in an embodiment of the present application can be applied to the scenario of migrating data from a distributed data acquisition device to a central disk cabinet. The central disk cabinet is a server with multiple storage devices mounted.

[0032] Optionally, obtaining the target data file sent by the distributed data acquisition device includes:

[0033] Obtain a log file and a verification file matching the log file. The log file is used to save the data collected by the distributed data acquisition device. The verification file is used to calculate the first hash value of the log file through a cryptographic hash function. The first hash value is used to perform integrity verification on the data saved in the log file before and after data migration.

[0034] When the distributed data acquisition device collects data, it can intercept data every first time period. The first time period can be 5 minutes or can be set according to actual needs. Intercepting data can be to set a cutting program log_cut.sh every 5 minutes in the Linux cron task. Each time the intercepted data generates a log file in time order, and the hash value of the log file can be calculated using a cryptographic hash function and the hash value is saved in the verification file.

[0035] The naming formats of the log file and the verification file can be named according to the target encoding format, and the target encoding format can be log_IP_port_time.processid.log.size.gz. Among them, IP is used to indicate the IP address of the data collection device that collects the data of this file, port is used to represent the port used for data collection, time is used to represent the time when this file is generated, processid is used to represent the process number of data collection, and size is used to represent the size of this file.

[0036] Step S204, parse the target data file and the encoding format of the target data file to obtain parsed data.

[0037] In the embodiment of the present application, after receiving the log file and the verification file sent by the distributed data collection device, the central switch cabinet can parse the log file, the verification file and the file names of the two.

[0038] Optionally, parsing the target data file and the encoding format of the target data file to obtain parsed data includes at least one of the following methods:

[0039] According to the encoding format of the log file, determine the first generation time of the log file, and the parsed data includes the first generation time;

[0040] According to the encoding format of the verification file, determine the second generation time of the verification file, and the parsed data includes the second generation time;

[0041] According to the encoding format of the log file, determine the first sending device identifier of the log file, and the parsed data includes the first sending device identifier;

[0042] According to the encoding format of the verification file, determine the second sending device identifier of the verification file, and the parsed data includes the second sending device identifier;

[0043] Determine the first data volume information of the log file, and determine the second data volume information from the encoding format of the log file. The parsed data includes the first data volume information and the second data volume information;

[0044] Determine the third data volume information of the verification file, and determine the fourth data volume information from the encoding format of the verification file. The parsed data includes the third data volume information and the fourth data volume information.

[0045] In the embodiments of the present application, the generation time of the log file, i.e., the intercepted time, can be determined from the file name of the log file; the IP address of the acquisition device, i.e., the first sending device identifier; the size of the log file before data migration, i.e., the second data volume information. The generation time of the verification file can be determined from the verification file; the Internet Protocol address (IP address) of the device that generated the verification file, i.e., the second sending device identifier; the size of the verification file before data migration, i.e., the fourth data volume information. The actual size of the log file after data migration, i.e., the first data volume information, can also be directly determined using the received log file, and the actual size of the verification file after data migration, i.e., the third data volume information, can be directly determined using the received verification file.

[0046] Step S206, when the parsed data indicates that the target data file meets the verification condition, it is determined that the target data file passes the integrity verification.

[0047] Optionally, when the parsed data indicates that the target data file meets the verification condition, determining that the target data file passes the integrity verification includes:

[0048] When the first time chain composed of multiple first generation times is consistent with the target time chain, it is determined that the log file passes the integrity verification; when the second time chain composed of multiple second generation times is consistent with the target time chain, it is determined that the verification file passes the integrity verification; when both the log file and the verification file pass the integrity verification, it is determined that the target data file passes the integrity verification.

[0049] In the embodiments of the present application, the above target time chain is the complete time chain of the distributed data acquisition device from the first data interception to the last data interception. Therefore, the generation times (intercepted times) of the received log file and verification file can be chained into a time chain. When the above complete time chain is obtained, it can be determined that all intercepted data has been received and the file is complete.

[0050] Optionally, when the parsed data indicates that the target data file meets the verification condition, determining that the target data file passes the integrity verification further includes:

[0051] When the first identifier set composed of multiple first sending device identifiers is consistent with the target identifier set, it is determined that the log file passes the integrity verification; when the second identifier set composed of multiple second sending device identifiers is consistent with the target identifier set, it is determined that the verification file passes the integrity verification; when both the log file and the verification file pass the integrity verification, it is determined that the target data file passes the integrity verification.

[0052] In the embodiments of the present application, the above-mentioned target identifier set is the set of IP addresses of all actually running distributed data collection devices. Therefore, a set can be formed by the IP addresses of the generating devices of all received log files and verification files. When this set contains the IP addresses of all actually running distributed data collection devices, it can be determined that the data sent by all distributed data collection devices has been received and the files are complete.

[0053] Optionally, when parsing data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity verification further includes:

[0054] When the first data volume is consistent with the second data volume, it is determined that the log file passes the integrity verification; when the third data volume is consistent with the fourth data volume, it is determined that the verification file passes the integrity verification; when both the log file and the verification file pass the integrity verification, it is determined that the target data file passes the integrity verification.

[0055] In the embodiments of the present application, the above-mentioned first data volume is the actual size of the log file after data migration, and the second data volume is the file size of the log file before data migration recorded in the log file name. Therefore, the file sizes of the log file before and after data migration can be compared. If they are the same, it can be determined that the log file is complete. The above-mentioned third data volume is the actual size of the verification file after data migration, and the fourth data volume is the file size of the verification file before data migration recorded in the verification file name. Therefore, the file sizes of the verification file before and after data migration can be compared. If they are the same, it can be determined that the verification file is complete.

[0056] Optionally, when parsing data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity verification further includes:

[0057] Use a cryptographic hash function to determine the second hash value of the log file; when the second hash value is consistent with the first hash value in the verification file, it is determined that the target data file passes the integrity verification.

[0058] In the embodiments of the present application, a cryptographic hash function such as the md5 algorithm can be used to calculate the hash values before and after data migration. The above-mentioned first hash value is the hash value of the log file before data migration recorded in the verification file, and the above-mentioned second hash value is the hash value obtained by the central disk cabinet performing a hash calculation on the received log file. The hash values of the log file before and after data migration can be compared. If the hash values are the same, it indicates that the data of the log file has not changed and the file is complete. Before calculating the hash value of the log file, the correctness of the cryptographic hash function can also be checked.

[0059] Optionally, if a log file is detected to be missing, for example, when individual devices do not send file data, the central cabinet notifies the device to retransmit the file. The reason for the missing file may be unstable network connection or the like. When the file is still missing after retransmission, an alarm message is generated to give a prompt for troubleshooting to the staff. In extremely rare cases of missing files, the missing files can also be ignored because the impact caused by extremely few missing files is almost negligible when analyzing a large amount of data.

[0060] The technical solution of this application is to obtain a target data file sent by a distributed data acquisition device, where the target data file is encoded according to a target encoding format when the distributed data acquisition device collects data; parse the target data file and the encoding format of the target data file to obtain parsed data; and determine that the target data file passes the integrity check when the parsed data indicates that the target data file meets the check conditions. This application solves the technical problem that data integrity cannot be guaranteed during the data migration process.

[0061] According to another aspect of the embodiments of this application, as Figure 3 shown, a data migration integrity check device is provided, including: a data file acquisition module 301, configured to obtain a target data file sent by a distributed data acquisition device, where the target data file is encoded according to a target encoding format when the distributed data acquisition device collects data; a file parsing module 303, configured to parse the target data file and the encoding format of the target data file to obtain parsed data; and a file check module 305, configured to determine that the target data file passes the integrity check when the parsed data indicates that the target data file meets the check conditions.

[0062] It should be noted that the data file acquisition module 301 in this embodiment can be used to execute step S202 in the embodiments of this application, the file parsing module 303 in this embodiment can be used to execute step S204 in the embodiments of this application, and the file check module 305 in this embodiment can be used to execute step S206 in the embodiments of this application.

[0063] Here, it should be noted that the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules, as a part of the device, can run in the hardware environment as Figure 1 shown, and can be implemented by software or by hardware.

[0064] Optionally, the data file acquisition module is specifically configured to: acquire a log file and a verification file that matches the log file. The log file is used to store the data collected by the distributed data acquisition device, and the verification file is used to obtain a first hash value calculated for the log file through a cryptographic hash function. The first hash value is used to perform integrity verification on the data stored in the log file before and after data migration.

[0065] Optionally, the file parsing module is specifically configured to: determine the first generation time of the log file according to the encoding format of the log file, and the parsed data includes the first generation time; determine the second generation time of the verification file according to the encoding format of the verification file, and the parsed data includes the second generation time; determine the first sending device identifier of the log file according to the encoding format of the log file, and the parsed data includes the first sending device identifier; determine the second sending device identifier of the verification file according to the encoding format of the verification file, and the parsed data includes the second sending device identifier; determine the first data volume information of the log file, and determine the second data volume information from the encoding format of the log file, and the parsed data includes the first data volume information and the second data volume information; determine the third data volume information of the verification file, and determine the fourth data volume information from the encoding format of the verification file, and the parsed data includes the third data volume information and the fourth data volume information.

[0066] Optionally, the file verification module is specifically configured to: determine that the log file passes the integrity verification when the first time chain composed of multiple first generation times is consistent with the target time chain; determine that the verification file passes the integrity verification when the second time chain composed of multiple second generation times is consistent with the target time chain; determine that the target data file passes the integrity verification when both the log file and the verification file pass the integrity verification.

[0067] Optionally, the file verification module is further configured to: determine that the log file passes the integrity verification when the first identifier set composed of multiple first sending device identifiers is consistent with the target identifier set; determine that the verification file passes the integrity verification when the second identifier set composed of multiple second sending device identifiers is consistent with the target identifier set; determine that the target data file passes the integrity verification when both the log file and the verification file pass the integrity verification.

[0068] Optionally, the file verification module is further configured to: determine that the log file passes the integrity verification when the first data volume is consistent with the second data volume; determine that the verification file passes the integrity verification when the third data volume is consistent with the fourth data volume; determine that the target data file passes the integrity verification when both the log file and the verification file pass the integrity verification.

[0069] Optionally, the file verification module is further configured to: determine a second hash value of the log file by using a cryptographic hash function; and determine that the target data file passes the integrity verification when the second hash value is consistent with the first hash value in the verification file.

[0070] According to another aspect of the embodiments of the present application, the present application provides an electronic device, as Figure 4 shown, including a memory 401, a processor 403, a communication interface 405, and a communication bus 407. A computer program that can run on the processor 403 is stored in the memory 401. The memory 401 and the processor 403 communicate through the communication interface 405 and the communication bus 407. When the processor 403 executes the computer program, the steps of the above method are implemented.

[0071] The memory and the processor in the above electronic device communicate through the communication bus and the communication interface. The communication bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, and the like.

[0072] The memory may include a Random Access Memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0073] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0074] According to still another aspect of the embodiments of the present application, there is also provided a computer-readable medium having non-volatile program code executable by a processor.

[0075] Optionally, in the embodiments of the present application, the computer-readable medium is configured to store program code for the processor to execute the following steps:

[0076] Obtain a target data file sent by a distributed data acquisition device, where the target data file is encoded in a target encoding format when the distributed data acquisition device collects data;

[0077] Analyze the target data file and the encoding format of the target data file to obtain parsed data;

[0078] When the parsed data indicates that the target data file meets the verification condition, determine that the target data file passes the integrity verification.

[0079] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments, and will not be elaborated here.

[0080] When the embodiments of the present application are specifically implemented, reference may be made to the above embodiments, and corresponding technical effects are achieved.

[0081] It can be understood that these embodiments described herein can be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0082] For software implementation, the technologies described herein can be implemented by units that execute the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented inside or outside the processor.

[0083] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0084] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0085] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0086] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0087] In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0088] When the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes. It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0089] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for integrity verification of data migration, characterized in that, it includes: Obtain a target data file sent by a distributed data acquisition device, where the target data file is encoded in a target encoding format when the distributed data acquisition device acquires data; Analyze the target data file and the encoding format of the target data file to obtain parsed data; When the parsed data indicates that the target data file meets the verification conditions, determine that the target data file passes the integrity verification; Analyzing the target data file and the encoding format of the target data file to obtain parsed data includes at least one of the following methods: determining a first generation time of the log file according to the encoding format of the log file, where the parsed data includes the first generation time; determining a second generation time of the verification file according to the encoding format of the verification file, where the parsed data includes the second generation time; determining a first sending device identifier of the log file according to the encoding format of the log file, where the parsed data includes the first sending device identifier; determining a second sending device identifier of the verification file according to the encoding format of the verification file, where the parsed data includes the second sending device identifier; determining a first data volume information of the log file and determining a second data volume information from the encoding format of the log file, where the parsed data includes the first data volume information and the second data volume information; determining a third data volume information of the verification file and determining a fourth data volume information from the encoding format of the verification file, where the parsed data includes the third data volume information and the fourth data volume information; When the parsed data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity verification includes: when a first time chain composed of multiple first generation times is consistent with a target time chain, determining that the log file passes the integrity verification; when a second time chain composed of multiple second generation times is consistent with the target time chain, determining that the verification file passes the integrity verification; when both the log file and the verification file pass the integrity verification, determining that the target data file passes the integrity verification.

2. The method according to claim 1, characterized in that, Obtaining the target data file sent by the distributed data acquisition device includes: Obtain a log file and a verification file matching the log file, where the log file is used to store data collected by the distributed data acquisition device, and the verification file is used to store a first hash value obtained by calculating the log file through a cryptographic hash function, and the first hash value is used for integrity verification of the data stored in the log file before and after data migration.

3. The method according to claim 1, characterized in that, When the parsed data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity verification further includes: When the first identification set composed of a plurality of the first sending device identifications is consistent with the target identification set, it is determined that the log file passes the integrity check; When the second identification set composed of a plurality of the second sending device identifications is consistent with the target identification set, it is determined that the verification file passes the integrity check; When both the log file and the verification file pass the integrity check, it is determined that the target data file passes the integrity check.

4. The method according to claim 3, wherein, when the parsed data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity check further includes: when the first data volume is consistent with the second data volume, determining that the log file passes the integrity check; when the third data volume is consistent with the fourth data volume, determining that the verification file passes the integrity check; when both the log file and the verification file pass the integrity check, determining that the target data file passes the integrity check.

5. The method according to any one of claims 2 to 4, wherein, when the parsed data indicates that the target data file meets the verification conditions, determining that the target data file passes the integrity check further includes: using the cryptographic hash function to determine the second hash value of the log file; when the second hash value is consistent with the first hash value in the verification file, determining that the target data file passes the integrity check.

6. A data migration integrity verification device applicable to the data migration integrity verification method according to any one of claims 1 to 5, wherein, it includes: a data file acquisition module, configured to acquire a target data file sent by a distributed data acquisition device, where the target data file is encoded according to a target encoding format when the distributed data acquisition device acquires data; a file parsing module, configured to parse the target data file and the encoding format of the target data file to obtain parsed data; a file verification module, configured to determine that the target data file passes the integrity check when the parsed data indicates that the target data file meets the verification conditions.

7. An electronic device includes a memory, a processor, a communication interface and a communication bus, and a computer program that can run on the processor is stored in the memory, and the memory and the processor communicate through the communication bus and the communication interface, wherein, when the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable medium having non-volatile program code executable by a processor, wherein, the program code causes the processor to execute the method according to any one of claims 1 to 5.

Images