A data verification method, device, equipment, and storage medium

Abstract

An embodiment of this specification provides a data verification method, apparatus, device, and storage medium. The method includes: when a preset time window slides with a first step length, obtaining a target data set within the preset time window in a target data table, and obtaining a source data set within the preset time window in a source data table; wherein, the target data has identification information that uniquely identifies the target data; the source data has identification information that uniquely identifies the source data; the first step length is equal to the length of the preset time window; determining whether the target data in the target data set is consistent with the source data in the source data set according to the identification information of the target data in the target data set and the identification information of the source data in the source data set; in the case where the target data set within the preset time window sliding with the first step length is consistent with the source data set, it is determined that the target data in the target data set is successfully verified, achieving fast verification of data synchronization and improving the accuracy of data synchronization.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and particularly to a data verification method, apparatus, device, and storage medium. Background Art

[0002] In some business scenarios, it is necessary to synchronize data between data tables in multiple databases. For example, a source database can synchronize data to a subordinate database so that the subordinate database can back up the business data in the source database.

[0003] In the prior art, in the scenario of data synchronization, it is usually to ensure that the synchronization process is error-free. Once an error occurs in the data synchronization, a synchronization instruction is regenerated for the part of the data where the error occurs. In some cases, during the data transmission process, problems such as data loss and content tampering may occur. These problems may not cause the synchronization to fail, but the actually stored data may be incorrect.

[0004] Therefore, there is a technical problem of low data synchronization accuracy in the existing data synchronization process. Summary of the Invention

[0005] In view of this, multiple embodiments of this specification are committed to providing a data verification method, apparatus, device, and storage medium, which can perform fast data verification between a source data table and a target data table during the data synchronization process, thereby improving the accuracy of data synchronization.

[0006] Multiple embodiments in this specification provide a data verification method, the method includes: when a preset time window slides with a first step length, obtaining a target data set within the preset time window in a target data table, and obtaining a source data set within the preset time window in a source data table; wherein, the target data in the target data set has identification information that uniquely identifies the target data; the source data in the source data set has identification information that uniquely identifies the source data; the first step length is equal to the length of the preset time window; determining whether the target data in the target data set is consistent with the source data in the source data set according to the identification information of the target data in the target data set and the identification information of the source data in the source data set; when the target data set within the preset time window sliding with the first step length is consistent with the source data set, it is determined that the target data in the target data set passes the verification.

[0007] One embodiment of this specification provides a data verification device, the device includes: an acquisition module, configured to acquire a target data set within the preset time window in the target data table and a source data set within the preset time window in the source data table when the preset time window slides with a first step length; wherein, the target data in the target data set has identification information that uniquely identifies the target data; the source data in the source data set has identification information that uniquely identifies the source data; the first step length is equal to the length of the preset time window; a determination module, configured to determine whether the target data in the target data set is consistent with the source data in the source data set according to the identification information of the target data in the target data set and the identification information of the source data in the source data set; a processing module, configured to, when the target data set within the preset time window sliding with the first step length is consistent with the source data set, determine that the target data in the target data set passes the verification.

[0008] An embodiment of this specification provides a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the method described in any of the above embodiments is implemented.

[0009] An embodiment of this specification provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any of the above embodiments is implemented.

[0010] Multiple embodiments provided in this specification, by sliding a preset time window, acquire a target data set within the preset time window in the target data table and a source data set within the preset time window in the source data table. According to the identification information of the target data in the target data set and the identification information of the source data in the source data set, it can be determined whether the target data in the target data set is consistent with the source data in the source data set, and when the target data set is consistent with the source data set, it is determined that the target data in the target data set passes the verification, thereby realizing fast data verification between the source data table and the target data table during the data synchronization process, and improving the accuracy of data synchronization. Brief Description of the Drawings

[0011] Figure 1 It is a schematic diagram of a data verification system provided for an embodiment of this specification.

[0012] Figure 2 It is a schematic diagram of the flow of a data verification method provided for an embodiment of this specification.

[0013] Figure 3 It is a schematic diagram of a data verification method provided for an embodiment of this specification.

[0014] Figure 4 Schematic diagram of a data verification device provided for an embodiment of this specification.

[0015] Figure 5 Schematic diagram of a computer device provided for an embodiment of this specification. Specific embodiments

[0016] In order to enable those skilled in the art to better understand the solutions of this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this specification.

[0017] Data synchronization is often required between multiple databases. For example, the data in the source data table of the source database can be synchronized to the target data table of the target database, so that the data in the source data table can be backed up, or the data synchronized from the source data table can be displayed or further applied through the target data table. In the related art, for the verification of the data synchronization scenario, it is usually to ensure that there are no errors in the synchronization process. Once an error occurs in the data synchronization, a synchronization instruction is regenerated for the part of the data that has an error. In some cases, during the data transmission process, the data in the source database or the source data table of the source database may be continuously written. When synchronizing the data in the source data table to the target data table, problems such as data loss and content tampering may occur. These problems may not cause the synchronization to fail, but the data stored in the actual target data table may be missing or incorrect.

[0018] Therefore, it is necessary to provide a data verification method. After sending the data of the source data table to the target data table, the target data set within a preset time window can be obtained in the target data table, and the source data set within the preset time window can be obtained in the source data table, and it can be determined whether the target data in the target data set is consistent with the source data in the source data set. When the target data set is consistent with the source data set, it is determined that the target data in the target data set passes the verification, thereby realizing fast data verification between the source data table and the target data table during the data synchronization process and improving the accuracy of data synchronization.

[0019] Please refer to Figure 1, an embodiment of this specification provides a data verification system. The data verification system may include a client and a server. The client may be an electronic device with network access capabilities. Specifically, for example, the client may be a desktop computer, a tablet computer, a laptop computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a television, a smart speaker, a microphone, etc. Among them, the smart wearable device includes but is not limited to a smart bracelet, a smart watch, smart glasses, a smart helmet, a smart necklace, etc. Alternatively, the client may also be software that can run on the electronic device. The server may be an electronic device with certain computing and processing capabilities. It may have a network communication module, a processor, a memory, etc. Of course, the server may also refer to the software running on the electronic device. The server may also be a distributed server, which may be a system in which multiple processors, memories, network communication modules, etc. operate in cooperation. Alternatively, the server may also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server may also be a new technical means capable of implementing the corresponding functions of the embodiment of the specification. For example, it may be a new form of "server" based on quantum computing.

[0020] In this embodiment, the data verification system may include a client running a source database, a client running a target database, and a server. Among them, the source database includes source data tables, and the target database includes target data tables. Specifically, the client running the source database may send the data in the source data table to the target data table in the target database. The server may obtain the source data in the source data table, and may obtain the target data in the target data table, and verify the target data based on the source data obtained from the source data table, and may send the obtained source data to the target data table as the target data based on the verification result.

[0021] Please refer to Figure 2 , an embodiment of this specification provides a data verification method. The data verification method in this embodiment may be applied to the server. Specifically, it may include the following steps S201 to S205.

[0022] Step S201: When the preset time window slides with the first step length, obtain a target data set within the preset time window in the target data table, and obtain a source data set within the preset time window in the source data table; wherein, the target data has identification information that uniquely identifies the target data, the source data has identification information that uniquely identifies the source data, and the first step length is equal to the length of the preset time window.

[0023] In some cases, the synchronization process between data tables can be a process of sending the source data in the source data table as target data to the target data table. Verifying the target data in the target data table can be that after sending the source data in the source data table as target data to the target data table, the server obtains a source data set from the source data table and a target data set from the target data table through a sliding preset time window, and determines whether the identification information of the source data in the source data set all exists in the target data set, so as to quickly verify the target data in the target data table.

[0024] In this embodiment, the server can slide the preset time window by a first step length. When the preset time window slides by the first step length, it obtains the target data set within the preset time window in the target data table and the source data set within the preset time window in the source data table. Specifically, the server can obtain the target data set and the source data set once every time the preset time window slides by the first step length. Among them, the source data in the source data table and the target data in the target data table both have corresponding time information. The time information can refer to the generation time or modification time of the source data or target data. Obtaining the target data set within the preset time window in the target data table can mean that in the target data table, according to the time information corresponding to the target data, the target data whose time information in the target data table is within the preset time window is determined, and the set of target data within the preset time window is used as the target data set. Correspondingly, obtaining the source data set within the preset time window in the source data table can mean that in the source data table, according to the time information corresponding to the source data, the source data whose time information in the source data table is within the preset time window is determined, and the set of source data within the preset time window is used as the source data set.

[0025] In this embodiment, the target data can have identification information that uniquely identifies the target data, and the source data can have identification information that uniquely identifies the source data. Specifically, the identification information that uniquely identifies the target data can refer to the primary key value of the target data. Exemplarily, when the target data has multiple primary key values, the identification information can be obtained by combining the multiple primary key values. For example, if the target data has 3 primary key values, namely a, b, and c, then the identification information of the target data can be "a - b - c" or "abc", etc. Correspondingly, the identification information that uniquely identifies the source data can refer to the primary key value of the source data, or can be obtained by combining the multiple primary key values of the source data.

[0026] In this embodiment, the first step length may be equal to the length of the preset time window. During the process of the server verifying the target data in the target data table, by setting the sliding step length of the preset time window to the first step length equal to the length of the preset time window, the target data in the target data table can be comprehensively verified. In other words, when the server slides the preset time window by the first step length, the preset time windows do not overlap, and the target data in the target data table belongs to and only belongs to one preset time window, and the source data in the source data table belongs to and only belongs to one preset time window. By means of the preset time window sliding by the first step length, the target data in the target data table and the source data in the source data table can be accurately and comprehensively obtained.

[0027] Step S203: Determine whether the target data in the target data set is consistent with the source data in the source data set according to the identification information of the target data in the target data set and the identification information of the source data in the source data set.

[0028] In some cases, when verifying the target data in the target data table, after obtaining the target data set and the source data set through the preset time window, it may be determined whether the identification information of the source data in the source data set all exists in the target data set.

[0029] In this embodiment, it can be determined whether the target data in the target data set is consistent with the source data in the source data set according to the identification information of the target data in the target data set and the identification information of the source data in the source data set. Specifically, whether the target data in the target data set is consistent with the source data in the source data set may mean whether the identification information of the source data in the source data set all exists in the target data set. In other words, whether the source data set is a subset of the target data set.

[0030] The fact that the target data in the target data set is consistent with the source data in the source data set indicates that the identification information of the source data in the source data set all exists in the target data set, that is, the source data set is a subset of the target data set. The fact that the target data in the target data set is inconsistent with the source data in the source data set indicates that the identification information of the source data in the source data set does not all exist in the target data set. It may be that some source data in the source data table is lost or tampered with during the synchronization process, resulting in the target data received by the target data table not being all the source data sent by the source data table.

[0031] Step S205: When the target data set within the preset time window sliding by the first step length is consistent with the source data set, it is determined that the target data in the target data set passes the verification.

[0032] In this embodiment, when the target data set obtained by sliding with the first step length within the preset time window is consistent with the source data set, it can be determined that the target data in the target data set passes the verification. Specifically, when all the identification information in the source data set within the preset time window exists in the target data set, it can be determined that the target data in the target data set passes the verification, which means that the source data in the source data set has been accurately synchronized to the target data set. In this embodiment, by sliding the preset time window with the first step length equal to the length of the preset time window, the target data in the target data table can be verified quickly and accurately.

[0033] In some embodiments, the data verification method may further include the following steps S207 to S211.

[0034] Step S207: When the target data set within the preset time window slid with the first step length is inconsistent with the source data set, slide the preset time window with the second step length. Wherein, the second step length is less than the length of the preset time window.

[0035] In some cases, when the target data set within the preset time window slid with the first step length is inconsistent with the source data set, the server can compensate the target data set, that is, determine the missing data in the target data set relative to the source data set, and compensate the target data set according to the source data corresponding to the missing data, so as to improve the accuracy rate in the data synchronization process.

[0036] In this embodiment, when the server compensates the target data set in the case where the target data set within the preset time window slid with the first step length is inconsistent with the source data set, the first step length can be adjusted to the second step length less than the length of the preset time window, and start sliding the preset time window with the second step length.

[0037] Exemplarily, please refer to Figure 3 , Figure 3It is a schematic diagram of a data verification method provided by this embodiment. The length of the preset time window can be 5 minutes, the first step length can be 5 minutes, and the second step length can be 1 minute. During the process of sequentially synchronizing the source data in the source data table to the target data table, the source data may not be sent to the target data table in time due to data accumulation or network latency. For example, the source data with identification information 5 and 6 may not be synchronized to the target data table in time due to data accumulation. As an example, the preset time window that slides with the first step length can slide to 10:00 - 10:05, and 10:00 - 10:05 is taken as a preset time window. Then, obtain the target data set within the preset time window 10:00 - 10:05 in the target data table, and obtain the source data set within the preset time window 10:00 - 10:05 in the source data table. The identification information of the source data in the source data table whose time information is within the preset time window 10:00 - 10:05 is 1, 2, 3, 4, 5 respectively, and the identification information of the target data in the target data table whose time information is within the preset time window 10:00 - 10:05 is 1, 2, 3, 4 respectively. Then, the obtained target data set includes the source data with identification information 1, 2, 3, 4, 5 respectively, and the obtained target data set includes the target data with identification information 1, 2, 3, 4 respectively. Since the identification information of the source data in the source data set does not all exist in the target data set, or rather, the source data set is not a subset of the target data set, the target data in the target data set is inconsistent with the source data in the source data set. Therefore, the step length of the preset time window can be adjusted from the first step length to the second step length and start sliding. For example, when the preset time window is 10:00 - 10:05, slide the preset time window with the second step length, then the next preset time window is 10:01 - 10:06.

[0038] In some other embodiments, when the server adjusts the step length of the preset time window from the first step length to the second step length and slides, it can also obtain the time information corresponding to the missing data within the preset time window that slides with the first step length, and based on the time information corresponding to the missing data, determine the start time for the preset time window to slide with the second step length, and slide the preset time window with the second step length based on the start time. Among them, the missing data is the source data in the relative complement of the target data set in the source data set, that is, the source data that is not correctly synchronized to the target data table. Exemplarily, based on the source data set including the source data with identification information 1, 2, 3, 4, 5 respectively and the target data set including the target data with identification information 1, 2, 3, 4 respectively, it can be determined that the missing data within the preset time window 10:00 - 10:05 is the source data with identification information 5, and thus the time information corresponding to the missing data is determined to be 10:04. The server can determine the start time for the preset time window to slide with the second step length based on the time information corresponding to the missing data.

[0039] As an example, the server can determine the start time of the preset time window sliding with the second step length based on the time information corresponding to the missing data and the length of the preset time window. For example, if the time information corresponding to the missing data is 10:04 and the length of the preset time window is 5 minutes, the initial time window of the preset time window starting to slide with the second step length can be set to 09:59 - 10:04. In other words, the time information corresponding to the missing data can be set as the end time of the initial time window, and the time obtained by subtracting the length of the preset time window from the time information corresponding to the missing data can be used as the start time of the initial time window.

[0040] As another example, since the time information corresponding to the source data with the identification information of 5 is 10:04 and it has been determined as missing data within the preset time window sliding with the first step length, when adjusting the step length of the preset time window to slide with the second step length, the time information obtained by adding the time information corresponding to the missing data and the second step length can be used as the end time of the initial time window sliding with the second step length. The time obtained by adding the time information corresponding to the missing data and the second step length and then subtracting the length of the preset time window can be used as the start time of the initial time window. That is, the initial time window of the preset time window starting to slide with the second step length can be set to 10:00 - 10:05.

[0041] In some embodiments, the inconsistency between the target data set and the source data set within the preset time window may be caused by multiple missing data. When determining the start time of the preset time window sliding with the second step length based on the time information corresponding to the missing data, it can be determined based on the minimum time information among the time information corresponding to each of the multiple missing data. In other words, the initial time window of the preset time window starting to slide with the second step length can be determined based on the minimum time information among the time information corresponding to each of the multiple missing data.

[0042] Step S209: During the process of the preset time window sliding with the second step length, continuously determine the missing data from the source data of the source data set, and continuously calculate the number of times the missing data is missing. Wherein, the number of times the missing data is missing is the number of times the source data of the source data set is determined as missing data.

[0043] In this embodiment, the server can obtain the target data set and the source data set once every time the preset time window slides with the second step length, and determine the missing data once every time the target data set and the source data set are obtained. Exemplarily, the missing data may be caused by reasons such as network latency and back pressure of the synchronization program, resulting in untimely synchronization of the source data. The step length of the preset time window sliding in this embodiment is the second step length that is less than the length of the preset time window, and the preset time windows sliding with the second step length overlap. During the process of the preset time window sliding with the second step length, the missing data is continuously determined from the source data of the source data set, and the number of missing times of the missing data is continuously calculated. In this way, it is possible to perform delayed verification on the missing data caused by network latency and back pressure of the synchronization program.

[0044] Step S211: When the number of missing times of the missing data reaches the set threshold, add the missing data as target data to the target data set.

[0045] In this embodiment, during the process of the preset time window sliding with the second step length, when the number of missing times of the missing data reaches the set threshold, the missing data can be added to the compensation queue, and based on the compensation queue, the missing data is added as target data to the target data set.

[0046] In this embodiment, during the process of the preset time window sliding with the second step length, when the target data set is consistent with the source data set, the preset time window can be slid with the first step length.

[0047] Exemplarily, please continue to refer to Figure 3, the set threshold can be set to 3 times. When the preset time window slides with the first step length to 10:00 - 10:05, the missing data can be determined as the source data with the identification information of 5. Adjust the step length of the preset time window from the first step length to the second step length and start sliding, that is, the next preset time window is 10:01 - 10:06. Within the preset time window of 10:01 - 10:06, the missing data can be determined as the source data with the identification information of 5 and the source data with the identification information of 6. Continue to slide the preset time window with the second step length to 10:02 - 10:07. Within the preset time window of 10:02 - 10:07, the missing data can be determined as the source data with the identification information of 5, the source data with the identification information of 6, and the source data with the identification information of 7. Since within the preset time window of 10:02 - 10:07, the missing times of the source data with the identification information of 5 being determined as missing data reach the set threshold of 3, the missing data can be added to the compensation queue, and the missing data can be added to the target data set as the target data through the compensation queue. When adding the missing data to the compensation queue, the missing times of the missing data can be cleared. Continue to slide the preset time window with the second step length to 10:03 - 10:08. Within the preset time window of 10:03 - 10:08, the missing data can be determined as the source data with the identification information of 6, 7, and 8 respectively. Since within the preset time window of 10:03 - 10:08, the missing times of the source data with the identification information of 6 being determined as missing data reach the set threshold of 3, the missing data can be added to the compensation queue, and the missing data can be added to the target data set as the target data through the compensation queue, and the missing times of the missing data can be cleared. Continue to slide the preset time window with the second step length to 10:04 - 10:09. Within the preset time window of 10:04 - 10:09, there is no missing data, and the target data set is the same as the source data set. Therefore, the step length of the preset time window can be adjusted from the second step length to the first step length and slide.

[0048] In this embodiment, in order to improve the throughput of target data verification, the missing data is asynchronously compensated through the compensation queue. Therefore, in order to prevent errors in compensation caused by the source data table having already added the source data corresponding to the missing data to the target data table during the data verification process, when adding the missing data to the target data set as the target data, it is necessary to determine whether there is the same target data as the missing data in the target data set, and at the same time, it is also necessary to determine whether the time information corresponding to the target data is earlier than the time information corresponding to the missing data when there is the same target data as the missing data in the target data set.

[0049] In this embodiment, the process of adding missing data as target data to the target data set may include: when the number of occurrences of the missing data reaches a set threshold, determining whether there is the same identification information as that of the missing data in the target data set, and when there is no identification information in the target data set that is the same as the identification information of the missing data, adding the missing data as target data to the target data set.

[0050] Correspondingly, when there is the same identification information as that of the missing data in the target data set, determining whether the time information corresponding to the missing data is greater than the time information corresponding to the delayed data in the target data set, where the delayed data is the target data in the target data set with the same identification information as that of the missing data. When the time information corresponding to the missing data is greater than the time information corresponding to the delayed data, adding the missing data as target data to the target data set so that the missing data replaces the delayed data.

[0051] In some cases, the compensation queue may have a situation where compensation is not successful, resulting in the preset time window being unable to compensate for the missing data. During the verification process of the target data in the target data set, the number of occurrences of the missing data will reach the maximum calculation times of the preset time window. Therefore, it is necessary to re-verify the missing data whose number of occurrences exceeds the maximum calculation times of the preset time window.

[0052] In some embodiments, the data verification method may further include the following steps.

[0053] When the number of occurrences of the missing data reaches the maximum calculation times of the preset time window, adding the missing data to the missed detection queue, verifying the missing data in the missed detection queue according to the number and time span of the missing data in the compensation queue, and removing the missing data in the missed detection queue after the verification of the missing data in the missed detection queue is successful. In this way, the accuracy of the target data verification can be further improved.

[0054] Please refer to Figure 4, an embodiment of this specification provides a data verification device. The data verification device in this embodiment may include an acquisition module, a determination module, and a processing module. Among them, the acquisition module is configured to, when a preset time window slides with a first step length, acquire a target data set within the preset time window in a target data table, and acquire a source data set within the preset time window in a source data table; where the target data has identification information that uniquely identifies the target data; the source data has identification information that uniquely identifies the source data; the first step length is equal to the length of the preset time window. The determination module is configured to determine whether the target data in the target data set is consistent with the source data in the source data set according to the identification information of the target data in the target data set and the identification information of the source data in the source data set. The processing module is configured to, when the target data set within the preset time window that slides with the first step length is consistent with the source data set, determine that the target data in the target data set is successfully verified.

[0055] Regarding the specific functions and effects achieved by the data verification device, reference may be made to other embodiments of this specification for comparison and explanation, and details will not be elaborated here. Each module in the data verification device can be implemented in whole or in part by software, hardware, and their combination. Each of the modules can be embedded in or independent of the processor in a computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0056] Please refer to Figure 5 . In some embodiments, a computer device may be provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the data verification method in the above embodiment is implemented.

[0057] This specification embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the computer, the computer executes the data verification method in any of the above embodiments.

[0058] This specification embodiment also provides a computer program product containing instructions. When the instructions are executed by the computer, the computer executes the data verification method in any of the above embodiments.

[0059] It can be understood that the specific examples in this article are only for helping those skilled in the art better understand the embodiments of this specification, rather than limiting the scope of the present invention.

[0060] It can be understood that in various embodiments of this specification, the magnitudes of the sequence numbers of each process do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this specification.

[0061] It can be understood that the various embodiments described in this specification can be implemented independently or in combination, and the embodiments of this specification do not limit this.

[0062] Unless otherwise specified, all technical and scientific terms used in the embodiments of this specification have the same meaning as those commonly understood by those skilled in the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in the embodiments of this specification includes any and all combinations of one or more of the related listed items. The singular forms "a", "above-mentioned", and "the" used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0063] It can be understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method embodiments can be completed by the integrated logic circuit in the hardware of the processor or by instructions in software form. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of this specification can be directly embodied as being executed and completed by the hardware decoding processor, or can be executed and completed by a combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0064] It can be understood that the memory in the embodiments of this specification can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0065] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this specification.

[0066] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0067] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0068] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0069] In addition, in each embodiment of this specification, each functional unit may be integrated into one processing unit, may exist physically alone for each unit, or two or more units may be integrated into one unit.

[0070] If the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this specification. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0071] The above is only the specific embodiment of this specification, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed in this specification can easily think of changes or substitutions, which should all be covered by the protection scope of this specification. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A data verification method, characterized in that, The method includes: When the preset time window slides with a first step length, obtain a target data set within the preset time window in the target data table, and obtain a source data set within the preset time window in the source data table; wherein, the target data in the target data set has identification information that uniquely identifies the target data; the source data in the source data set has identification information that uniquely identifies the source data; the first step length is equal to the length of the preset time window; Determine whether the target data in the target data set is consistent with the source data in the source data set according to the identification information of the target data in the target data set and the identification information of the source data in the source data set; When the target data set within the preset time window sliding with the first step length is consistent with the source data set, it is determined that the target data in the target data set passes the verification; Wherein, the method further includes: When the target data set within the preset time window sliding with the first step length is inconsistent with the source data set, slide the preset time window with a second step length; wherein, the second step length is less than the length of the preset time window; During the process of sliding the preset time window with the second step length, continuously determine the missing data from the source data in the source data set, and continuously calculate the number of occurrences of the missing data; the missing data is the source data in the relative complement of the target data set in the source data set; the number of occurrences of the missing data is the number of times the source data in the source data set is determined to be the missing data; When the number of occurrences of the missing data reaches a set threshold, add the missing data as target data to the target data set.

2. The method according to claim 1, wherein When the target data set within the preset time window sliding with the first step length is inconsistent with the source data set, sliding the preset time window with the second step length includes: When the target data set within the preset time window sliding with the first step length is inconsistent with the source data set, obtain the time information corresponding to the missing data within the preset time window sliding with the first step length; Based on the time information corresponding to the missing data within the preset time window sliding with the first step length, determine the start time for sliding the preset time window with the second step length; Slide the preset time window with the second step length based on the start time.

3. The method according to claim 1, characterized in that When the number of occurrences of the missing data reaches a set threshold, adding the missing data as target data to the target data set includes: When the number of occurrences of the missing data reaches a set threshold, determine whether there is identification information in the target data set that is the same as the identification information of the missing data; When there is no identification information in the target data set that is the same as the identification information of the missing data, add the missing data as target data to the target data set; Correspondingly, when there is identification information in the target dataset that is the same as the identification information of the missing data, determine whether the time information corresponding to the missing data is greater than the time information corresponding to the delayed data in the target dataset; the delayed data is the target data in the target dataset whose identification information is the same as the identification information of the missing data. When the time information corresponding to the missing data is greater than the time information corresponding to the delayed data, add the missing data as target data to the target dataset so that the missing data replaces the delayed data.

4. The method according to claim 3, characterized in that, When the number of missing times of the missing data reaches a set threshold, adding the missing data as target data to the target dataset includes: When the number of missing times of the missing data reaches a set threshold, add the missing data to a compensation queue. Based on the compensation queue, add the missing data as target data to the target dataset.

5. The method according to claim 4, wherein The method further includes: When the number of missing times of the missing data reaches the maximum number of calculations in the preset time window, add the missing data to a missed detection queue. According to the quantity and time span of the missing data in the compensation queue, verify the missing data in the missed detection queue, and after the verification of the missing data in the missed detection queue is successful, remove the missing data in the missed detection queue.

6. The method according to claim 1, wherein The method further includes: During the process of the preset time window sliding with the second step length, when the target dataset is consistent with the source dataset, slide the preset time window with the first step length.

7. A data verification device, characterized in that, The device includes: An acquisition module, configured to, when the preset time window slides with the first step length, acquire a target dataset within the preset time window in a target data table and acquire a source dataset within the preset time window in a source data table; wherein, the target data in the target dataset has identification information that uniquely identifies the target data; the source data in the source dataset has identification information that uniquely identifies the source data; the first step length is equal to the length of the preset time window. A determination module, configured to determine whether the target data in the target dataset is consistent with the source data in the source dataset according to the identification information of the target data in the target dataset and the identification information of the source data in the source dataset. A processing module, configured to, when the target dataset within the preset time window sliding with the first step length is consistent with the source dataset, determine that the target data in the target dataset passes the verification. Wherein, the device is further configured to: When the target dataset within the preset time window sliding with the first step length is inconsistent with the source dataset, slide the preset time window with the second step length; wherein, the second step length is less than the length of the preset time window. During the process of sliding at the second step length within the preset time window, continuously determine the missing data from the source data of the source dataset, and continuously calculate the number of occurrences of the missing data; the missing data is the source data in the relative complement of the target dataset in the source dataset; the number of occurrences of the missing data is the number of times the source data of the source dataset is determined to be the missing data. When the number of occurrences of the missing data reaches a set threshold, add the missing data as target data to the target dataset.

8. A computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the method described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of claims 1 to 6 is implemented.

Images