Data verification method, device and equipment

Abstract

The invention relates to a data verification method, device and equipment, and relates to the technical field of data processing, and the method comprises the steps: obtaining historical access information corresponding to each data segment included in a storage system; based on the historical access information corresponding to each data segment, level information corresponding to each data segment is determined, and the level information is used for identifying the access frequency of the data segment; obtaining the target data segment and target level information corresponding to the target data segment in response to an instruction triggered by a user for performing consistency verification on the target data segment; on the basis of the verification strategy corresponding to the target level information, the target data block corresponding to the verification strategy is extracted from the target data segment, and the data consistency of the target data block is verified, so that the resource consumption is reduced, and the verification efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data verification method, device and equipment. Background Art

[0002] With the digitalization of industry and life, massive amounts of data are generated every day, and the amount of data stored in storage systems is also increasing. To ensure the reliability and integrity of the storage system, it is necessary to periodically perform consistency checks on the data in the storage system.

[0003] In the related art, during each verification process, all data blocks in the storage system need to be fully verified to ensure data consistency in the storage system. However, as the amount of data in the storage system increases, the time and resources consumed in the consistency verification of the data also increase, resulting in increased resource consumption of the system, which in turn affects the normal upper-layer services and the IO (Input Output) performance of the storage system. Summary of the invention

[0004] The present invention provides a data verification method, device and equipment, which are used to solve the problem of large resource consumption and low verification efficiency of current data consistency verification.

[0005] In a first aspect, the present application provides a data verification method, the method comprising:

[0006] Obtaining historical access information corresponding to each data segment included in the storage system;

[0007] Based on the historical access information corresponding to each data segment, determine the level information corresponding to each data segment, where the level information is used to identify the frequency of access to the data segment;

[0008] In response to a user-triggered instruction to perform consistency check on a target data segment, acquiring the target data segment and target level information corresponding to the target data segment;

[0009] Based on the verification strategy corresponding to the target level information, a target data block corresponding to the verification strategy is extracted from the target data segment, and the data consistency of the target data block is verified.

[0010] The above method determines the level information corresponding to the data segment, that is, the frequency of access, based on the historical access information of each data segment. After receiving the instruction to perform consistency check on the data segment, based on the verification strategy corresponding to the level information of the data segment, the target data block is extracted from the data segment for consistency check. This method effectively reduces the amount of data consistency verification data, reduces the consumption of system resources without affecting the effect of data consistency verification, improves the efficiency of data verification, and thus improves the IO performance of upper-level business and storage system.

[0011] In a possible implementation manner, the level information includes first level information and second level information, and the level information corresponding to each data segment is determined based on the historical access information corresponding to each data segment, including:

[0012] For each data segment, if the historical access information corresponding to the data segment is within a preset information value interval, it is determined that the data segment corresponds to first-level information, and the first-level information is used to indicate that: the frequency of access to the data segment is higher than the reference frequency;

[0013] If the historical access information corresponding to the data segment is not within the above information value range, it is determined that the data segment corresponds to the second level information, and the second level information is used to indicate that: the frequency of access to the data segment is not higher than the reference frequency;

[0014] The above historical access information includes at least one of the number of accesses, access time and access frequency corresponding to the data segment within a preset time range.

[0015] The above method can effectively determine whether the target data segment corresponds to the first level information or the second level information by determining whether the historical access information is within the preset information value range, that is, accurately determine the level information corresponding to the target data segment, thereby ensuring the accuracy of subsequent data block extraction.

[0016] In a possible implementation manner, when the target level information is first level information, the step of extracting a target data block corresponding to the verification strategy based on the verification strategy corresponding to the target level information from the target data segment includes:

[0017] Based on the target sampling rule corresponding to the target data segment, the target data segment is randomly sampled, and each obtained data block is determined as the target data block corresponding to the verification strategy;

[0018] Among them, the above-mentioned target sampling rules are determined based on the correspondence between each reference value range of pre-defined historical access information and the reference sampling rules, and the historical access information corresponding to the above-mentioned target data segment; the above-mentioned target sampling rules include: at least one of the sampling data volume, sampling interval and sampling algorithm.

[0019] The above method sets the correspondence between each reference value range of historical access information and the reference sampling rules, and determines the sampling rules corresponding to the target data segment based on the correspondence, extracts data from the data segment based on the sampling rules, and further refines the extraction rules, which can further reduce the amount of data to be verified and improve data verification efficiency.

[0020] In a possible implementation manner, before randomly sampling the target data segment based on the target sampling rule corresponding to the target data segment, the method further includes:

[0021] Obtain the reference time interval corresponding to each data block in the target data segment. The reference time interval is the time interval between the last consistency check of the data block and the current time:

[0022] From the above data blocks, select data blocks whose reference time interval is greater than a preset threshold;

[0023] The target sampling rule corresponding to the target data segment is used to randomly sample the target data segment, including:

[0024] Based on the target sampling rule corresponding to the target data segment, the data blocks selected from the target data segment are randomly sampled.

[0025] The above method, by screening the data blocks whose time interval between the last consistency check and the current time is greater than a preset threshold, and randomly extracting data blocks from them, can ensure that data blocks that have not been checked for a long time are extracted first, thereby further increasing the accuracy of data consistency checking.

[0026] In a possible implementation manner, when the target level information is the second level information, the step of extracting the target data block corresponding to the verification strategy based on the verification strategy corresponding to the target level information from the target data segment includes:

[0027] A full data block is extracted from the target data segment, and the full data block is determined as a target data block corresponding to the verification strategy.

[0028] The above method performs data verification on the entire amount of data for the target data segment whose target level information is the second level information, that is, the data segment that is less frequently accessed, and can ensure the accuracy and effectiveness of the data verification.

[0029] In a possible implementation, the above method further includes:

[0030] In response to a user-triggered instruction to access a target data segment, acquiring the target data segment;

[0031] Based on each data block included in the target data segment, verifying the data consistency of the target data segment;

[0032] After determining that the verification result is passed, access to the above target data segment is allowed, and the access information is recorded.

[0033] The above method performs a data consistency check on a target data segment when the user accesses it, thereby ensuring the validity of the data during subsequent accesses.

[0034] In a second aspect, an embodiment of the present application provides a data verification device, the device comprising:

[0035] An acquisition module, used to acquire historical access information corresponding to each data segment included in the storage system;

[0036] A determination module, used to determine level information corresponding to each data segment based on historical access information corresponding to each data segment, wherein the level information is used to identify the frequency of access to the data segment;

[0037] A response module, configured to obtain the target data segment and target level information corresponding to the target data segment in response to an instruction triggered by a user to perform consistency check on the target data segment;

[0038] The verification module is used to extract the target data block corresponding to the verification strategy from the target data segment based on the verification strategy corresponding to the target level information, and verify the data consistency of the target data block.

[0039] In a possible implementation manner, the level information includes first level information and second level information, and the determination module is specifically configured to:

[0040] For each data segment, if the historical access information corresponding to the data segment is within a preset information value interval, it is determined that the data segment corresponds to first-level information, and the first-level information is used to indicate that: the frequency of access to the data segment is higher than the reference frequency;

[0041] If the historical access information corresponding to the data segment is not within the above information value range, it is determined that the data segment corresponds to the second level information, and the second level information is used to indicate that: the frequency of access to the data segment is not higher than the reference frequency;

[0042] The above historical access information includes at least one of the number of accesses, access time and access frequency corresponding to the data segment within a preset time range.

[0043] In a possible implementation manner, when the target level information is the first level information, the verification module is specifically configured to:

[0044] Based on the target sampling rule corresponding to the target data segment, the target data segment is randomly sampled, and each obtained data block is determined as the target data block corresponding to the verification strategy;

[0045] Among them, the above-mentioned target sampling rules are determined based on the correspondence between each reference value range of pre-defined historical access information and the reference sampling rules, and the historical access information corresponding to the above-mentioned target data segment; the above-mentioned target sampling rules include: at least one of the sampling data volume, sampling interval and sampling algorithm.

[0046] In a possible implementation manner, the verification module is specifically used to:

[0047] Obtain the reference time interval corresponding to each data block in the target data segment. The reference time interval is the time interval between the last consistency check of the data block and the current time:

[0048] From the above data blocks, select data blocks whose reference time interval is greater than a preset threshold;

[0049] Based on the target sampling rule corresponding to the target data segment, the data blocks selected from the target data segment are randomly sampled.

[0050] In a possible implementation manner, when the target level information is the second level information, the verification module is specifically configured to:

[0051] All data blocks extracted from the target data segment are determined as target data blocks corresponding to the verification strategy.

[0052] In a possible implementation manner, the above device further includes a recording module, which is specifically configured to:

[0053] In response to a user-triggered instruction to access a target data segment, acquiring the target data segment;

[0054] Based on each data block included in the target data segment, verifying the data consistency of the target data segment;

[0055] After determining that the verification result is passed, access to the above target data segment is allowed, and the access information is recorded.

[0056] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the data verification method when executing the computer program.

[0057] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps in the above-mentioned data verification method of the present application are implemented.

[0058] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which is stored in a computer-readable storage medium; when the processor of a memory access device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the memory access device executes the steps in the above-mentioned data verification method of the present application.

[0059] For each of the above-mentioned second to fifth aspects and the technical effects that may be achieved by each of the above-mentioned aspects, please refer to the above-mentioned description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0061] Figure 1 A schematic diagram of a data verification method provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of a possible data verification example provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of a possible level information update example provided in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of a data verification device provided in an embodiment of the present application;

[0065] Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0067] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

[0068] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0069] In the technical solution of this application, the acquisition, transmission, storage, and use of data are in compliance with the requirements of relevant national laws and regulations.

[0070] In order to facilitate those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, the following is a brief description of the professional terms involved:

[0071] (1) Data segment: A data segment composed of RAID (Redundant Array of Independent Disks) stripes.

[0072] (2) Hot data: data that is frequently accessed in the storage system.

[0073] (3) Cold data: data in the storage system that is not frequently accessed.

[0074] (4) IO (Input Output): refers to the direction of data flow, including inflow and outflow.

[0075] With the digitalization of industry and life, massive amounts of data are generated every day, and the data stored in storage systems is also increasing. For storage systems, it is necessary to perform periodic scans and checks on the data stored in the storage system through the data consistency check task of the storage system to ensure the consistency of the system data and to ensure the reliability and integrity of the system data.

[0076] In the related art, during each verification process, all data blocks in the storage system need to be fully scanned and verified to ensure data consistency in the storage system. However, as the amount of data in the storage system increases, the time and resources consumed in the consistency verification of the data also increase, resulting in increased resource consumption of the system, which in turn affects the normal upper-layer services and the IO (Input Output) performance of the storage system.

[0077] To overcome the above problems, an embodiment of the present application provides a data verification method, which determines the level information corresponding to the data segment, that is, the frequency of access, based on the historical access information of each data segment. After receiving an instruction to perform a consistency check on the data segment, the target data block is extracted from the data segment for consistency verification based on the verification strategy corresponding to the level information of the data segment. This method effectively reduces the amount of verification data for data consistency, reduces the consumption of system resources without affecting the effect of data consistency verification, improves the efficiency of data verification, and thereby improves the IO performance of upper-level business and storage systems.

[0078] The present application is further described in detail below in conjunction with the accompanying drawings. Figure 1 FIG. 1 is a flow chart of a data verification method provided in an embodiment of the present application. The specific implementation process of the method is as follows:

[0079] Step S101, obtaining historical access information corresponding to each data segment included in the storage system;

[0080] In some embodiments, the operation of the above step S101 can be performed once at every preset time interval. The above preset time interval can be set based on demand. Specifically, it can be set to a fixed value, or it can be set to a dynamic value that is adjusted in real time based on information such as the level information of the data segment determined currently. The specific adjustment rules are not limited in the embodiments of this application.

[0081] In a possible implementation manner, in the embodiment of the present application, each time a user access operation to a data segment is received, the following process is performed:

[0082] Responding to a user-triggered instruction to access the target data segment, acquiring the target data segment;

[0083] Based on each data block included in the target data segment, verifying the data consistency of the target data segment;

[0084] After determining that the verification result is passed, access to the target data segment is allowed, and the access information is recorded.

[0085] In some embodiments, the user-triggered indication of accessing the target data segment carries identification information or address information corresponding to the target data segment to be accessed. Therefore, the identification information of each target data segment to be accessed can be obtained from the indication, and the target data segment can be obtained based on the identification information, or the address information of each target data segment to be accessed can be obtained from the indication, and each target data segment can be obtained from the position corresponding to the address information.

[0086] In some embodiments, after acquiring the target data segment and before performing the access operation, in the embodiments of the present application, it is also necessary to perform a consistency check on the target data segment. All data blocks included in the target data segment may include data blocks and check blocks. When performing a consistency check, a consistency check can be performed based on the data blocks and the check blocks.

[0087] Optionally, after determining that the verification result corresponding to the target data is verified as passed, subsequent access operations are allowed to be performed, and access information corresponding to the access operation is recorded. In the embodiment of the present application, the specific content of the access information is not limited. For example, the access information may include: access time corresponding to the access operation, access result, visitor identification, etc. When it is determined that the verification result corresponding to the target data is verified as failed, the subsequent access operation is no longer performed, and a prompt message that there is a data consistency error in the current target data segment is fed back to the user, so that the user can perform subsequent repair operations.

[0088] In some embodiments, a designated area can be set in the storage system in the embodiments of the present application to store historical access information corresponding to each data segment, and the historical access information includes: at least one of the number of accesses, access time, and access frequency corresponding to the data segment within a preset time range, wherein the preset time range can be set based on demand, for example, set to within one month.

[0089] It should be noted that the above access time refers to the access time corresponding to each access operation corresponding to the data segment within the preset time range, and the above access frequency refers to the ratio between the number of accesses within the preset time range and the time interval corresponding to the preset time range.

[0090] Step S102, determining level information corresponding to each data segment based on historical access information corresponding to each data segment;

[0091] The level information is used to identify the frequency of access to the data segment.

[0092] In a possible implementation manner, the level information includes first level information and second level information, and the level information corresponding to each data segment is determined based on the historical access information corresponding to each data segment, including:

[0093] For each data segment, if the historical access information corresponding to the data segment is within a preset information value interval, it is determined that the data segment corresponds to first-level information, and the first-level information is used to indicate that: the frequency of access to the data segment is higher than the reference frequency;

[0094] If the historical access information corresponding to the data segment is not within the above information value range, it is determined that the data segment corresponds to the second level information, and the second level information is used to indicate that: the frequency of access to the data segment is not higher than the reference frequency;

[0095] The above historical access information includes at least one of the number of accesses, access time and access frequency corresponding to the data segment within a preset time range.

[0096] In some embodiments, the above-mentioned level information in the embodiments of the present application can be roughly divided into two categories, namely, first level information (hot data level) and second level information (cold data level). When determining the level information corresponding to a data segment based on historical access information, the corresponding level can be determined by pre-setting a value range (or threshold) corresponding to each historical access information.

[0097] In some embodiments, for the number of visits in the historical access information, a first preset threshold (i.e., a number threshold, and the value corresponding to the threshold can be set based on demand) can be set as a reference in the embodiments of the present application, and the interval greater than or equal to the first preset threshold is used as the first information value interval corresponding to the number of visits.

[0098] For the access time in the historical access information, a second preset threshold (i.e., a time threshold, the value corresponding to the threshold can be set based on demand) can be set as a reference in the embodiment of the present application, and the interval whose time interval with the current time is less than the second preset threshold is used as the second information value interval corresponding to the access time. Specifically, if the access time includes multiple times, the access time with the smallest interval with the current time (i.e., the most recent access time) is used as the target access time. If the target access time is within the second information value interval, it is determined that the access time is within the second information value interval.

[0099] For the access frequency in the historical access information, a third preset threshold (i.e., a frequency threshold, the value corresponding to which can be set based on demand) can be set as a reference in an embodiment of the present application, and an interval greater than or equal to the third preset threshold can be used as a third information value interval corresponding to the access frequency.

[0100] In some embodiments, if the above-mentioned historical access information includes multiple (i.e., greater than one) parameters, then when determining the level information corresponding to the data segment based on whether the historical access information is located in the information value interval, the above-mentioned information value interval includes the information value intervals corresponding to each parameter (i.e., the first information value interval, the second information value interval, and the third information value interval mentioned above), and it can be determined whether each parameter included in the historical access information is located in the corresponding information value interval, and the level information corresponding to the data segment is determined based on the first number of parameters located in the corresponding information value interval and the second number of parameters included in the historical access information.

[0101] Specifically, the above method of determining the level information of the data segment based on the first quantity and the second quantity can be set based on demand, for example, it can be set to: the difference between the first quantity and the second quantity is less than a preset quantity threshold (such as 1), or it can be set to: the ratio between the first quantity and the second quantity is greater than a preset ratio (such as 1 / 2), etc.

[0102] In some embodiments, the storage system in the embodiments of the present application periodically updates and stores the level information corresponding to each data segment through the methods described in S101-S102 above.

[0103] Step S103, in response to an instruction triggered by a user to perform consistency check on a target data segment, obtaining the target data segment and target level information corresponding to the target data segment;

[0104] In some embodiments, the user-triggered indication for performing a consistency check on the target data segment may include the following two types: a first indication for performing a consistency check on a specified individual data segment (target data segment) in the storage system, and a second indication for performing a consistency check on all data segments (target data segments) in the storage system.

[0105] If the first indication is currently received, and the first indication carries identification information or address information corresponding to the target data segment to be verified, then the identification information of each target data segment to be verified can be obtained from the first indication, and the target data segment can be obtained based on the identification information, or the address information of each target data segment to be verified can be obtained from the first indication, and each target data segment can be obtained from the position corresponding to the address information.

[0106] If the second indication is currently received, and the first indication carries a full amount identifier (used to identify the consistency check of the full amount of data segments), the full amount of data in the storage system is used as the target data segment, that is, the storage system is traversed to obtain each target data segment.

[0107] Step S104: based on the verification strategy corresponding to the target level information, extract the target data block corresponding to the verification strategy from the target data segment, and verify the data consistency of the target data block.

[0108] In a possible implementation manner, when the target level information is first level information, the step of extracting a target data block corresponding to the verification strategy based on the verification strategy corresponding to the target level information from the target data segment includes:

[0109] Based on the target sampling rule corresponding to the target data segment, the target data segment is randomly sampled, and each obtained data block is determined as the target data block corresponding to the verification strategy;

[0110] Among them, the above-mentioned target sampling rules are determined based on the correspondence between each reference value range of pre-defined historical access information and the reference sampling rules, and the historical access information corresponding to the above-mentioned target data segment; the above-mentioned target sampling rules include: at least one of the sampling data volume, sampling interval and sampling algorithm.

[0111] In some embodiments, the correspondence between each reference value range and the sampling rule (i.e., the reference sampling rule) is pre-defined in the embodiments of the present application. When the target level information is determined to be the first level information, that is, some data blocks are extracted from the target data segment for consistency verification. When performing the verification, the reference value range of the historical status information is first determined, and then based on the correspondence between each reference value range and the sampling rule (i.e., the reference sampling rule), the target sampling rule corresponding to the target data segment is determined, and based on this, the data blocks in the data segment are sampled.

[0112] It should be noted that the value settings of the above-mentioned reference value ranges of the present application, as well as the correspondence between each reference value range and the sampling rule can be set based on demand; for example, it can be set as follows: different reference value ranges correspond to different amounts of sampling data, while other parameters are the same, or different reference value ranges correspond to different amounts of sampling data, different sampling intervals, and the same sampling algorithm; but it must be ensured that the higher the frequency with which the target data segment is accessed (i.e., the greater the number of accesses, the closer the access time is to the current time, and the higher the access frequency), the smaller the number of corresponding data blocks extracted.

[0113] In a possible implementation manner, before randomly sampling the target data segment based on the target sampling rule corresponding to the target data segment, the method further includes:

[0114] Obtain the reference time interval corresponding to each data block in the target data segment. The reference time interval is the time interval between the last consistency check of the data block and the current time:

[0115] From the above data blocks, select data blocks whose reference time interval is greater than a preset threshold;

[0116] The target sampling rule corresponding to the target data segment is used to randomly sample the target data segment, including:

[0117] Based on the target sampling rule corresponding to the target data segment, the data blocks selected from the target data segment are randomly sampled.

[0118] In a specific implementation, in an embodiment of the present application, after each data consistency check is performed, the time of the most recent data consistency check corresponding to each data block in each data segment (hereinafter referred to as the reference time for ease of explanation) can be recorded, so that when data blocks are subsequently sampled, data blocks that have not been checked for data consistency for a long time can be given priority based on the reference time corresponding to each data block.

[0119] It should be noted that the value of the above-mentioned preset threshold can be set based on demand, and can also be adjusted in real time based on the extraction results corresponding to subsequent data blocks, such as lowering or increasing the value of the preset threshold.

[0120] In a possible implementation manner, when the target level information is the second level information, the step of extracting the target data block corresponding to the verification strategy based on the verification strategy corresponding to the target level information from the target data segment includes:

[0121] A full data block is extracted from the target data segment, and the full data block is determined as a target data block corresponding to the verification strategy.

[0122] In some embodiments, when the target level information is the second level information, it can be determined that the frequency of access to the current target data segment is reduced. Therefore, to ensure the accuracy of the verification, all data blocks in the target data segment will be extracted for data consistency verification in the embodiments of the present application.

[0123] The above method extracts data segments from data segments for data consistency verification based on the access frequency corresponding to the data segments, thereby effectively reducing the amount of data for data consistency verification. Without affecting the effect of data consistency verification, it reduces the consumption of system resources and improves the efficiency of data verification, thereby improving the IO performance of upper-level business and storage systems.

[0124] The following combination Figure 2 and Figure 3 , the above data verification process is elaborated in detail with a specific example, and the process specifically includes: a data consistency verification process after receiving the user's instruction to perform consistency verification on the target data segment, and a level information update process of the data segment that is periodically executed.

[0125] Specifically, Figure 2 As shown, the above data consistency verification process specifically includes the following steps:

[0126] Step S201, in response to an instruction triggered by a user to perform consistency check on a target data segment, obtaining the target data segment and target level information corresponding to the target data segment;

[0127] Step S202, if the target level information is the first level information, then based on the correspondence between each reference value range of the predefined historical access information and the reference sampling rule, and the historical access information corresponding to the target data segment, determine the target sampling rule corresponding to the target data segment;

[0128] The target sampling rule includes at least one of sampling data volume, sampling interval and sampling algorithm.

[0129] Step S203, selecting data blocks whose reference time interval is greater than a preset threshold from the data blocks in the target data segment;

[0130] The reference time interval is the time interval between the last consistency check of the data block and the current time.

[0131] Step S204, based on the target sampling rule corresponding to the target data segment, randomly sampling the data blocks screened out in the target data segment;

[0132] For example, randomly extract data blocks of different granularities such as 8K or 64K from the data segment for consistency check;

[0133] Step S205, performing data consistency check based on the extracted target data block;

[0134] Step S206: If the target level information is the second level information, a full amount of data blocks are extracted from the target data segment to perform a data consistency check.

[0135] Specifically, Figure 3 As shown, the above level information update process specifically includes the following steps:

[0136] Step S301, obtaining historical access information corresponding to each data segment included in the storage system at intervals of a preset time;

[0137] The above historical access information includes at least one of the number of accesses, access time and access frequency corresponding to the data segment within a preset time range.

[0138] Step S302, for each data segment, determine whether the corresponding historical access information is within a preset information value range; if yes, proceed to step S303, if no, proceed to step S304;

[0139] Step S303, determining that the data segment corresponds to the first level information, and based on this, updating the level information of the data segment;

[0140] Step S304: determine whether the data segment corresponds to the second level information, and based on this, update the level information of the data segment.

[0141] Based on the same inventive concept, the present application also provides a data verification device, see Figure 4 , the device comprises:

[0142] The acquisition module 401 is used to acquire the historical access information corresponding to each data segment included in the storage system;

[0143] A determination module 402, configured to determine level information corresponding to each data segment based on historical access information corresponding to each data segment, wherein the level information is used to identify how frequently the data segment is accessed;

[0144] A response module 403 is used to obtain the target data segment and target level information corresponding to the target data segment in response to an instruction triggered by a user to perform consistency check on the target data segment;

[0145] The verification module 404 is used to extract a target data block corresponding to the verification strategy from the target data segment based on the verification strategy corresponding to the target level information, and verify the data consistency of the target data block.

[0146] In a possible implementation manner, the level information includes first level information and second level information, and the determination module 402 is specifically configured to:

[0147] For each data segment, if the historical access information corresponding to the data segment is within a preset information value interval, it is determined that the data segment corresponds to first-level information, and the first-level information is used to indicate that: the frequency of access to the data segment is higher than the reference frequency;

[0148] If the historical access information corresponding to the data segment is not within the above information value range, it is determined that the data segment corresponds to the second level information, and the second level information is used to indicate that: the frequency of access to the data segment is not higher than the reference frequency;

[0149] The above historical access information includes at least one of the number of accesses, access time and access frequency corresponding to the data segment within a preset time range.

[0150] In a possible implementation manner, when the target level information is the first level information, the verification module 404 is specifically configured to:

[0151] Based on the target sampling rule corresponding to the target data segment, the target data segment is randomly sampled, and each obtained data block is determined as the target data block corresponding to the verification strategy;

[0152] Among them, the above-mentioned target sampling rules are determined based on the correspondence between each reference value range of pre-defined historical access information and the reference sampling rules, and the historical access information corresponding to the above-mentioned target data segment; the above-mentioned target sampling rules include: at least one of the sampling data volume, sampling interval and sampling algorithm.

[0153] In a possible implementation manner, the verification module 404 is specifically configured to:

[0154] Obtain the reference time interval corresponding to each data block in the target data segment. The reference time interval is the time interval between the last consistency check of the data block and the current time:

[0155] From the above data blocks, select data blocks whose reference time interval is greater than a preset threshold;

[0156] Based on the target sampling rule corresponding to the target data segment, the data blocks selected from the target data segment are randomly sampled.

[0157] In a possible implementation manner, when the target level information is the second level information, the verification module 404 is specifically configured to:

[0158] A full data block is extracted from the target data segment, and the full data block is determined as a target data block corresponding to the verification strategy.

[0159] In a possible implementation manner, the above device further includes a recording module, which is specifically configured to:

[0160] In response to a user-triggered instruction to access a target data segment, acquiring the target data segment;

[0161] Based on each data block included in the target data segment, verifying the data consistency of the target data segment;

[0162] After determining that the verification result is passed, access to the above target data segment is allowed, and the access information is recorded.

[0163] Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application, and the electronic device can realize the function of the aforementioned data verification device, referring to Figure 5 , the electronic device comprises:

[0164] At least one processor 501, and a memory 502 connected to the at least one processor 501. The specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present application. Figure 5 In the example, the processor 501 and the memory 502 are connected via a bus 500. The bus 500 is Figure 5 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 500 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller, and there is no limitation on the name.

[0165] In the embodiment of the present application, the memory 502 stores instructions that can be executed by at least one processor 501. The at least one processor 501 can execute the data verification method discussed above by executing the instructions stored in the memory 502. The processor 501 can implement Figure 4 The functions of each module in the device shown.

[0166] Among them, the processor 501 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 502 and calling data stored in the memory 502, the various functions of the device and processing data, the device can be monitored as a whole.

[0167] In one possible design, the processor 501 may include one or more processing units, and the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0168] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the data verification method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0169] The memory 502 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 502 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 502 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0170] By designing and programming the processor 501, the code corresponding to the application anomaly detection method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1 The steps of the data verification method of the embodiment shown are as follows: How to design and program the processor 501 is a technique known to those skilled in the art and will not be described in detail here.

[0171] An embodiment of the present application also provides a computer-readable storage medium that stores computer-executable instructions required to execute the above-mentioned processor, which includes a program for executing the above-mentioned processor.

[0172] In some possible implementations, various aspects of the data verification method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the above-mentioned program product is run on an electronic device, the above-mentioned program code is used to enable the above-mentioned electronic device to execute the steps of the data verification method according to various exemplary implementations of the present application described above in this specification.

[0173] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0174] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0177] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0178] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data verification method, characterized in that: The method comprises: Obtaining historical access information corresponding to each data segment included in the storage system; Based on the historical access information corresponding to each of the data segments, determining the level information corresponding to each of the data segments, wherein the level information is used to identify the frequency with which the data segment is accessed; In response to a user-triggered instruction to perform consistency check on a target data segment, acquiring the target data segment and target level information corresponding to the target data segment; Based on the verification strategy corresponding to the target level information, a target data block corresponding to the verification strategy is extracted from the target data segment, and data consistency of the target data block is verified.

2. The method according to claim 1, characterized in that The level information includes first level information and second level information, and determining the level information corresponding to each data segment based on the historical access information corresponding to each data segment includes: For each data segment, if the historical access information corresponding to the data segment is within a preset information value interval, it is determined that the data segment corresponds to first-level information, and the first-level information is used to indicate that: the frequency of access to the data segment is higher than the reference frequency; If the historical access information corresponding to the data segment is not within the information value interval, it is determined that the data segment corresponds to the second level information, and the second level information is used to indicate that: the frequency of access to the data segment is not higher than the reference frequency; The historical access information includes at least one of the number of accesses, access time and access frequency corresponding to the data segment within a preset time range.

3. The method according to claim 2, characterized in that When the target level information is first level information, the step of extracting a target data block corresponding to the verification strategy from the target data segment based on the verification strategy corresponding to the target level information includes: Based on the target sampling rule corresponding to the target data segment, randomly sampling the target data segment, and determining each obtained data block as a target data block corresponding to the verification strategy; Among them, the target sampling rule is determined based on the correspondence between each reference value range of pre-defined historical access information and the reference sampling rule, and the historical access information corresponding to the target data segment; the target sampling rule includes: at least one of the sampling data volume, sampling interval and sampling algorithm.

4. The method according to claim 3, characterized in that Before randomly sampling the target data segment based on the target sampling rule corresponding to the target data segment, the method further includes: Obtain the reference time interval corresponding to each data block in the target data segment, where the reference time interval is the time interval between the last consistency check of the data block and the current time: From the data blocks, select data blocks whose reference time interval is greater than a preset threshold; The randomly sampling the target data segment based on the target sampling rule corresponding to the target data segment includes: Based on the target sampling rule corresponding to the target data segment, the data blocks screened out from the target data segment are randomly sampled.

5. The method according to claim 2, characterized in that: When the target level information is the second level information, the step of extracting a target data block corresponding to the verification strategy from the target data segment based on the verification strategy corresponding to the target level information includes: A full data block is extracted from the target data segment, and the full data block is determined as a target data block corresponding to the verification strategy.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: In response to a user-triggered instruction to access a target data segment, acquiring the target data segment; Based on each data block included in the target data segment, verifying the data consistency of the target data segment; After determining that the verification result is passed, access to the target data segment is allowed, and the access information is recorded.

7. A data verification device, characterized in that: The device comprises: An acquisition module, used to acquire historical access information corresponding to each data segment included in the storage system; A determination module, configured to determine level information corresponding to each of the data segments based on historical access information corresponding to each of the data segments, wherein the level information is used to identify how frequently the data segment is accessed; A response module, configured to obtain the target data segment and target level information corresponding to the target data segment in response to an instruction triggered by a user to perform consistency check on the target data segment; A verification module is used to extract a target data block corresponding to the verification strategy from the target data segment based on the verification strategy corresponding to the target level information, and to verify the data consistency of the target data block.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to implement the method according to any one of claims 1 to 6 when executing a computer program stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 6.

Images