Data processing method and device, computer equipment and storage medium

Abstract

The invention discloses a data processing method and device, computer equipment and a storage medium, and relates to the technical field of data access. The method comprises the steps that according to a preset writing rule, data is written into a plurality of storage units in a target storage area in sequence, and the preset writing rule at least comprises a data writing sequence and a data writing strategy between the storage units adjacent to the data writing sequence; reading the data in the plurality of storage units in sequence based on the data writing sequence; matching the data in the current storage unit with the data in the preorder storage unit; and if the data in the current storage unit and the data in the preorder storage unit accord with the data writing strategy, determining that the data in the current storage unit passes the verification. Thus, it is not needed to introduce additional storage for storing the verification data, the cost of data reliability verification is saved, and meanwhile the problem that additional storage faults affect data verification is avoided.

Description

Technical Field

[0001] The present application relates to the field of data access technology, and more specifically, to a data processing method, device, computer equipment and storage medium. Background Art

[0002] In the related art, the current data reliability verification method is mainly based on comparison, which evaluates the integrity and accuracy of the data by comparing the tested data with the verification data, and the verification data is often stored in an additional storage device or storage system. In this way, the cost of data reliability verification is increased due to the need for an additional storage device or storage system; and the disconnection or failure of the storage device or storage system storing the verification data will also affect the results of the data reliability verification. Summary of the invention

[0003] The present application proposes a data processing method, apparatus, computer device and storage medium to improve the above-mentioned defects.

[0004] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising: writing data to a plurality of storage units in a target storage area in sequence according to a preset write rule, the preset write rule comprising at least a data write order and a data write strategy between storage units with adjacent write orders; reading the data in the plurality of storage units in sequence based on the data write order; matching the data in a current storage unit with the data in a preceding storage unit, the current storage unit being a storage unit read at a current moment among the plurality of storage units, the preceding storage unit being a storage unit read before the current storage unit; if the data in the current storage unit and the data in the preceding storage unit comply with the data write strategy, determining that the data in the current storage unit has passed verification.

[0005] In the second aspect, an embodiment of the present application provides a data processing device, which includes: a data writing module, a data reading module and a data verification module. Among them, the data writing module is used to write data to multiple storage units in the target storage area in sequence according to a preset writing rule, and the preset writing rule at least includes a data writing order and a data writing strategy between storage units with adjacent writing orders; the data reading module is used to read the data in the multiple storage units in sequence based on the data writing order; the data verification module is used to match the data in the current storage unit with the data in the previous storage unit, the current storage unit is the storage unit read at the current moment among the multiple storage units, and the previous storage unit is the storage unit that is read before the current storage unit; if the data in the current storage unit and the data in the previous storage unit meet the data writing strategy, it is determined that the data in the current storage unit has been verified.

[0006] In a third aspect, an embodiment of the present application further provides a computer device, comprising: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a program code is stored, and the program code can be called by a processor to execute the above method.

[0008] In the scheme provided by the present application, data is written to multiple storage units in the target storage area in sequence according to a preset write rule, and the preset write rule at least includes a data write order and a data write strategy between storage units with adjacent write orders; based on the data write order, data in multiple storage units are read in sequence; the data in the current storage unit is matched with the data in the previous storage unit, the current storage unit is the storage unit read at the current moment among the multiple storage units, and the previous storage unit is the storage unit read before the current storage unit; if the data in the current storage unit and the data in the previous storage unit meet the data write strategy, it is determined that the data in the current storage unit has passed the verification. In this way, there is no need to introduce additional storage for storing verification data, but by reading the data of the current storage unit and the previous storage unit, and matching and verifying, it can be determined whether the data in the current storage unit meets the preset write strategy to complete the data reliability verification, saving the cost of data reliability verification, and since there is no need to introduce additional storage, the occurrence of the problem of the impact of additional storage failure on data verification is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of a data processing method provided in an embodiment of the present application is shown.

[0011] Figure 2 A schematic diagram of the data writing and data verification process provided by an embodiment of the present application is shown.

[0012] Figure 3 A schematic diagram of dividing a storage unit provided in an embodiment of the present application is shown.

[0013] Figure 4 A schematic diagram of a linked list for writing data into a linked list provided in an embodiment of the present application is shown.

[0014] Figure 5 A schematic diagram of the contents of a storage unit provided in an embodiment of the present application is shown.

[0015] Figure 6 A flow chart of a data processing method provided in another embodiment of the present application is shown.

[0016] Figure 7 A schematic diagram of data writing into a storage unit provided in an embodiment of the present application is shown.

[0017] Figure 8 A data verification schematic diagram of a storage unit provided in an embodiment of the present application is shown.

[0018] Fig. 9 Shows Figure 6 Schematic diagram of the flow chart of the sub-steps of step S250.

[0019] Fig.10 It is a block diagram of a data processing device provided according to an embodiment of the present application.

[0020] Fig.11 It is a block diagram of a computer device according to an embodiment of the present application for executing a data processing method according to an embodiment of the present application.

[0021] Fig.12 It is a storage unit of an embodiment of the present application for storing or carrying program codes for implementing the data processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0023] It should be noted that in some processes described in the specification, claims and the above-mentioned drawings of the present application, multiple operations that appear in a specific order are included, and these operations may not be executed or executed in parallel in the order in which they appear in this article. The sequence numbers of operations such as S110, S120, etc. are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or less operations, and these operations may be executed in sequence or in parallel. And, the terms "first", "second", etc. in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0024] In related technologies, the verification method for data reliability is mainly based on comparison, which evaluates the integrity and accuracy of the data by comparing the tested data with the verification data. Specific comparison methods generally include copy comparison, verification value comparison, and idempotent data comparison.

[0025] Copy comparison, in addition to writing the data to the test target during the test, a complete copy of the data needs to be written to another storage specifically used to store verification data (this storage can be various types of storage devices or storage systems). During verification, the reliability of the data is judged by reading the two storage data for comparison. Obviously, the copy comparison method requires additional storage devices or storage systems, and the storage overhead is one-to-one, which increases the cost of data reliability verification; and the disconnection or failure of the storage device or storage system storing the verification data will also affect the verification test results.

[0026] Verification value comparison, in addition to writing data to the test target during the test process, the verification value calculated by the data test will be written into another storage specifically used to store verification data. During verification, the data is read and the verification value is recalculated and compared with the original verification value to judge the reliability of the data. Obviously, the verification value comparison method also requires additional storage devices or storage systems, which increases the cost of data reliability verification; and the disconnection or failure of the storage device or storage system storing the verification data will also affect the verification test results; at the same time, additional computing power is required to calculate the verification value.

[0027] Idempotent data comparison: data written during the test is generated by calculation based on a certain "seed data". In addition to writing the data to the test target, another storage record will be used to record whether an update has occurred at a certain location and its sequence number (metadata). During verification, data is generated again by reading the data and the "seed data". At the same time, metadata information is read and compared with the content of the storage target to judge the reliability of the data. Obviously, the idempotent data comparison method also requires additional storage devices or storage systems, which increases the cost of data reliability verification; and the disconnection or failure of the storage device or storage system storing the verification data will also affect the verification test results.

[0028] It can be seen that the data reliability verification methods in the related art all rely on additional storage devices or storage systems, which have the problem of high cost; and the disconnection or failure of the additional storage devices or storage systems will also affect the verification test results.

[0029] Based on the above problems, the inventor proposes a data processing method, device, computer equipment and storage medium. The data processing method provided in the embodiment of the present application is described in detail below.

[0030] Please refer to Figure 1 , Figure 1 A flowchart of a data processing method provided in an embodiment of the present application is shown below. Figure 1 The data processing method provided in the embodiment of the present application is described in detail. The data processing method may include the following steps:

[0031] Step S110: writing data to a plurality of storage units in the target storage area in sequence according to a preset writing rule, wherein the preset writing rule at least includes a data writing sequence and a data writing strategy between storage units with adjacent writing sequences.

[0032] In this example, see Figure 2 First, the metadata will be initialized, that is, the metadata parameters will be written to Figure 3The metadata area corresponding to the target storage area shown is in the metadata area. Metadata, also known as intermediate data or relay data, is data that describes data. It is mainly information that describes data attributes and is used to support functions such as indicating storage location, historical data, resource search, and file records. Metadata is a kind of electronic catalog. In order to achieve the purpose of cataloging, it is necessary to describe and collect the content or characteristics of the data, thereby achieving the purpose of assisting data retrieval.

[0033] The metadata parameters at least include a storage unit division strategy. Then, the target storage area can be divided according to the storage unit division strategy to obtain multiple storage units, such as Figure 3 Storage units 1 to 16 are shown.

[0034] Optionally, the storage unit division strategy may include a preset unit size of the storage unit. In this case, the size of each storage unit in the multiple storage units obtained by dividing the target storage area is the preset unit size. For example, the preset unit size of the storage unit is a hardware sector of 512 bytes. In this case, the size of each storage unit in the multiple storage units obtained by dividing the target storage area is 512 bytes. Among them, the sector refers to the area divided on the disk. Each track on the disk is divided into several arc segments, which are the sectors of the disk. The reading and writing of the hard disk is based on the sector as the basic unit.

[0035] Optionally, the storage unit division strategy may also include a preset number of storage units, in which case the target storage area is evenly divided into the preset number of storage units. For example, the size of the target storage area is 1024 bytes, and the preset number is 16, in which case the target storage area is evenly divided into 16 storage units. Of course, the specific method used to divide the target storage area into multiple storage units is not limited in this embodiment.

[0036] Furthermore, after completing the division of the target storage area, data can be written to multiple storage units in the target storage area in sequence based on the data writing order and the data writing strategy between storage units with adjacent writing orders. Specifically, the aforementioned initialized metadata parameters may include, in addition to the storage unit division strategy, a preset series connection algorithm, a data writing strategy, and a preset generation algorithm. The preset series connection algorithm is used to indicate the data writing order of writing data to multiple storage units. It should be noted that after multiple storage units are divided, a unit identifier will be assigned to each storage unit, and the unit identifier may be a digital identifier.

[0037] That is to say, based on the preset series connection algorithm, the storage unit currently to be added to the write linked list is determined as the target storage unit from multiple storage units. Among them, the preset series connection algorithm can be to connect the storage units in series in the order of digital identifiers from small to large to form a write linked list. The preset series connection algorithm can also be to connect the storage units in series in the order of digital identifiers from large to small to form a write linked list. The preset series connection algorithm can also be to connect the storage units with odd digital identifiers in the order of odd digital identifiers from small to large, and then connect them in series in the order of even digital identifiers from small to large to form a write linked list. Figure 4 Of course, the preset series connection algorithm can also be to connect the storage units with odd digital identifiers in descending order, and then connect them in descending order according to the even digital identifiers to form a write chain list. The preset series connection algorithm can also be to randomly combine multiple storage units to form a write chain list.

[0038] It should be noted that the header written into the linked list is determined based on the preset concatenation algorithm in the metadata area, such as Figure 5 As shown, after determining that the head of the write linked list is storage unit 1 based on the preset concatenation algorithm in the metadata area, the unit identifier of storage unit 1 is recorded in the metadata area. Further, for storage unit 1, the preset generation algorithm in the metadata area is used to generate data for writing into storage unit 1, and then the generated data is written into storage unit 1.

[0039] Furthermore, if the target storage unit is not the head of the write linked list, the data to be written is generated using a preset generation algorithm based on the data in the storage unit that is added to the write linked list before the target storage unit. Figure 5 The storage unit 2 in the target storage unit is the storage unit 1 that is added to the write linked list before the target storage unit. At this time, the preset generation algorithm is used to generate the data to be written according to the data in the storage unit 1. Finally, based on the data writing strategy between the storage units adjacent in the writing order, the data to be written is written to the target storage unit. Among them, the data writing strategy can be understood as:

[0040] That is to say, this embodiment writes data to multiple storage units in the target storage area in the form of a linked list until all storage units have completed data writing; wherein, the length of the linked list depends on the size of the metadata area, such as the storage unit number is stored in 4 bytes, the metadata area is 64 bytes, and the length of the linked list is 64 / 4=16. It should be noted that if the linked list has reached the maximum length, such as 16, when writing data in the form of a linked list, the next storage unit to be written is determined according to the data writing order, and the address of the storage unit is hung at the end of the linked list, and then the unit closest to the head of the linked list is removed to ensure that the length of the linked list is fixed, and then the data is written to the storage unit to be written.

[0041] Step S120: based on the data writing order, sequentially read the data in the plurality of storage units.

[0042] Furthermore, if Figure 2 As shown, after completing the writing of data to multiple storage units in the target storage area, the metadata area can be read, and based on the header identifier in the header in the metadata area, the storage unit that needs to be read at the current moment is determined from the multiple storage units. Figure 5 As shown, by reading the metadata area, it can be learned that in the data writing stage, the header of the write linked list is storage unit 1, so storage unit 1 is used as the current storage unit to read data.

[0043] Step S130: Match the data in the current storage unit with the data in the previous storage unit, wherein the current storage unit is the storage unit currently read among the multiple storage units, and the previous storage unit is the storage unit that was read before the current storage unit.

[0044] Step S140: If the data in the current storage unit and the data in the previous storage unit comply with the data writing strategy, it is determined that the verification of the data in the current storage unit has passed.

[0045] Since in the data writing stage, data is written to the storage units adjacent in the writing order based on the data writing strategy between the storage units adjacent in the writing order; that is, there is a certain regularity in the data in any two adjacent storage units. Therefore, in order to verify the reliability of the data in multiple storage units, after completing the data writing to multiple storage units, the data in each storage unit can be read in turn based on the data writing order, and the data in the current storage unit read is matched with the data in the previous storage unit and the data in the adjacent previous storage unit to determine whether the data in the current storage unit and the data in the adjacent previous storage unit comply with the data writing strategy in the data writing stage. Optionally, if the data in the current storage unit and the data in the adjacent previous storage unit comply with the data writing strategy, it indicates that the data in the current storage unit has integrity and reliability, that is, it is determined that the data in the current storage unit has passed the verification.

[0046] In this embodiment, there is no need to introduce additional storage for storing verification data. Instead, by reading the data of the current storage unit and the previous storage unit, and performing matching and verification, it is possible to determine whether the data in the current storage unit complies with the preset write policy to complete the data reliability verification, thereby saving the cost of data reliability verification. In addition, since there is no need to introduce additional storage, it is avoided that during the execution of the fault scenario case, the business branch of the storage system in the fault scenario is not covered as expected due to the unavailability of the additional storage itself, which ultimately leads to the failure to timely discover the reliability problem of the storage system.

[0047] Please refer to Figure 6 , Figure 6 A flowchart of a data processing method provided in another embodiment of the present application is shown below. Figure 6 The data processing method provided in the embodiment of the present application is described in detail. The data processing method may include the following steps:

[0048] Step S210: writing data to a plurality of storage units in the target storage area in sequence according to a preset writing rule, wherein the preset writing rule at least includes a data writing sequence and a data writing strategy between storage units with adjacent writing sequences.

[0049] Step S220: Read the metadata area, where the metadata area at least includes magic number information.

[0050] Step S230: If the magic number information in the metadata area meets a preset condition, the data in the plurality of storage units are read in sequence based on the data writing sequence.

[0051] The magic number information is parameter information used to determine the file type. The content of the first few bytes in the file is fixed (either intentionally filled or originally so). The content of these bytes is also called the magic number, because the file type can be determined based on the content of these bytes. Obviously, in order to ensure that the metadata area read is correct, the magic number information can be pre-set in the metadata area. Based on this, in the process of verifying the data reliability of the target storage area, the metadata area corresponding to the target storage area is first read, and the magic number information in the metadata area is obtained, and then it is determined whether the content of the first few bytes of the metadata area (i.e., the magic number information) is preset information. If the magic number information is preset information, it is determined that the magic number information in the metadata area matches the preset information; if the magic number information is not preset information, it is determined that the magic number information in the metadata area does not match the preset information, that is, it indicates that the storage area currently read is not the metadata area. In the case where the magic number information in the metadata area matches the preset information, it indicates that the data area read at this time is indeed the metadata area, and further based on the data writing order, the data in multiple storage units are read in sequence. In this way, when the magic number information in the metadata area matches the preset information, it indicates that the data integrity is good, and the subsequent data reading operation is continued, thereby enhancing the security and reliability of data reading.

[0052] Step S240: Match the header data in the current storage unit with the tail data in the previous storage unit, wherein the header data is the other data in the current storage unit except the first data, and the tail data is the other data in the previous storage unit except the last data.

[0053] In some embodiments, the write strategy between storage units that are adjacent in writing order may be to remove the first data in the preceding storage unit and add the target data to the end of the data in the preceding storage unit after the first data is removed; wherein the target data may be generated based on the data in the preceding storage unit using a preset generation algorithm. Finally, the data in the preceding storage unit with the target data added thereto is written to the current storage unit. For example, see Figure 7 , the data in the preceding storage unit can be {23456}. Based on the aforementioned write strategy, the first data 2 in the preceding storage unit is removed, and the preset generation algorithm is used to generate the target data 7 based on the data in the preceding storage unit. Finally, the target data 7 is added to the tail of the data {3456} in the preceding storage unit after the first data is removed, and the data in the preceding storage unit with the target data added is written into the current storage unit, that is, the data in the current storage unit is finally {34567}.

[0054] In this way, in the data reading stage, the data in the current storage unit is matched with the data in the previous storage unit, that is, the head data in the current storage unit is matched with the tail data in the previous storage unit. The head data is the data other than the first data in the current storage unit, and the tail data is the data other than the last data in the previous storage unit. Obviously, if the head data in the current storage unit is the same as the tail data in the previous storage unit, such as Figure 8 As shown, the header data in the current storage unit and the tail data in the previous storage unit are both {3456}, then it can be determined that the header data in the current storage unit matches the tail data in the previous storage unit. Optionally, if the header data in the current storage unit is different from the tail data in the previous storage unit, for example, the header data in the current storage unit is {3456}, and the tail data in the previous storage unit is {3458}, then it can be determined that the header data in the current storage unit does not match the tail data in the previous storage unit.

[0055] Step S250: If the header data in the current storage unit matches the tail data in the previous storage unit, it is determined that the data in the current storage unit has passed the verification.

[0056] Furthermore, when it is determined that the header data in the current storage unit matches the tail data in the previous storage unit, it can be determined that the reliability check of the data in the current storage unit has passed. In this way, the data reliability test of the storage system is implemented directly by verifying the data in each storage unit in the target storage area, without the need to introduce additional storage devices or storage systems for storing verification data, thereby improving the availability of the test system itself and reducing the test cost.

[0057] Optionally, if the header data in the current storage unit does not match the tail data in the previous storage unit, it is determined that the data verification in the current storage unit has failed, and a prompt message is output to prompt the tester that the data in the current storage unit may be tampered with or damaged.

[0058] In this embodiment, Figure 4 Taking the writing order as an example, storage unit 1, storage unit 2, storage unit 5 ... storage unit 14 and storage unit 16 will be read in sequence, and the data verification process from step S280 to step S290 will be performed.

[0059] In some embodiments, see Fig. 9 , step S250 may include the contents of step S251 to step S253:

[0060] Step S251: If the header data in the current storage unit matches the tail data in the previous storage unit, verification data is generated according to a preset generation algorithm and the data in the previous storage unit.

[0061] After determining that the header data in the current storage unit matches the tail data in the previous storage unit, in order to improve the accuracy of data verification, verification data for the current storage unit can be further generated according to a preset generation algorithm and the data in the previous storage unit.

[0062] Optionally, if the pre-set generation algorithm is not included in the preceding storage unit, verification data is generated according to the pre-set generation algorithm in the metadata area and the data in the preceding storage unit. That is to say, the write data for all storage units is generated by the pre-set generation algorithm in the metadata area. In this way, in the data writing stage, a unified pre-set generation algorithm can be used to quickly generate data and write it to the current storage unit; in the verification stage after data reading, a unified pre-set generation algorithm can also be used to quickly generate verification data.

[0063] Optionally, if the preceding storage unit includes a preset generation algorithm, the verification data is generated according to the preset generation algorithm in the preceding storage unit and the data in the preceding storage unit. That is to say, corresponding preset generation algorithms can be set for different storage units in the target storage area, that is, the preset generation algorithms corresponding to different storage units can be different or the same, wherein the preset generation algorithm used by each storage unit can be stored in the storage unit before the adjacent writing sequence. In this way, the probability of data collision between storage units can be lower. Furthermore, in the verification stage after the data is read, the verification data is generated for each current storage unit using the preset generation algorithm in its preceding storage unit.

[0064] Step S252: Match the data in the current storage unit with the verification data.

[0065] Step S253: If the data in the current storage unit matches the verification data, it is determined that the verification of the data in the current storage unit has passed.

[0066] Furthermore, the verification data generated for the current storage unit is matched with the data in the current storage unit read, that is, it is determined whether the verification data generated by the current storage unit is equal to the data in the current storage unit. If they are equal, it is determined that the data in the current storage unit matches the verification data, and then it can be determined that the data in the current storage unit has passed the verification. If they are not equal, it is determined that the data in the current storage does not match the verification data, and then it is determined that the data in the current storage unit has failed the verification, and a prompt message is output to prompt that the data in the current storage unit may be tampered with or damaged.

[0067] Optionally, the verification data generated for the current storage unit may be the last data in the current storage unit. That is, when the header data in the current storage unit matches the tail data in the previous storage unit, the verification data generated for the current storage unit will be further matched with the last data in the current storage unit read. If the verification data is the same as the last data in the current storage unit read, the characterization data verification passes; if the verification data is not the same as the last data in the current storage unit read, the characterization data verification fails.

[0068] Optionally, if the current storage unit is the first storage unit in the data writing sequence, there is no need to perform step S240 and step S250, and verification data is directly generated according to the preset generation algorithm and the data in the preceding storage unit; all the data read in the current storage unit are matched with the verification data, and if all the data in the current storage unit is the same as the verification data, it is determined that the two match, and then it is determined that the data in the current storage unit has passed the verification. If part of the data in the current storage unit is different from the verification data, it is determined that the two do not match, and it is determined that the data in the current storage unit has failed the verification; and a prompt message is output to prompt the tester that the data in the current storage area is damaged or tampered with.

[0069] In some embodiments, the metadata area may also include a timestamp. If it is determined that the data in the current storage unit fails the verification, the timestamp corresponding to the current storage unit can be obtained. If the timestamp is a preset timestamp, it is determined that the data in the current storage area is damaged; if the timestamp is not a preset timestamp, and the time represented by the timestamp is closer to the current time than the time represented by the preset timestamp, it is determined that the data in the current storage unit is tampered with.

[0070] In this embodiment, data reliability testing of the storage system is achieved by mutually verifying data in adjacent storage units within the target storage area, without the need to introduce additional storage for storing verification data, thereby improving the availability of the test system itself and reducing testing costs.

[0071] Please refer to Fig.10 , which shows a structural block diagram of a data processing device 300 provided in an embodiment of the present application. The device 300 may include: a data writing module 310, a data reading module 320 and a data checking module 330.

[0072] The data writing module 310 is used to write data to multiple storage units in the target storage area in sequence according to a preset writing rule, wherein the preset writing rule at least includes a data writing sequence and a data writing strategy between storage units with adjacent writing sequences.

[0073] The data reading module 320 is used to read the data in the plurality of storage units in sequence based on the data writing order.

[0074] The data verification module 330 is used to match the data in the current storage unit with the data in the previous storage unit, where the current storage unit is the storage unit read at the current moment among the multiple storage units, and the previous storage unit is the storage unit that was read before the current storage unit; if the data in the current storage unit and the data in the previous storage unit comply with the data write strategy, it is determined that the data in the current storage unit has been verified.

[0075] In some embodiments, the data verification module 330 can be specifically used to: match the header data in the current storage unit with the tail data in the previous storage unit, the header data being the other data in the current storage unit except the first data, and the tail data being the other data in the previous storage unit except the last data; if the header data in the current storage unit matches the tail data in the previous storage unit, it is determined that the data in the current storage unit has been verified.

[0076] In some embodiments, the data verification module 330 may further include: a verification data generation unit and a verification unit. Among them, the first verification unit may be used to generate verification data according to a preset generation algorithm and the data in the preceding storage unit if the header data in the current storage unit matches the tail data in the preceding storage unit. The verification unit may be used to match the data in the current storage unit with the verification data; if the data in the current storage unit matches the verification data, it is determined that the data in the current storage unit has been verified.

[0077] In this manner, the metadata area corresponding to the target storage area includes a preset generation algorithm, and the verification data generation unit can be specifically used to: if the preset generation algorithm is not included in the pre-order storage unit, generate the verification data according to the preset generation algorithm in the metadata area and the data in the pre-order storage unit; if the pre-order storage unit includes a preset generation algorithm, generate the verification data according to the preset generation algorithm in the pre-order storage unit and the data in the pre-order storage unit.

[0078] In some embodiments, the data processing device 300 may include: a metadata reading module and a magic number checking module. The metadata reading module may be used after writing data to multiple storage units in the target storage area in sequence according to the preset writing rule. The magic number checking module may be used to execute the reading of data in the multiple storage units in sequence based on the data writing order if the magic number information in the metadata area matches the preset information.

[0079] In some embodiments, the data processing device 300 may include: a metadata writing module and a storage partitioning module. The metadata writing module may be used to write metadata parameters to the metadata area corresponding to the target storage area before writing data to the multiple storage units in the target storage area in sequence according to the preset writing rule, and the metadata parameters at least include a storage unit partitioning strategy. The storage partitioning module may be used to partition the target storage area according to the storage unit partitioning strategy to obtain the multiple storage units.

[0080] In this manner, the data writing module 310 can be specifically used to determine, from the multiple storage units, a storage unit currently to be added to the write linked list as a target storage unit based on the preset series connection algorithm; generate data to be written based on the preset generation algorithm and data in the storage unit that was previously added to the write linked list before the target storage unit; and write the data to be written into the target storage unit based on the data writing strategy.

[0081] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0082] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0083] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0084] In summary, according to the preset write rule, data is written to multiple storage units in the target storage area in sequence, and the preset write rule at least includes the data write order and the data write strategy between storage units with adjacent write orders; based on the data write order, the data in the multiple storage units are read in sequence; the data in the current storage unit is matched with the data in the previous storage unit, the current storage unit is the storage unit read at the current moment among the multiple storage units, and the previous storage unit is the storage unit read before the current storage unit; if the data in the current storage unit and the data in the previous storage unit meet the data write strategy, it is determined that the data in the current storage unit is verified. In this way, there is no need to introduce additional storage for storing verification data, but by reading the data of the current storage unit and the previous storage unit, and matching and verifying, it can be determined whether the data in the current storage unit meets the preset write strategy to complete the data reliability verification, saving the cost of data reliability verification, and because there is no need to introduce additional storage, it is avoided that the storage system fails to cover the business branch as expected in the fault scenario due to the unavailability of the additional storage itself during the execution of the fault scenario case, which ultimately leads to the failure to discover the reliability problem of the storage system in time.

[0085] The following will be combined Fig.11 A computer device provided by the present application is described.

[0086] Reference Fig.11 , Fig.11 The structural block diagram of a computer device 400 provided in an embodiment of the present application is shown, and the above method provided in an embodiment of the present application can be executed by the computer device 400. The computer device can be an electronic terminal with a data processing function, and the electronic terminal includes but is not limited to a tablet computer, a laptop computer, and a desktop computer; of course, the computer device can also be a server, and the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content De livery Network, CDN), and big data and artificial intelligence platforms.

[0087] The computer device 400 in the embodiment of the present application may include one or more of the following components: a processor 401, a memory 402, and one or more applications, wherein the one or more applications may be stored in the memory 402 and configured to be executed by one or more processors 401, and the one or more programs are configured to execute the method as described in the aforementioned method embodiment.

[0088] The processor 401 may include one or more processing cores. The processor 401 uses various interfaces and lines to connect various parts of the entire computer device 400, and executes various functions and processes data of the computer device 400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 402, and calling data stored in the memory 402. Optionally, the processor 401 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 401 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem can also be integrated into the processor 401 and implemented separately through a communication chip.

[0089] The memory 402 may include a random access memory (RAM) or a read-only memory (ROM). The memory 402 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as the various corresponding relationships described above) created by the computer device 400 during use.

[0090] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0091] In several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or modules, which may be electrical, mechanical or other forms.

[0092] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0093] Please refer to Fig.12 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 500 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0094] The computer readable storage medium 500 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer readable storage medium 500 has storage space for program code 510 that executes any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 510 can be compressed, for example, in an appropriate form.

[0095] In some embodiments, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Writing data to a plurality of storage units in the target storage area in sequence according to a preset writing rule, wherein the preset writing rule at least includes a data writing sequence and a data writing strategy between storage units with adjacent writing sequences; Based on the data writing order, sequentially reading the data in the plurality of storage units; Matching data in a current storage unit with data in a previous storage unit, wherein the current storage unit is a storage unit read at the current moment among the multiple storage units, and the previous storage unit is a storage unit read before the current storage unit; If the data in the current storage unit and the data in the previous storage unit comply with the data writing strategy, it is determined that the data in the current storage unit has passed the verification.

2. The method according to claim 1, characterized in that The matching of the data in the current storage unit with the data in the previous storage unit includes: Matching the header data in the current storage unit with the tail data in the previous storage unit, wherein the header data is other data in the current storage unit except the first data, and the tail data is other data in the previous storage unit except the last data; If the data in the current storage unit and the data in the previous storage unit meet the preset write rule, determining that the data in the current storage unit has passed the verification includes: If the header data in the current storage unit matches the tail data in the preceding storage unit, it is determined that the data in the current storage unit has passed the verification.

3. The method according to claim 2, characterized in that If the tail data in the current storage unit matches the head data in the previous storage unit, determining that the data in the current storage unit has passed the verification includes: If the header data in the current storage unit matches the tail data in the previous storage unit, generating verification data according to a preset generation algorithm and the data in the previous storage unit; Matching the data in the current storage unit with the verification data; If the data in the current storage unit matches the verification data, it is determined that the verification of the data in the current storage unit passes.

4. The method according to claim 3, characterized in that The metadata area corresponding to the target storage area includes a preset generation algorithm, and the verification data is generated according to the preset generation algorithm and the data in the preceding storage unit, including: If the pre-order storage unit does not contain a preset generation algorithm, generating the verification data according to the preset generation algorithm in the metadata area and the data in the pre-order storage unit; If the preceding storage unit contains a preset generation algorithm, the verification data is generated according to the preset generation algorithm in the preceding storage unit and the data in the preceding storage unit.

5. The method according to claim 1, characterized in that After writing data to the plurality of storage units in the target storage area in sequence according to the preset writing rule, the method further includes: Reading a metadata area, wherein the metadata area includes at least magic number information; If the magic number information in the metadata area matches the preset information, the data in the plurality of storage units are sequentially read based on the data writing sequence.

6. The method according to any one of claims 1 to 5, characterized in that: Before writing data to a plurality of storage units in the target storage area in sequence according to the preset writing rule, the method further includes: Writing metadata parameters into a metadata area corresponding to the target storage area, the metadata parameters at least including a storage unit division strategy; The target storage area is divided according to the storage unit division strategy to obtain the multiple storage units.

7. The method according to claim 6, characterized in that The metadata parameters further include a preset concatenation algorithm, the data writing strategy, and a preset generation algorithm, wherein the preset concatenation algorithm is used to indicate the data writing order of writing data to the plurality of storage units; The step of sequentially writing data to a plurality of storage units in the target storage area according to a preset writing rule includes: Based on the preset series connection algorithm, determining, from the plurality of storage units, a storage unit currently to be added to the write linked list as a target storage unit; Generate the data to be written by using the preset generation algorithm and according to the data in the storage unit that is previously added to the write linked list and located in the target storage unit; Based on the data writing strategy, the data to be written is written into the target storage unit.

8. A data processing device, characterized in that: The device comprises: A data writing module, used to write data to a plurality of storage units in a target storage area in sequence according to a preset writing rule, wherein the preset writing rule at least includes a data writing sequence and a data writing strategy between storage units with adjacent writing sequences; A data reading module, configured to read the data in the plurality of storage units in sequence based on the data writing order; A data verification module is used to match the data in the current storage unit with the data in the previous storage unit, wherein the current storage unit is the storage unit read at the current moment among the multiple storage units, and the previous storage unit is the storage unit that was read before the current storage unit; if the data in the current storage unit and the data in the previous storage unit comply with the data write strategy, it is determined that the data in the current storage unit has been verified.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images