Hardware data repair method and system for heterogeneous data

Abstract

The present application relates to the technical field of data recovery, in particular to a hardware data repair method and system for heterogeneous data.A hardware data repair system for heterogeneous data comprises a data recovery request uploading module, a data recovery operation selection module and an online data recovery operation module.The present application constructs the storage device to be executed by the user to perform data recovery into a virtual digital body, and the data recovery operation for the virtual digital body can be executed without the operation for the storage device body, thereby avoiding further damage to the storage device and ensuring the normal progress of data recovery;the data block is divided and recombined based on the byte distribution characteristics, the file as a whole can be accurately identified in the case of fragmented storage, the data is preliminarily recovered through the anchoring of the identifier and the decoding of the encoding mode, the preliminary preparation work of data recovery is automatically completed, and the efficiency of the data recovery operation is improved.

Description

Technical Field

[0001] This invention relates to the field of data recovery technology, specifically to a hardware data repair method and system for heterogeneous data. Background Technology

[0002] Storage devices often suffer from file system corruption, data fragmentation, accidental deletion, and formatting after prolonged use or during malfunctions, rendering the original data inaccessible. Existing data recovery methods typically rely on file system metadata or disk imaging tools, but these methods have significant limitations when dealing with fragmented storage devices, corrupted file systems, or quick formatting. Especially when the storage device is unrecognized by the operating system, traditional recovery tools often fail to effectively identify valid data on the device, resulting in a complex and inefficient recovery process. Summary of the Invention

[0003] This invention constructs the storage device from which the user intends to perform data recovery as a virtual digital entity. Data recovery operations are performed on this virtual digital entity without requiring physical operations on the storage device itself, thus avoiding further damage and ensuring successful data recovery. Furthermore, it divides and reassembles data blocks based on byte distribution characteristics, enabling accurate identification of the entire file even in fragmented storage conditions. Through identifier anchoring and encoding decoding, it performs preliminary data recovery, automatically completing the initial preparation work for data recovery and improving the efficiency of the data recovery operation.

[0004] This invention provides a hardware data repair method for heterogeneous data, comprising:

[0005] Get the data recovery request from the user;

[0006] The system determines the type of fault in the data recovery request. If the fault type is physical damage, an offline data recovery work order and contract are created, and the offline data recovery operation is executed. If the fault type is not physical damage, an online data recovery work order and contract are created, and the online data recovery operation is executed.

[0007] The online data recovery operation specifically includes scanning the storage device to be recovered using a disk copying tool, constructing a corresponding virtual digital body for the storage device, which includes a mapping of all data recorded on the storage device, arranging all data marked as free space in the virtual digital body according to the address order of the storage device to form a dataset to be recovered, dividing the dataset into data blocks to obtain several data blocks to be recovered, and determining the character distribution feature vector of the data blocks to be recovered. Each data item in the character distribution feature vector is the normalized value of the number of times the corresponding byte pattern appears in the data block to be recovered.

[0008] Based on the character distribution feature vector of the data blocks to be recovered, all data blocks to be recovered are divided into several data sets of files to be recovered. Each data set of files to be recovered stores several data blocks of files to be recovered belonging to the same file.

[0009] For each dataset of files to be recovered, determine the file type corresponding to the dataset, anchor the dataset to be recovered based on the identifier corresponding to the file type, and then decode the dataset to be recovered through encoding to output the data recovery file.

[0010] As a preferred approach, the dataset to be recovered is divided into data blocks to obtain several data blocks to be recovered, and the character distribution feature vector of the data blocks to be recovered is determined. This specifically includes the following steps:

[0011] The dataset to be recovered is divided into several data blocks to be processed according to a preset window;

[0012] For each data block to be processed, a byte distribution analysis operation is performed. Starting from the first position of the data block, byte pattern statistics are performed to construct all byte sequence data and character distribution feature vectors. Then, starting from the second position of the data block, byte pattern statistics are performed again to construct all byte sequence data and character distribution feature vectors. This process is repeated, with the starting position of each byte pattern statistics shifted one data position forward until it has been shifted 8 times. The character distribution feature vectors are then output. All character distribution feature vectors are sequentially fed into the byte distribution analysis model for processing. The file type and its corresponding confidence score are output. The highest confidence score is selected as the target confidence score, and the file type corresponding to the target confidence score is selected as the target file type. All byte sequence data corresponding to the target confidence score are combined to form the target byte partition sequence and output.

[0013] Iterate through all data blocks to be processed. If the target text types of two adjacent data blocks are the same, merge the two adjacent data blocks. If the target text types of two adjacent data blocks are different, determine whether a data block move operation has been performed. The data block move operation is to move the dividing point between two adjacent data blocks forward or backward by 8 data positions. If a data block move operation has been performed, select the currently selected data block to be processed and perform a change rate judgment operation. If no data block move operation has been performed, randomly perform a data block move operation and then perform a change rate judgment operation.

[0014] The rate of change judgment operation is as follows: First, determine whether the target confidence level corresponding to the data to be processed is positive. If the target confidence level is positive, determine whether it is higher than the confidence threshold. If it is not higher than the confidence threshold, continue with the same data block movement operation as before, and continue with the rate of change judgment operation after performing the byte distribution analysis operation. If the target confidence level is higher than the confidence threshold, fix the split point between two adjacent data blocks and continue traversing the next two adjacent data blocks. If the target confidence level is not positive, perform the opposite data block movement operation as before, and continue with the rate of change judgment operation after performing the byte distribution analysis operation.

[0015] Once the split points between all adjacent data blocks to be processed have been fixed, the data blocks to be processed are recorded as data blocks to be recovered, and the character distribution feature vector and the corresponding target byte partition sequence of the data blocks to be recovered are output.

[0016] As a preferred aspect, byte pattern statistics specifically involve dividing the data block to be processed into 8 data bits to obtain all byte sequence data. The byte sequence is the data of the 8 data bits that are divided. The occurrence frequency of byte patterns in all byte sequence data is counted, and the normalized values ​​of the occurrence frequency of all byte patterns are used to form a character distribution feature vector. Each item in the character distribution feature vector corresponds to a byte pattern.

[0017] As a preferred aspect, all data blocks to be recovered are divided into several data sets to be recovered based on the character distribution feature vector of the data blocks to be recovered. Specifically, the following steps are included: performing cluster analysis based on the character distribution feature vector of the data blocks to be recovered, and forming a data set to be recovered from all the data sets in each cluster. Furthermore, all the data sets to be recovered from the data sets to be recovered are arranged in the address order within the storage device.

[0018] As a preferred approach, during the process of anchoring the dataset of files to be recovered based on the identifier corresponding to the file type, the bytes in the target byte segmentation sequence are queried and matched with the identifier corresponding to the file type, thereby achieving anchoring.

[0019] As a preferred approach, training the byte distribution analysis model includes the following steps:

[0020] Obtain several training samples labeled with file types. The training samples include character distribution feature vectors. Combine all the training samples labeled with file types into a training set. Train the byte distribution analysis model using the training set. During training, use the labeled file types and their corresponding confidence scores as the target output. The confidence score for the labeled file types is set to 1 by default. If the accuracy of the byte distribution analysis model is higher than the accuracy threshold, output the trained byte distribution analysis model. Otherwise, continue training the byte distribution analysis model using the training set.

[0021] This invention also provides a hardware data repair system for heterogeneous data, comprising:

[0022] The data recovery request upload module is used to obtain data recovery requests from users;

[0023] The data recovery operation selection module is used to determine the fault type in the data recovery request. If the fault type in the data recovery request is physical damage, an offline data recovery work order and contract are created, and the offline data recovery operation is executed. If the fault type in the data recovery request is not physical damage, an online data recovery work order and contract are created, and the online data recovery operation is executed.

[0024] The online data recovery module is used to perform online data recovery operations. Specifically, the online data recovery operation includes: scanning the storage device to be recovered using a disk copying tool; constructing a corresponding virtual digital entity for the storage device, which includes a mapping of all data burned into the storage device; arranging all data marked as free space in the virtual digital entity according to the storage device's address order to form a dataset to be recovered; dividing the dataset into several data blocks to be recovered, and determining the character distribution feature vector of each data block, where each item is the normalized value of the frequency of the corresponding byte pattern in the data block; dividing all data blocks to be recovered based on the character distribution feature vector to form several file datasets to be recovered, each containing several data blocks belonging to the same file; for each file dataset to be recovered, determining the file type, anchoring the dataset according to the identifier corresponding to the file type, and then decoding the dataset using an encoding method to output the recovered data file.

[0025] The present invention has the following advantages:

[0026] This invention constructs the storage device from which the user intends to perform data recovery as a virtual digital entity. Data recovery operations are performed on this virtual digital entity without requiring physical operations on the storage device itself, thus avoiding further damage and ensuring successful data recovery. Furthermore, it divides and reassembles data blocks based on byte distribution characteristics, enabling accurate identification of the entire file even in fragmented storage conditions. Through identifier anchoring and encoding decoding, it performs preliminary data recovery, automatically completing the initial preparation work for data recovery and improving the efficiency of the data recovery operation. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the hardware data repair method and system for heterogeneous data used in an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this invention.

[0029] Example 1: A hardware data repair method for heterogeneous data, comprising:

[0030] The system retrieves data recovery requests from users. These requests include descriptions of the storage devices requiring data recovery, such as customer company, contact person, contact information, address, serial number, capacity, brand, fault type, and fault description. The fault description is a user-defined description of the storage device, such as "Data was lost due to a power outage in the D: / xxx / xxx / xxx folder." Fault types in the requests include accidental deletion, quick formatting, full formatting, file system corruption, and physical damage. It should be noted that this application pertains to a data recovery platform where users can register, log in, and place data recovery orders. Several engineers are registered on the platform. Because the storage devices ordered by users have different hardware architectures, storage media, data formats, technical standards, and operating systems, the data recovery platform is designed for heterogeneous data.

[0031] It's important to clarify that accidental deletion refers to a user or system accidentally deleting a file or folder (meaning permanent deletion within the system). The data itself doesn't disappear from the storage device; instead, it's marked as deleted and the space is marked as free by the operating system, making it inaccessible. If this data hasn't been overwritten by new data, it can be recovered. Quick formatting resets the file system, making it believe all space on the storage device is free. In reality, the data still exists on the storage device, allowing for data recovery. Full formatting completely erases all data on the storage device, making recovery impossible. File system corruption refers to the storage device's file system failing due to operational errors, power outages, or virus attacks, rendering the data inaccessible. In this case, the data still exists on the storage device and can be recovered. Physical damage refers to the storage device becoming inoperable due to mechanical failure, damaged read / write heads, or bad sectors. In this case, appropriate hardware repair operations must be performed.

[0032] The system determines the type of fault in the data recovery request. If the fault is physical damage, an offline data recovery work order and contract are created, and the offline data recovery operation is executed. This offline data recovery work order will be automatically assigned to an engineer. The offline data recovery operation requires the user to mail the corresponding storage device to the offline data recovery center, where the engineer will perform hardware repair operations on the storage device and subsequent data recovery operations. If the fault is not physical damage, an online data recovery work order and contract are created, and the online data recovery operation is executed.

[0033] Online data recovery operations specifically include scanning the storage device to be recovered using a disk copying tool, constructing a corresponding virtual digital entity for the storage device, which includes a mapping of all data burned onto the storage device. It should be noted that if the storage device cannot be recognized by the operating system, the disk copying tool can still read it through an interface, and the user can select the location to be recovered. Then, all data marked as free space in the storage device's virtual digital entity is arranged according to the storage device's address order to form a dataset to be recovered. Data within the storage device is generally stored in binary form; therefore, the dataset to be recovered is also composed of binary data. If the fault is due to quick formatting or file system corruption, all storage space is considered free space, and the dataset to be recovered is divided into data blocks to obtain... Take a block of data to be recovered and determine its character distribution feature vector. The character distribution feature vector refers to the distribution of byte patterns composed of 8 bits of data. Here, byte patterns refer to "00110011" and "00101011", etc. Each data item in the character distribution feature vector is the normalized value of the number of times the corresponding byte pattern appears in the data block to be recovered. It should be noted that different file types have different byte distribution characteristics. For example, printable bytes account for a higher proportion in text types, identifier bytes (such as header tags) account for a higher proportion in image types, and in audio files, bytes corresponding to the frame structure account for a higher proportion in MP3 types. Therefore, the byte distribution characteristics can be used to determine whether the binary data in the free space belongs to the same data type, thereby assisting in subsequent data recovery.

[0034] Based on the character distribution feature vector of the data blocks to be recovered, all data blocks to be recovered are divided into several data sets of files to be recovered. Each data set of files to be recovered stores several data blocks to be recovered belonging to the same file. It should be noted that data belonging to the same file may be stored in different locations due to fragmented storage methods. Identifying and piecing together these data blocks belonging to the same file can facilitate subsequent data recovery operations.

[0035] It should be noted that when data is stored in a storage device, the fragmentation of the data storage occurs because the free space of the storage device is not contiguous before the data is stored. That is, the data content of the same file is distributed in different locations on the storage device. Therefore, when performing data recovery, since the specific data content can no longer be accessed, it is necessary to analyze whether the fragmented data belongs to the same file.

[0036] For each dataset of files to be recovered, the file type is determined. File types include TXT text, CSV, JSON, XML, MP3, MP4, and JPG. The dataset is anchored according to the identifier corresponding to the file type, and then decoded using the appropriate encoding method to output the recovered data file. It should be noted that the output recovered data file is only a preliminary recovery result. This is because the stored data still includes invalid and redundant data, which may affect the anchoring of the identifier. Furthermore, there are multiple encoding methods; UT-8 is commonly used, but an incorrect encoding method can still result in an invalid output file. Therefore, it is necessary to send the data to the engineers for adjustment of the identifier anchoring and selection of the encoding method until a meaningful data file is output.

[0037] The identifier corresponding to the file type is determined by the matching rule set set by the operator. For example, the identifiers corresponding to curly braces {} and square brackets [] in a JSON file, and the identifiers corresponding to "" in an XML file. <tag> ...< / tag> "Corresponding identifiers, etc.;

[0038] This application constructs the storage device from which the user intends to perform data recovery as a virtual digital entity. By performing data recovery operations on the virtual digital entity, operations can be performed without targeting the actual storage device, thus avoiding further damage to the storage device and ensuring the normal progress of data recovery. Furthermore, it divides and reassembles data blocks based on byte distribution characteristics, enabling accurate identification of the entire file even in fragmented storage conditions. Additionally, it performs preliminary data recovery through identifier anchoring and encoding decoding, automatically completing the preliminary preparation work for data recovery and improving the efficiency of the data recovery operation.

[0039] The dataset to be recovered is divided into data blocks to obtain several data blocks to be recovered, and the character distribution feature vector of the data blocks to be recovered is determined. The specific steps include the following:

[0040] The dataset to be recovered is divided into several data blocks to be processed according to a preset window. The preset window is set by the operator and is generally the size of one sector of the disk, i.e., 512 bytes.

[0041] For each data block to be processed, a byte distribution analysis operation is performed. Starting from the first position of the data block, byte pattern statistics are performed to construct all byte sequence data and character distribution feature vectors. Then, starting from the second position of the data block, byte pattern statistics are performed again to construct all byte sequence data and character distribution feature vectors. This process is repeated, with the starting position of each byte pattern statistics shifted one data position forward, until eight shifts are performed. The character distribution feature vector is then output. It should be noted that due to the existence of invalid and redundant data and different encoding methods, a staggered approach is needed to analyze the byte distribution characteristics of the dataset to be recovered from multiple dimensions to improve the accuracy of data partitioning and data reconstruction. All character distribution feature vectors are sequentially fed into the byte distribution analysis model for processing, outputting the file type and corresponding confidence score. The byte distribution analysis model is based on a feedforward neural network and generally includes an input layer, several hidden layers, and an output layer. The output layer outputs the file type and corresponding confidence score. The highest confidence score is selected as the target confidence score, and the file type corresponding to the target confidence score is selected as the target file type. All byte sequence data corresponding to the target confidence score are combined to form the target byte partitioning sequence and output.

[0042] Byte pattern statistics specifically involve dividing the data block to be processed into 8 data bits to obtain all byte sequence data. A byte sequence is the data of the 8 data bits that are divided. The occurrence frequency of byte patterns in all byte sequence data is counted, and the normalized values ​​of all byte pattern occurrences are used to form a character distribution feature vector. The character distribution feature vector has 256 items, and each item corresponds to a byte pattern.

[0043] Iterate through all data blocks to be processed. If two adjacent data blocks correspond to the same target text type, merge them. If two adjacent data blocks correspond to different target text types, determine whether a data block move operation has been performed. A data block move operation moves the split point between two adjacent data blocks forward or backward by 8 data positions. If a data block move operation has been performed, select the currently selected data block and perform a change rate judgment operation. If no data block move operation has been performed, randomly perform a data block move operation and then perform a change rate judgment operation. It should be noted that the data block move operation combined with the change rate judgment operation is to determine whether data blocks belong to the same file and to split them between different file types. When iterating through all data blocks to be processed, first analyze and judge the currently selected data block with its preceding adjacent data blocks, and then analyze and judge the currently selected data block with its following adjacent data blocks.

[0044] The rate of change judgment operation involves determining whether the target confidence level corresponding to the data to be processed is positive. If it is, it indicates that the data block movement operation has achieved a positive effect, resulting in better file type segmentation. Next, it checks whether the target confidence level of the data to be processed is higher than a confidence threshold set by the operator. If the target confidence level is not higher than the confidence threshold, the same data block movement operation as before is performed, and the rate of change judgment operation is repeated after the byte distribution analysis operation. If the target confidence level is higher than the confidence threshold... This indicates that the currently selected data to be processed does not contain data of other file types. The split point between two adjacent data blocks to be processed is fixed, and the process continues to traverse the next two adjacent data blocks to be processed. If the target confidence score corresponding to the data to be processed is not positive, the data block movement operation is reversed compared to the previous one. After performing the byte distribution analysis operation, the change rate judgment operation is performed. It should be noted that since the currently selected data block to be processed needs to be judged with the preceding and following adjacent data blocks to be processed respectively, the split point between each pair of adjacent data blocks to be processed will be adjusted twice before it is truly fixed.

[0045] Once the split points between all adjacent data blocks to be processed have been fixed, the data blocks to be processed are recorded as data blocks to be recovered, and the character distribution feature vector and the corresponding target byte partition sequence of the data blocks to be recovered are output.

[0046] Based on the character distribution feature vector of the data blocks to be recovered, all data blocks to be recovered are divided into several datasets of files to be recovered. The specific steps include the following:

[0047] Cluster analysis is performed based on the character distribution feature vector of the data blocks to be recovered. All the data sets of the files to be recovered in each cluster form a data set of the files to be recovered. The clustering algorithm is K-means, and all the data sets of the files to be recovered are arranged in the address order within the storage device.

[0048] During the process of anchoring the dataset of files to be recovered based on the identifier corresponding to the file type, the bytes in the target byte segmentation sequence are queried and matched with the identifier corresponding to the file type, thereby achieving anchoring.

[0049] Training the byte distribution analysis model involves the following steps:

[0050] Obtain several training samples labeled with file types. These training samples include character distribution feature vectors. It should be noted that these character distribution feature vectors are constructed based on the actual file storage data corresponding to the file type in the training samples, with each byte corresponding to data in the actual file. All training samples labeled with file types are combined into a training set. The byte distribution analysis model is trained using this training set. During training, the labeled file type and its corresponding confidence level are used as the target output. The confidence level for each labeled file type is set to 1 by default. A loss value is constructed based on the difference between the predicted output and the target output of the byte distribution analysis model. The parameters are updated using gradient descent to minimize the loss value, thus training the byte distribution analysis model. The accuracy of the byte distribution analysis model is then checked against an accuracy threshold set by the operator. If the accuracy exceeds the threshold, the trained byte distribution analysis model is output; otherwise, training continues using the training set.

[0051] Example 2: A hardware data repair system for heterogeneous data, such as... Figure 1 As shown, it includes:

[0052] The data recovery request upload module is used to obtain data recovery requests from users. These requests include a description of the storage device from which data recovery is to be performed, such as the customer's company, contact person, contact information, address, serial number, capacity, brand, fault type, and fault description. The fault description is a user-defined description of the storage device from which data recovery is to be performed, such as "Data was lost in the D: / xxx / xxx / xxx folder due to a power outage." Fault types in the data recovery request include accidental deletion, quick formatting, full formatting, file system corruption, and physical damage. It should be noted that this application pertains to a data recovery platform where users can register, log in, and place data recovery orders. Several engineers are registered on the platform. Because the storage devices ordered by users have different hardware architectures, storage media, data formats, technical standards, operating systems, and other characteristics, the data recovery platform is designed for heterogeneous data.

[0053] The data recovery operation selection module is used to determine the fault type in the data recovery request. If the fault type in the data recovery request is physical damage, an offline data recovery work order and contract will be created, and the offline data recovery operation will be executed. This offline data recovery work order will be automatically assigned to an engineer. The offline data recovery operation requires the user to mail the corresponding storage device to the offline data recovery center, where the engineer will perform the hardware repair operation on the storage device and subsequent data recovery operations. If the fault type in the data recovery request is not physical damage, an online data recovery work order and contract will be created, and the online data recovery operation will be executed.

[0054] The online data recovery module is used to perform online data recovery operations. Specifically, the online data recovery operation includes scanning the storage device to be recovered using a disk copying tool, constructing a corresponding virtual digital entity for the storage device, which contains a mapping of all data burned onto the storage device. It should be noted that if the storage device cannot be recognized by the operating system, the disk copying tool can still read it through an interface. The user can also select the location to be recovered. Then, all data marked as free space in the storage device's virtual digital entity is arranged according to the storage device's address order to form the dataset to be recovered. Data is typically stored in binary format, therefore the dataset to be recovered is also composed of binary data. If the failure is due to quick formatting or file system corruption, all storage space is considered free space. The dataset to be recovered is divided into several data blocks, and the character distribution feature vector of each data block is determined. This character distribution feature vector refers to the distribution of byte patterns composed of 8 bits of data, such as "00110011" and "00101011". Each data item in the character distribution feature vector is the normalized value of the frequency of the corresponding byte pattern in the data block to be recovered. It should be noted that different file types have different byte distribution characteristics. For example, printable bytes account for a higher proportion in text types, identifier bytes (such as header tags) account for a higher proportion in image types, and in audio files, the bytes corresponding to the frame structure account for a higher proportion in MP3 types. Therefore, the byte distribution characteristics can be used to determine whether the binary data in the free space belongs to the same data type, thus assisting in subsequent data recovery. Based on the character distribution feature vector of the data blocks to be recovered, all data blocks to be recovered are divided into several datasets of files to be recovered. Each dataset of files to be recovered stores several data blocks to be recovered belonging to the same file. It should be noted that data belonging to the same file may be stored in different locations due to fragmented storage methods. Identifying these data blocks belonging to the same file again and concatenating them together can facilitate subsequent data recovery operations. For each dataset of files to be recovered, the file type corresponding to the dataset is determined. File types include TXT text, CSV files, JSON files, XML files, MP3 files, MP4 files, and JPG files, etc. The dataset of files to be recovered is anchored according to the identifier corresponding to the file type, and then decoded through encoding to output the data recovery file.It should be noted that the output data recovery file is only a preliminary recovery result. This is because the stored data also includes invalid and redundant data, which may affect the anchoring of identifiers. Furthermore, there are various encoding methods; UT-8 is commonly used, but an incorrect encoding method can still result in an invalid output file. Therefore, it needs to be sent to the engineer's side for adjustment of identifier anchoring and selection of the encoding method until a meaningful data file is output.

[0055] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims. Parts not described in detail in this specification are prior art known to those skilled in the art.

Claims

1. A hardware data repair method for heterogeneous data, characterized in that, include: Get the data recovery request from the user; The system determines the type of fault in the data recovery request. If the fault type is physical damage, an offline data recovery work order and contract are created, and the offline data recovery operation is executed. If the fault type is not physical damage, an online data recovery work order and contract are created, and the online data recovery operation is executed. Physical damage refers to the inability of a storage device to operate due to mechanical failure, damaged read / write heads, or bad sectors. In such cases, corresponding hardware repair operations must be performed. The online data recovery operation specifically includes scanning the storage device to be recovered using a disk copying tool, constructing a corresponding virtual digital body for the storage device, which includes a mapping of all data recorded on the storage device, arranging all data marked as free space in the virtual digital body according to the address order of the storage device to form a dataset to be recovered, dividing the dataset into data blocks to obtain several data blocks to be recovered, and determining the character distribution feature vector of the data blocks to be recovered. Each data item in the character distribution feature vector is the normalized value of the number of times the corresponding byte pattern appears in the data block to be recovered. Based on the character distribution feature vector of the data blocks to be recovered, all data blocks to be recovered are divided into several data sets of files to be recovered. Each data set of files to be recovered stores several data blocks of files to be recovered belonging to the same file. For each dataset of files to be recovered, determine the file type corresponding to the dataset, anchor the dataset to be recovered based on the identifier corresponding to the file type, and then decode the dataset to be recovered through encoding to output the data recovery file; The dataset to be recovered is divided into data blocks, resulting in several data blocks to be recovered. The specific steps include the following: The dataset to be recovered is divided into several data blocks to be processed according to a preset window; For each data block to be processed, a byte distribution analysis operation is performed. Starting from the first position of the data block, byte pattern statistics are performed to construct all byte sequence data and character distribution feature vectors. Then, starting from the second position of the data block, byte pattern statistics are performed again to construct all byte sequence data and character distribution feature vectors. This process is repeated, with the starting position of each byte pattern statistics shifted one data position forward until it has been shifted 8 times. The character distribution feature vectors are then output. All character distribution feature vectors are sequentially fed into the byte distribution analysis model for processing. The file type and its corresponding confidence score are output. The highest confidence score is selected as the target confidence score, and the file type corresponding to the target confidence score is selected as the target file type. All byte sequence data corresponding to the target confidence score are combined to form the target byte partition sequence and output. Byte pattern statistics specifically involve dividing the data block to be processed into 8 data bits to obtain all byte sequence data. A byte sequence is the data of the 8 data bits that are divided. The occurrence count of byte patterns in all byte sequence data is counted, and the normalized values ​​of all byte pattern occurrence counts are used to form a character distribution feature vector. Each item in the character distribution feature vector corresponds to a byte pattern.

2. The hardware data repair method for heterogeneous data according to claim 1, characterized in that, Determining the character distribution feature vector of the data block to be recovered includes the following steps: Iterate through all data blocks to be processed. If the target text types of two adjacent data blocks are the same, merge the two adjacent data blocks. If the target text types of two adjacent data blocks are different, determine whether a data block move operation has been performed. The data block move operation is to move the dividing point between two adjacent data blocks forward or backward by 8 data positions. If a data block move operation has been performed, select the currently selected data block to be processed and perform a change rate judgment operation. If no data block move operation has been performed, randomly perform a data block move operation and then perform a change rate judgment operation. The rate of change judgment operation is as follows: First, determine whether the target confidence level corresponding to the data to be processed is positive. If the target confidence level is positive, determine whether it is higher than the confidence threshold. If it is not higher than the confidence threshold, continue with the same data block movement operation as before, and continue with the rate of change judgment operation after performing the byte distribution analysis operation. If the target confidence level is higher than the confidence threshold, fix the split point between two adjacent data blocks and continue traversing the next two adjacent data blocks. If the target confidence level is not positive, perform the opposite data block movement operation as before, and continue with the rate of change judgment operation after performing the byte distribution analysis operation. Once the split points between all adjacent data blocks to be processed have been fixed, the data blocks to be processed are recorded as data blocks to be recovered, and the character distribution feature vector and the corresponding target byte partition sequence of the data blocks to be recovered are output.

3. The hardware data repair method for heterogeneous data according to claim 2, characterized in that, Based on the character distribution feature vector of the data blocks to be recovered, all data blocks to be recovered are divided into several data sets to be recovered. Specifically, the following steps are included: performing cluster analysis based on the character distribution feature vector of the data blocks to be recovered, and forming a data set to be recovered from all data blocks in each cluster. Furthermore, all data sets to be recovered from the data blocks are arranged in the order of their addresses within the storage device.

4. The hardware data repair method for heterogeneous data according to claim 3, characterized in that, During the process of anchoring the dataset of files to be recovered based on the identifier corresponding to the file type, the bytes in the target byte segmentation sequence are queried and matched with the identifier corresponding to the file type, thereby achieving anchoring.

5. A hardware data repair method for heterogeneous data according to claim 4, characterized in that, Training the byte distribution analysis model involves the following steps: Obtain several training samples labeled with file types. The training samples include character distribution feature vectors. Combine all the training samples labeled with file types into a training set. Train the byte distribution analysis model using the training set. During training, use the labeled file types and their corresponding confidence scores as the target output. The confidence score for the labeled file types is set to 1 by default. If the accuracy of the byte distribution analysis model is higher than the accuracy threshold, output the trained byte distribution analysis model. Otherwise, continue training the byte distribution analysis model using the training set.

6. A hardware data repair system for heterogeneous data, characterized in that, The system employs a hardware data repair method for heterogeneous data as described in any one of claims 1-5, comprising: The data recovery request upload module is used to obtain data recovery requests from users; The data recovery operation selection module is used to determine the fault type in the data recovery request. If the fault type in the data recovery request is physical damage, an offline data recovery work order and contract are created, and the offline data recovery operation is executed. If the fault type in the data recovery request is not physical damage, an online data recovery work order and contract are created, and the online data recovery operation is executed. The online data recovery module is used to perform online data recovery operations. Specifically, the online data recovery operation includes: scanning the storage device to be recovered using a disk copying tool; constructing a corresponding virtual digital entity for the storage device, which includes a mapping of all data burned into the storage device; arranging all data marked as free space in the virtual digital entity according to the storage device's address order to form a dataset to be recovered; dividing the dataset into several data blocks to be recovered, and determining the character distribution feature vector of each data block, where each item is the normalized value of the frequency of the corresponding byte pattern in the data block; dividing all data blocks to be recovered based on the character distribution feature vector to form several file datasets to be recovered, each containing several data blocks belonging to the same file; for each file dataset to be recovered, determining the file type, anchoring the dataset according to the identifier corresponding to the file type, and then decoding the dataset using an encoding method to output the recovered data file.

Images