Data storage apparatus and method for anonymous perceptual deduplication

Abstract

A data storage device includes a memory and a controller, wherein the controller receives an indication of data to be anonymized. The controller also parses a data element to be stored and generates one or more copies of data portions to be anonymized. The controller also deletes the one or more data portions to be anonymized to generate a modified data element to be stored. The controller also generates a copy of the modified data element to be stored with deduplication. Thus, the data storage device accounts for data anonymization during deduplication (i.e., anonymization-aware deduplication) so that efficient deduplication can be achieved even in the presence of anonymized data.

Description

Technical Field

[0001] This invention generally relates to the field of data protection and backup, and more specifically, to a data storage device and method for anonymously aware deduplication. Background Technology

[0002] Typically, data backups are used to protect and recover data in the event of a data loss incident in the primary storage system. Examples of data loss incidents may include, but are not limited to, data corruption, hardware or software failure in the primary storage device, accidental data deletion, and hacking or malicious attacks. Therefore, for security reasons, separate backup systems or secondary storage devices (e.g., data storage devices) are widely used to store backups of data present in the primary storage system.

[0003] Currently, data anonymization is used to protect the privacy of confidential or private information, such as credit card numbers and social security numbers. In data anonymization, confidential or personal data, such as personally identifiable information, is often obfuscated to keep the person associated with the data anonymous. However, it can be observed that data anonymization has a negative impact on the effectiveness of deduplication because if the original data is used as a reference, the anonymized data will not undergo deduplication. The term "deduplication" generally refers to the elimination of duplicate or redundant information. As data anonymization changes the data, it is no longer the same as the original data, thus increasing the workload of backup and deduplication. Furthermore, existing deduplication methods (such as variable-length deduplication) are effective when large identical data chunks exist in the data stream to be backed up. However, if the data is changed frequently or the changes are very small, these existing deduplication methods are ineffective. For example, even changing the characters in a chunk may present the data chunk as a new chunk, and traditional deduplication methods will not find any identical data chunks, thus reducing the effectiveness of deduplication. Therefore, over time, as more duplicate data is stored, it consumes a significant amount of storage space in traditional secondary storage, thus depleting the secondary storage's capacity. This degrades the performance of the secondary storage, a situation that is undesirable. Furthermore, the cost of file storage, along with all associated costs including storage hardware, remains a significant challenge.

[0004] Therefore, in light of the above discussion, it is necessary to overcome the shortcomings associated with traditional data storage systems and deduplication methods. Summary of the Invention

[0005] This invention aims to provide a data storage device and method for anonymity-aware deduplication. It seeks to provide a technical solution to address the existing problems of inefficient deduplication related to data anonymization. The object of this invention is to provide a technical solution that at least partially overcomes the problems encountered in the prior art, and to provide an improved data storage device and method that takes into account data anonymization during deduplication (i.e., anonymity-aware deduplication), thereby achieving efficient deduplication even in the presence of anonymized data.

[0006] The objective of this invention is achieved through the technical solutions provided in the appended independent claims. Advantageous embodiments of the invention are further defined in the dependent claims.

[0007] On one hand, the present invention provides a data storage device, including a memory and a controller, wherein the memory is used to store one or more data elements, and the controller is used to store at least one of the one or more data elements using deduplication. The data storage device is characterized in that the controller is further configured to: receive an instruction for data to be anonymized; receive a data element to be stored; parse the data element to be stored to locate one or more data portions to be anonymized; generate a copy of the one or more data portions to be anonymized; delete the one or more data portions to be anonymized from the data element to be stored, generating a modified data element to be stored; generate a copy of the modified data element to be stored using deduplication; and store the copy of the modified data element to be stored and the copy of the one or more data portions to be anonymized.

[0008] The data storage device of the present invention considers data anonymization during deduplication (i.e., anonymity-aware deduplication), thus achieving efficient deduplication even in the presence of anonymized data. In the data storage device, the one or more portions of data to be anonymized are separately parsed and stored by the controller to achieve effective deduplication of the data elements received for storage. Furthermore, the present invention efficiently utilizes storage space by not storing duplicate data, as in anonymity-aware deduplication, thereby improving system performance compared to conventional storage, where a large amount of storage space is occupied by duplicate data. This is because conventional systems do not consider the presence of anonymized data, further impacting the system performance of conventional memory.

[0009] In one implementation, the controller is further configured to store a copy of the modified data element to be stored in a first storage area; and to store the one or more portions of data to be anonymized in a second storage area.

[0010] By storing copies of the modified data elements and the one or more portions of data to be anonymized separately, the data storage device is able to perform deduplication on the modified data elements.

[0011] In another implementation, the controller is further configured to store the one or more portions of data to be anonymized, wherein the one or more portions of data to be anonymized are appended to a copy of the modified data element to be stored.

[0012] By attaching one or more data portions to be anonymized, the data storage device is able to efficiently store and retrieve the data elements when needed.

[0013] In another implementation, the controller is further configured to generate a corresponding pointer to each of the one or more data portions to be anonymized, and to store copies of the one or more data portions to be anonymized and the corresponding pointers.

[0014] Pointers to each of the one or more data portions to be anonymized are generated and stored, enabling easy retrieval of the data elements. Furthermore, the use of pointers reduces storage space requirements compared to traditional systems.

[0015] In another implementation, the corresponding pointer indicates the position of the corresponding unanonymized data portion within the data element.

[0016] The pointer corresponds to the address of the source (i.e., within the data element) of the one or more portions of data to be anonymized. Therefore, the data element can be easily retrieved when needed.

[0017] In another implementation, the controller is also configured to parse the data element to be stored in order to locate one or more data portions to be anonymized by searching for data portions in a data field designated as an anonymized field.

[0018] Parsing the data field to be anonymized can effectively and quickly extract anonymous or unanonymized data from the received data elements. Therefore, duplicate data deletion can be automatically implemented on the data elements.

[0019] In another implementation, the controller is also configured to parse the data element to be stored in order to locate one or more data portions to be anonymized by searching for data portions having a data format that matches the anonymization format.

[0020] Parsing the data to be anonymized according to its format can accurately identify and effectively parse anonymous or unanonymized data from the received data elements. Therefore, duplicate data deletion can be effectively achieved on the data elements.

[0021] In another implementation, the controller is further configured to recover a data element by: receiving a copy of the modified data element to be recovered and copies of one or more anonymous data portions; extracting the modified data element by utilizing deduplication extraction to generate a recovered copy of the data element; and inserting the one or more anonymous data portions into the recovered copy of the data element.

[0022] The controller is used to generate a restored copy of the data element by combining the modified data element with one or more anonymous data portions. Therefore, the data element can be retrieved (or restored) efficiently and accurately when needed.

[0023] On the other hand, the present invention provides a method for a data storage device, wherein the data storage device includes a memory for storing one or more data elements, the method including storing at least one of the one or more data elements using deduplication, the method being characterized in that the method further includes: receiving an instruction for data to be anonymized; receiving a data element to be stored; parsing the data element to be stored to locate one or more data portions to be anonymized; generating a copy of the one or more data portions to be anonymized; deleting the one or more data portions to be anonymized from the data element to be stored, generating a modified data element to be stored; generating a copy of the modified data element to be stored using deduplication; and storing the copy of the modified data element to be stored and the copy of the one or more data portions to be anonymized.

[0024] The method of this invention considers data anonymization during deduplication (i.e., anonymity-aware deduplication), achieving efficient deduplication even in the presence of anonymized data. In this method, one or more portions of data to be anonymized are parsed and stored separately to achieve effective deduplication of the data elements received for storage. Furthermore, the method of this invention efficiently utilizes storage space by not storing duplicate data, as is the case in anonymity-aware deduplication, thereby improving performance compared to conventional methods where a large amount of storage space is occupied by duplicate data because conventional systems do not consider the presence of anonymized data.

[0025] In one implementation, the method further includes recovering a data element by: receiving a copy of the modified data element to be recovered and copies of one or more anonymous data portions; extracting the modified data element by utilizing deduplication extraction to generate a recovered copy of the data element; and inserting the one or more anonymous data portions into the recovered copy of the data element.

[0026] The method generates a restored copy of a data element by combining the modified data element with one or more anonymous data portions. Therefore, the method can efficiently retrieve the data element when needed.

[0027] In yet another aspect, the present invention provides a computer-readable medium carrying computer instructions that, when loaded into and executed by a controller of a data storage device, enable the data storage device to implement the methods described above.

[0028] The computer-readable medium of the present invention is capable of performing deduplication on data with anonymous data or data to be anonymized, which is impossible or ineffective for data with anonymous data compared to conventional storage.

[0029] On the other hand, the present invention provides a data storage device, including a memory for storing one or more data elements. The data storage device further includes a deduplication storage software module for storing at least one of the one or more data elements using deduplication. The data storage device is characterized in that it further includes: a software module for receiving an instruction for anonymized data; a software module for receiving a data element to be stored; a software module for parsing the data element to be stored to locate one or more data portions to be anonymized; a software module for generating copies of the one or more data portions to be anonymized; a software module for deleting the one or more data portions to be anonymized from the data element to be stored, generating a modified data element to be stored; a software module for generating a copy of the modified data element to be stored using deduplication; and a software module for storing the copy of the modified data element to be stored and the copies of the one or more data portions to be anonymized.

[0030] By using the software module of the present invention, the data storage device takes into account data anonymization during deduplication (i.e., anonymity-aware deduplication), achieving efficient deduplication even in the presence of anonymized data. In the data storage device, the one or more portions of data to be anonymized are separately parsed and stored by the software module to achieve effective deduplication of the data elements received for storage. Furthermore, the present invention efficiently utilizes storage space, as in anonymity-aware deduplication, by not storing duplicate data, thereby improving system performance compared to conventional storage.

[0031] In one implementation, the data storage device further includes a data element recovery software module for recovering data elements. The data element recovery software module includes: a data element receiving software module for receiving a copy of the modified data element to be recovered and copies of one or more anonymous data portions; a deduplication extraction software module for extracting the modified data element by utilizing deduplication extraction to generate a recovered copy of the data element; and an insertion software module for inserting the one or more anonymous data portions into the recovered copy of the data element.

[0032] The recovery software module generates a recovered copy of the data element by combining the modified data element with one or more anonymous data portions. Therefore, the data element can be retrieved efficiently when needed.

[0033] It should be understood that all of the above implementation methods can be combined together.

[0034] It should be noted that all devices, elements, circuits, units, and modules described in this application can be implemented in software elements or hardware elements, or any combination thereof. The steps performed by the various entities described in this application, and the functions to be performed by the various entities, are intended to refer to the various entities performing the respective steps and functions. Even in the description of the following specific embodiments, if a particular function or step to be performed by an external entity is not reflected in the description of the specific detailed elements of the entity performing that particular step or function, it will be clear to those skilled in the art that these methods and functions can be implemented in the corresponding software or hardware elements, or in any combination of such elements. It is understood that the features of the present invention are readily combined in various combinations without departing from the scope of the invention as defined by the appended claims.

[0035] Other aspects, advantages, features, and objects of the invention will be clearly described by the accompanying drawings and the detailed description of illustrative embodiments interpreted in conjunction with the appended claims. Attached Figure Description

[0036] The foregoing summary of the invention and the following detailed description of illustrative embodiments can be better understood when read in conjunction with the accompanying drawings. Exemplary structures of the invention are shown in the drawings to illustrate the invention. However, the invention is not limited to the specific methods and means disclosed herein. Furthermore, those skilled in the art will understand that these drawings are not drawn to scale. Where possible, the same elements are represented by the same numbers.

[0037] The embodiments of the present invention will now be described by way of example only, in conjunction with the accompanying drawings.

[0038] Figure 1AThis is a block diagram of a data storage device for anonymously aware deduplication provided in an embodiment of the present invention.

[0039] Figure 1B A block diagram of a data storage device for anonymously aware deduplication provided in another embodiment of the present invention.

[0040] Figure 2 A flowchart of a method for anonymized deduplication of data storage devices provided in an embodiment of the present invention.

[0041] In the accompanying diagrams, underlined numbers indicate the item to which the underlined number is located or the item adjacent to the underlined number. Ununderlined numbers are associated with the item identified by the line linking the ununderlined number to that item. When a number is ununderlined but has an associated arrow, the ununderlined number identifies the general item pointed to by the arrow. Detailed Implementation

[0042] The following detailed description illustrates embodiments of the present invention and their implementation. While some embodiments of the present invention have been disclosed, those skilled in the art will recognize that other embodiments for carrying out or practicing the present invention can also be implemented.

[0043] Figure 1A A block diagram of a data storage device provided in an embodiment of the present invention. (See reference...) Figure 1A The image shows a data storage device 100A. The data storage device 100A includes a memory 102 and a controller 104. In one implementation, the memory 102 includes a first storage area 108 and a second storage area 110, and the data storage device 100A also includes a network interface 112.

[0044] On one hand, the present invention provides a data storage device 100A, including a memory 102 and a controller 104. The memory 102 is used to store one or more data elements 106, and the controller 104 is used to store at least one of the one or more data elements 106 using deduplication. The data storage device 100A includes the controller 104, which is further configured to: receive an instruction for data to be anonymized; receive a data element to be stored; parse the data element to be stored to locate one or more data portions to be anonymized; generate a copy of the one or more data portions to be anonymized; delete the one or more data portions to be anonymized from the data element to be stored, generating a modified data element to be stored; generate a copy of the modified data element to be stored using deduplication; and store the copy of the modified data element to be stored and the copy of the one or more data portions to be anonymized.

[0045] Memory 102 refers to the hardware memory of data storage device 100A. Memory 102 includes suitable logic, circuitry, or interfaces for storing one or more data elements 106, pointers, and other data according to instructions received from controller 104. Furthermore, memory 102 can be used to further store instructions executable by controller 104. Examples of implementations of memory 102 may include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, solid-state drive (SSD), and / or CPU cache. Memory 102 may store an operating system or other program products (including one or more operating algorithms) to operate the data storage device 100A. The first storage area 108 and the second storage area 110 refer to blocks or files or other related storage areas in memory 102, respectively.

[0046] One or more data elements 106 refer to incoming information or data streams arriving at data storage device 100A. For example, one or more data elements 106 may arrive as an input / output (I / O) request during deduplication (i.e., when performing a backup from a primary storage system (e.g., a host server) to an auxiliary storage system such as data storage device 100A). In one example, one or more data elements 106 may include personal data, such as personally identifiable information or data that may be subject to data privacy and security constraints under various regulations or data protection laws (e.g., the Data Protection Act (DPA)). One or more data elements 106 may be structured data, such as data with defined fields such as name, age, phone number, credit card number, address, etc. For example, one or more data elements 106 may include the string "Dani,33,0541111111,4580800080001999,A street", representing the sequential fields of name, age, phone number, credit card number, and address. In another example, one or more data elements 106 can be unstructured data, where data that needs to be anonymous can appear anywhere in the dataset without a specific field or a known location.

[0047] Network interface 112 includes appropriate logic, circuitry, and / or interfaces that can be used to communicate with one or more external devices, such as user equipment or servers. Examples of network interface 112 may include, but are not limited to, network interface cards (NICs), antennas, radio frequency (RF) transceivers, or network ports.

[0048] Controller 104 is used to store at least one of one or more data elements 106 using deduplication. In one implementation, controller 104 is used to execute instructions stored in memory 102. In one example, processor 104 may be a general-purpose processor. Examples of controller 104 may include, but are not limited to, microprocessors, microcontrollers, complex instruction set computing (CISC) processors, application-specific integrated circuit (ASIC) processors, reduced instruction set computing (RISC) processors, very long instruction word (VLIW) processors, central processing units (CPUs), state machines, data processing units, and other processors or control circuits. Furthermore, controller 104 may refer to one or more separate processors, processing devices, or processing units as part of a machine, such as data storage device 100A. Controller 104 is used to store at least one of one or more data elements 106 using deduplication. Deduplication enables data storage device 100A to prevent duplicate data from being stored in auxiliary memory such as memory 102. Therefore, a significant amount of storage space is saved, further improving the system performance of the data storage device 100A. In one example, the data storage device 100A refers to auxiliary storage. At least one data element of one or more data elements 106 stored using deduplication can be confidential data, such as credit card numbers, social security numbers, or any other personal information.

[0049] In operation, controller 104 is used to receive instructions on data to be anonymized. Data anonymization refers to the information sanitization process performed by controller 104 to achieve privacy protection. It is the process of encrypting, obfuscating, or deleting personally identifiable information from one or more data elements 106 so that the person associated with the data remains anonymous. The instructions on data to be anonymized can be provided by the user. In one example, controller 104 can be used to receive user input indicating an instruction to select settings or configurations for identifying sensitive data. For example, the user can configure anonymization that may be applied to the data as needed, such as General Data Protection Regulation (GDPR) anonymization, anonymization of only parts such as credit card numbers, etc. These selected or configured settings and configurations can be pre-stored in memory 102.

[0050] The controller 104 is also used to receive data elements to be stored. The data elements to be stored can be received by the controller 104 from an external device communicatively coupled to the data storage device 100A via a communication network. The data storage device 100A can be an auxiliary storage device for storing backups of data from user devices or a primary storage system. In one example, the data storage device 100A is used to store data from multiple user devices or host servers within an organization. In one example, data elements can be received during backup to perform deduplication. Each data element corresponds to one or more data elements 106.

[0051] In one example, the data elements to be stored received by controller 104 are given in the table (1) below (in practice, such a table can have millions of rows).

[0052] Table 1

[0053] Name age telephone number Credit Card Number address Dani 33 0541111111 4580800080001999 Street A Josef 55 0541222222 4580123434001999 Street B

[0054] The data elements are as follows: name (64 characters), age (3 characters), phone number (10 characters), credit card number (16 characters), and address (64 characters). In another example, controller 104 receives alphanumeric strings as the data elements to be stored: Dani, 33, 0541111111, 4580800080001999, Street A; Josef, 55, 0541222222, 4580123434001999, Street B.

[0055] Controller 104 is also used to parse the data element to be stored to locate one or more data portions to be anonymized. Controller 104 parses the data element to be stored according to received instructions for data anonymization (i.e., user configuration). In one example, the algorithm used by controller 104 to detect which data is sensitive (i.e., confidential) data may operate according to the configuration received by controller 104 from the user. For example, in the case of structured data, the location of a given data field is predefined or known, and it will be parsed accordingly. For example, in the data element received by controller 104, the credit card number data field is identified at a starting offset '67' (16 characters) and a starting offset '224' (16 characters) from the beginning of the alphanumeric string. In another example, one or more data portions to be anonymized (e.g., sensitive data) can be identified according to user configuration and rule usage patterns. Advantageously, one or more data portions to be anonymized are parsed by controller 104 to enable efficient deduplication of the data elements received by controller 104 for storage. In conventional techniques, data portions to be anonymized are not parsed from data elements, resulting in inefficient deduplication.

[0056] According to one embodiment, the controller 104 is further configured to parse the data element to be stored to locate one or more portions of data to be anonymized by searching for data portions in a data field designated as a field to be anonymized. Here, a data field refers to a broad category of data that can be stored in a structured form. In one example, the data field could be a name, age, phone number, credit card number, or residential address, where the credit card number can be identified as the field to be anonymized. In such an example, the data element of the data field (credit card number) is parsed and stored for anonymization.

[0057] According to one embodiment, controller 104 is further configured to parse the data element to be stored to locate one or more portions of data to be anonymized by searching for data portions having a data format that matches the anonymization format. Here, data format refers to the defined format of the data element, such as the number of characters or letters or numbers used, or certain patterns. Controller 104 is configured to receive anonymization formats (i.e., user-specified formats) from the user. In one example, data elements are parsed by searching for data portions having a data format that matches the user-specified format; if the received data is structured, unstructured, or has some unique patterns, this is done by understanding the format of the data element. In one example, data formats such as credit card numbers may have a unique format of only a specified length (e.g., 16 digits) of numbers, or an age may have a format of 1 or 2 digits. Credit card numbers have very specific formats and built-in checksums that facilitate finding the data format of credit card numbers. Similar to credit card numbers, unique patterns can be identified for the data to be anonymized to isolate such data portions.

[0058] Controller 104 is also used to generate one or more copies of the data portion to be anonymized. Generating one or more copies of the data portion to be anonymized enables controller 104 to separate the data portion to be anonymized from the rest of the data element received by controller 104. In one example, the one or more copies of the data portion to be anonymized are generated based on the start and end offsets of the data portion. In one example, in the data element received by controller 104, the credit card number is identified at the start offset '67' and the end offset '224', therefore a copy of the data portion (16 bits / character in length) between offset '67' and offset '224' is generated.

[0059] According to one embodiment, one or more data portions can be anonymized before parsing. Therefore, one or more copies of the already anonymized data portions are generated. In one example, controller 104 may include algorithms for performing data portion anonymization. These algorithms identify the data portions to be anonymized based on indications received of the data to be anonymized.

[0060] In one example, the data elements received by controller 104 (before anonymization) can be represented as: Dani, 33, 0541111111, 4580800080001999, Street A; Josef, 55, 0541222222, 4580123434001999, Street B. Furthermore, when executing the algorithm for anonymization, the data elements (after anonymization) can be represented as: Dani, 33, 0541111111, 1111222233334444, Street A; Josef, 55, 0541222222, 1111222233334444, Street B. In such examples, the 16-bit data format is identified as the data format used for anonymization.

[0061] In another example, the data elements received by controller 104 (before anonymization) can be represented by Table 2, given below. Furthermore, when the algorithm for anonymization is executed, the data elements (after anonymization) can be represented by Table 3, given below (here only credit card numbers are anonymized). The table width is only 157 characters (i.e., 64 characters for name, 3 characters for age, 10 characters for phone number, 16 characters for credit card number, and 64 characters for address). In conventional techniques, variable-size deduplication would fail to identify identical blocks, resulting in zero deduplication between Table 1 and Table 2. Furthermore, storage space utilization could be low due to the possibility of millions of rows in the tables. However, in data storage device 100A, data elements received from controller 104 are used to parse one or more data portions to be anonymized or one or more data portions that have already been anonymized. As a result, deduplication is efficiently performed by data storage device 100A, thereby efficiently utilizing storage space.

[0062] Table 2

[0063] Name age telephone number Credit Card Number address Dani 33 0541111111 4580800080001999 Street A Josef 55 0541222222 4580123434001999 Street B

[0064] Table 3

[0065] Name age telephone number Credit Card Number address Dani 33 0541111111 1111222233334444 Street A Josef 55 0541222222 1111222233334444 Street B

[0066] The controller 104 is also configured to delete one or more portions of data to be anonymized from the data element to be stored, generating a modified data element to be stored. The data portion identified as the data to be anonymized is separated from the rest of the data, and a pointer to the original location of the data is retained. Compared to conventional techniques, deduplication can now be performed on modified data elements to identify duplicate data elements, where duplicate data elements are identified as new data elements because anonymous data exists in the data elements in conventional techniques.

[0067] The controller 104 is also configured to generate copies of the modified data elements to be stored for deduplication. By storing copies of the modified data elements to be stored separately from copies of one or more data portions to be anonymized, duplicate data elements can be identified.

[0068] In an exemplary implementation, the data received by controller 104 is represented as: Dani (64 bytes), 33 (3 bytes), 0541111111 (10 bytes), 4580800080001999 (16 bytes), Street A (64 bytes), Josef (64 bytes), 55 (3 bytes), 0541222222 (10 bytes), 4580123434001999 (16 bytes), Street B (64 bytes). In this example, the 16-digit credit card number is identified at offsets '67' and '224', parsed and separated. Therefore, the modified data elements are represented as: Dani (64 bytes), 33 (3 bytes), 0541111111 (10 bytes), Street A (64 bytes), Josef (64 bytes), 55 (3 bytes), 0541222222 (10 bytes), Street B (64 bytes), and the data to be anonymized is represented as: 4580800080001999 (offset '67'), 4580123434001999 (offset '224').

[0069] In another exemplary implementation, the data received by controller 104, when containing anonymous data, is represented as: Dani (64 bytes), 33 (3 bytes), 0541111111 (10 bytes), 1111222233334444 (16 bytes), Street A (64 bytes), Josef (64 bytes), 55 (3 bytes), 0541222222 (10 bytes), 1111222233334444 (16 bytes), Street B (64 bytes). In such an example, the 16-digit credit card number is identified at offsets '67' and '224', parsed, and separated. Therefore, the modified data elements are represented as: Dani (64 bytes), 33 (3 bytes), 0541111111 (10 bytes), Street A (64 bytes), Josef (64 bytes), 55 (3 bytes), 0541222222 (10 bytes), Street B (64 bytes), and the anonymized data portion is represented as: 1111222233334444 (offset '67'), 1111222233334444 (offset '224').

[0070] According to one embodiment, the modified data element also undergoes variable-length deduplication. The modified data element is divided into different variable-size blocks (i.e., variable-size data blocks) according to a block partitioning algorithm. Furthermore, the controller 104 is used to hash each block using a hash algorithm. Therefore, each block has a hash signature generated using a hashing algorithm. In one example, a content-based hashing algorithm, such as Rabin hashing, can be used to positionally isolate data elements based on their content. Furthermore, if the hash signature of a block is the same as the hash signature of a block previously stored in data storage device 100A, the modified data element is identified as a duplicate data element, and only a pointer to the block is stored (not the same block), significantly reducing storage space compared to conventional systems and methods. If the hash signature of a given block is different from the hash signature of a block previously stored in memory 102, the block is stored in memory 102, and the hash of the new block is added to the previous hash signature.

[0071] The controller 104 is also used to store copies of the modified data elements to be stored and copies of one or more data portions to be anonymized. Suspected data portions to be anonymized, or identified and anonymized data portions, are stored separately. Compared to conventional memory, which does not distinguish between anonymized and anonymized data portions and directly stores these data elements, thus storing duplicate data elements, the data storage device 100A of the present invention stores anonymized or anonymized data portions separately from other data portions of the received data elements, thus preventing the storage of duplicate data elements.

[0072] According to one embodiment, the controller 104 is further configured to store a copy of the modified data element to be stored in a first storage area 108; and to store one or more data portions to be anonymized in a second storage area 110. Storing the copy of the modified data element and the one or more data portions to be anonymized separately enables deduplication to be performed on the modified data element. Compared to conventional techniques, where received data elements are directly stored without parsing and deleting one or more data portions, resulting in no deduplication or invalid deduplication.

[0073] According to one embodiment, the controller 104 is further configured to store one or more portions of data to be anonymized, wherein the one or more portions of data to be anonymized are attached to a copy of the modified data element to be stored. Attaching one or more portions of data to be anonymized to the copy of the modified data element allows for efficient storage and retrieval of the data element when needed. For example, associations are defined between the one or more portions of data to be anonymized and the copy of the modified data element to ensure accurate retrieval upon user request.

[0074] In some implementations, controller 104 is further configured to generate a corresponding pointer to each of the one or more data portions to be anonymized, and store copies of the one or more data portions to be anonymized and the corresponding pointers. Generating and storing pointers to each of the one or more data portions to be anonymized enables the retrieval of the original data (i.e., the received data element) during the retrieval of the data element.

[0075] According to one embodiment, the corresponding pointer indicates the position of the corresponding data portion to be anonymized within the data element. The pointer corresponds to the address of the source (i.e., within the data element) of the one or more data portions to be anonymized. Therefore, these anonymized data portions can be inserted back into their original location (or position) within the data element, thus allowing for easy retrieval (recovery) of the data element when needed.

[0076] According to one embodiment, controller 104 is further configured to recover a data element by: receiving a copy of the modified data element to be recovered and copies of one or more anonymous data portions; extracting the modified data element using deduplication extraction to generate a recovered copy of the data element; and inserting the one or more anonymous data portions into the recovered copy of the data element. In one example, controller 104 recovers a data element upon receiving a recovery request from a user (e.g., via a user device or host server). Based on the recovery request, controller 104 retrieves a copy of the modified data element to be recovered and copies of one or more anonymous data portions from memory 102. Controller 104 is configured to generate a recovered copy of the data element by combining the modified data element with the one or more anonymous data portions. In one example, if the previously stored modified data element is a duplicate and only the address of the original data element is stored, the original data element is received by controller 104, and one or more anonymous data portions are inserted into the appropriate position within the data element.

[0077] Advantageously, the data storage device 100A of the present invention takes into account data anonymity during deduplication, and can achieve efficient deduplication even in the presence of anonymous data. Without storing duplicate data, deduplication can efficiently utilize the storage space of the memory 102 of the data storage device 100A, thereby improving system performance.

[0078] Figure 1B A block diagram of a data storage device provided according to another embodiment of the present invention. (See also...) Figure 1BThe data storage device 100B is shown. The data storage device 100B also includes a deduplication storage software module 114 and software modules 116A to 116G installed in the memory 102. The data storage device 100B also includes a data element recovery software module 118, which further includes a data element receiving software module 120A, a deduplication extraction software module 120B, and an insertion software module 120C. A controller 104 and a network interface 112 are also shown.

[0079] On the other hand, the present invention provides a data storage device 100B, including a memory 102 for storing one or more data elements 106. The data storage device 100B further includes a deduplication storage software module 114 for storing at least one of the one or more data elements 106 using deduplication. The data storage device 100B further includes: a software module 116A for receiving an instruction for data to be anonymized; a software module 116B for receiving a data element to be stored; a software module 116C for parsing the data element to be stored to find one or more data portions to be anonymized; a software module 116D for generating a copy of the one or more data portions to be anonymized; a software module 116E for deleting the one or more data portions to be anonymized from the data element to be stored, generating a modified data element to be stored; a software module 116F for generating a copy of the modified data element to be stored using deduplication; and a software module 116G for storing the copy of the modified data element to be stored and the copy of the one or more data portions to be anonymized.

[0080] When the deduplication storage software module 114 is executed by the controller 104, it is used to store at least one of one or more data elements 106 using deduplication. Deduplication is performed on the data storage device 100B upon receiving one or more data elements 106 for storage. One or more portions of data to be anonymized from the received data elements are parsed from the data elements, and deduplication is further performed on the remaining data elements (i.e., modified data) by the deduplication storage software module 114. In one example, variable-size deduplication is performed by the deduplication storage module 114. In such an example, modified data elements are chunked. Furthermore, each chunk has a hash signature generated using a hash algorithm. Additionally, if the hash signature of a chunk is the same as the hash signature of a previously stored chunk, the modified data element is identified as a duplicate data element, and a pointer to the chunk is stored.

[0081] In an exemplary implementation, the deduplication storage software module 114 may include software modules 116A to 116G. In one implementation, the deduplication storage software module 114 (including software modules 116A to 116G) may be implemented as a separate circuit in the data storage device 100B. Alternatively, in another implementation, the deduplication storage software module 114 may be implemented as a circuit that performs various operations of software modules 116A to 116G. In one implementation, software modules 116A to 116G may be operatively and communicatively coupled to each other and executed by the controller 104 of the data storage device 100B to perform their respective operations.

[0082] Data storage device 100B includes software module 116A, which, when executed, receives an indication of data to be anonymized. When executed by controller 104, software module 116A receives the indication of data to be anonymized. This indication refers to user input received by controller 104. The user input is provided by the user as an indication through software module 116A. For example, software module 116A may be a graphical user interface presented on a display associated with data storage device 100B. The user may provide user input indicating the selection of a specific user configuration or setting that indicates how the data to be anonymized is identified or which configuration is suitable for the data to be anonymized.

[0083] The data storage device 100B includes a software module 116B, which, when executed, receives data elements to be stored. When the software module 116B is executed by the controller 104, the network interface 112 can receive the data elements to be stored.

[0084] Data storage device 100B includes software module 116C, which, when executed, parses the data element to be stored to locate one or more data portions to be anonymized. Software module 116C parses the data element to be stored when executed by controller 104. Software module 116C may include a parser to parse the data element to be stored. The parser can locate all data portions or blocks to be anonymized. Software module 116C is used to parse the data element to be stored based on an instruction to anonymize data received from software module 116A. In one example, when software module 116C is executed by controller 104, software module 116C parses the data element according to the data field to be anonymized. In another example, software module 116C parses the data element according to a data format that matches the anonymization format.

[0085] The data storage device 100B includes a software module 116D, which, when executed by the controller 104, generates one or more copies of the data portion to be anonymized. When the controller 104 executes the software module 116D, the software module 116D generates one or more copies of the data portion to be anonymized to separate the data portion to be anonymized from the remaining data elements received by the software module 116B.

[0086] Data storage device 100B includes software module 116E. When controller 104 executes software module 116E, software module 116E deletes one or more portions of data to be anonymized from the data element to be stored, generating a modified data element to be stored. When controller 104 executes software module 116E, software module 116E separates the data portion identified as to be anonymized from the rest of the data and retains a pointer to the original location of the data. This allows for duplicate data deletion of the modified data element, thus identifying duplicate data elements.

[0087] Data storage device 100B includes software module 116F, which, when executed by controller 104, generates a copy of the modified data element to be stored for deduplication. When controller 104 executes software module 116F, it stores the copy of the modified data element separately from copies of one or more data portions to be anonymized, thereby enabling the identification of duplicate data elements.

[0088] Data storage device 100B includes software module 116G, which, when executed by controller 104, stores copies of the modified data elements to be stored and copies of one or more data portions to be anonymized. Software module 116G stores suspected data portions to be anonymized, or identified and anonymized data portions, separately.

[0089] According to one embodiment, the data storage device 100B further includes a data element recovery software module 118 for recovering data elements. The data element recovery software module 118 includes: a data element receiving software module 120A for receiving a copy of the modified data element to be recovered and copies of one or more anonymous data portions; a deduplication extraction software module 120B for extracting the modified data element using deduplication extraction to generate a recovered copy of the data element; and an insertion software module 120C for inserting the one or more anonymous data portions into the recovered copy of the data element. In one example, when the controller 104 executes the data element recovery software module 118, the data element recovery software module 118 recovers the data element upon receiving a recovery request from a user via a user equipment. According to the recovery request, when the controller 104 executes the data element receiving software module 120A, the data element receiving software module 120A receives a copy of the modified data element to be recovered and copies of one or more anonymous data portions from the memory 102. If the previously stored modified data element is a duplicate, and only the address of the original data element is stored, then when the controller 104 executes the deduplication extraction software module 120B, the deduplication extraction software module 120B receives the original data element through the network interface 112. When the controller 104 executes the insertion software module 120C, the insertion software module 120C generates a restored copy of the data element by combining the modified data element with one or more anonymous data portions.

[0090] Advantageously, the software modules 118A to 118G of the present invention are capable of efficiently and effectively deduplicating data elements having one or more data portions to be anonymized. Therefore, the storage space of the memory 102 (e.g., auxiliary memory) utilized by the data storage device 100B is reduced.

[0091] Figure 2 This is a flowchart of a method for a data storage device provided according to an embodiment of the present invention. Method 200 is performed at a data storage device (e.g., data storage device 100A or 100B) as described in FIG1. ​​Method 200 includes steps 202 to 214.

[0092] On one hand, the present invention provides a method 200 for a data storage device (e.g., data storage device 100A or 100B), the data storage device including a memory 102 for storing one or more data elements 106, the method 200 including storing at least one of the one or more data elements 106 using deduplication, wherein the method 200 further includes: receiving an instruction for data to be anonymized; receiving a data element to be stored; parsing the data element to be stored to locate one or more data portions to be anonymized; generating a copy of the one or more data portions to be anonymized; deleting the one or more data portions to be anonymized from the data element to be stored, generating a modified data element to be stored; generating a copy of the modified data element to be stored using deduplication; storing the copy of the modified data element to be stored and the copy of the one or more data portions to be anonymized.

[0093] In step 202, method 200 includes receiving an instruction to anonymize data. The instruction to anonymize data is received by a data storage device (e.g., data storage device 100A or 100B). The instruction to anonymize data may be provided by a user. In one example, user input may be received, which is an instruction to select a setting or configuration for identifying sensitive data. These selected or configured settings and configurations may be pre-stored in memory.

[0094] In step 204, method 200 further includes receiving data elements to be stored. Controller 104 is configured to receive the data elements to be stored via a network interface 112 of a data storage device (e.g., data storage device 100A or 100B). The data elements to be stored may be received by controller 104 from an external device (e.g., main memory) communicatively coupled to the data storage device (e.g., data storage device 100A or 100B) via a communication network.

[0095] In step 206, method 200 further includes parsing the data element to be stored to locate one or more portions of data to be anonymized. The data element can be parsed by a parser via a controller 104 of the data storage device (e.g., data storage device 100A or 100B). The data element to be stored is parsed according to received instructions for data anonymization (i.e., user configuration). In one example, the algorithm used by controller 104 to detect which data is sensitive (i.e., confidential) may operate based on configuration received by controller 104 from the user (i.e., user input).

[0096] According to one embodiment, data elements are parsed to locate one or more portions of data to be anonymized by searching for data portions in data fields designated as fields to be anonymized. In one example, the data fields could be name, age, phone number, credit card number, or residential address, where the credit card number can be identified as the field to be anonymized.

[0097] According to one embodiment, data elements are parsed to locate one or more portions of data to be anonymized by searching for data portions with a data format that matches the anonymization format. In one example, data elements are parsed by searching for data portions with a data format that matches the anonymization format, and if the received data is structured, unstructured, or has some unique pattern, this is done by understanding the format of the data elements.

[0098] In step 208, method 200 further includes generating one or more copies of the data portion to be anonymized. Controller 104 generates one or more copies of the data portion to be anonymized, enabling the data portion to be anonymized to be separated from the remaining received data elements. In one example, one or more copies of the data portion to be anonymized are generated based on a start offset and an end offset of the data portion, which define the start and end of the one or more data portions to be anonymized (note that in the credit card example, the credit card data has two start points, each followed by 16 characters).

[0099] In step 210, method 200 further includes deleting one or more portions of data to be anonymized from the data element to be stored, generating a modified data element to be stored. Controller 104 separates the data portion identified as the data to be anonymized from the rest of the data and retains a pointer to the original location of the data. Duplicate data deletion can now be implemented on the modified data element to identify duplicate data elements.

[0100] In step 212, method 200 further includes generating a copy of the modified data element to be stored for deduplication. The copy of the modified data element to be stored is stored separately from copies of one or more data portions to be anonymized, thereby enabling the identification of duplicate data elements.

[0101] In step 214, method 200 further includes storing copies of the modified data elements and copies of one or more data portions to be anonymized. Controller 104 stores the suspected data portions to be anonymized or the identified and anonymized data portions separately.

[0102] According to one embodiment, method 200 further includes storing a copy of the modified data element to be stored in a first storage area 108; and storing one or more data portions to be anonymized in a second storage area 110. Storing the copy of the modified data element and the one or more data portions to be anonymized separately enables deduplication to be performed on the modified data element. In conventional techniques, received data elements are directly stored without parsing and deleting one or more data portions, resulting in no deduplication.

[0103] According to one embodiment, method 200 further includes storing one or more portions of data to be anonymized, wherein the one or more portions of data to be anonymized are appended to a copy of the modified data element to be stored. Appending one or more portions of data to be anonymized to a copy of the modified data element to be stored enables efficient storage and retrieval of the data element when needed.

[0104] According to one embodiment, the controller 200 further includes generating a corresponding pointer to each of the one or more data portions to be anonymized, and storing copies of the one or more data portions to be anonymized and the corresponding pointers. Generating and storing the pointer to each of the one or more data portions to be anonymized enables the retrieval of the original data (i.e., the received data element) during the retrieval of the data element.

[0105] According to one embodiment, method 200 includes: the corresponding pointer indicating the position of a corresponding portion of data to be anonymized within the data element. The pointer corresponds to the address of the source (i.e., within the data element) of the one or more portions of data to be anonymized.

[0106] According to one embodiment, method 200 further includes recovering a data element by: receiving a copy of the modified data element to be recovered and copies of one or more anonymous data portions; extracting the modified data element using deduplication extraction to generate a recovered copy of the data element; and inserting the one or more anonymous data portions into the recovered copy of the data element. In one example, controller 104 is capable of recovering a data element upon receiving a recovery request from a user (via a user device). Upon the recovery request, controller 104 receives a copy of the modified data element to be recovered and copies of one or more anonymous data portions from memory. Controller 104 is also capable of generating a recovered copy of the data element by combining the modified data element with one or more anonymous data portions. In one example, if the previously stored modified data element is a duplicate and only the address of the original data element is stored, the original data element is received by controller 104.

[0107] Steps 202 to 214 are merely illustrative, and other alternatives may be provided, in which one or more steps are added, one or more steps are deleted, or one or more steps are provided in a different order, without departing from the scope of the claims herein.

[0108] In one aspect, a computer-readable medium carrying computer instructions is provided, which, when loaded into and executed by a controller 104 of a data storage device (e.g., data storage device 100A or 100B), enable the data storage device to implement method 200. Examples of implementations of the computer-readable medium include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) cards, solid-state drives (SSDs), computer-readable storage media, and / or CPU cache. In another aspect, a computer program product is provided, comprising a non-transitory computer-readable storage medium. Computer program code is stored on the non-transitory computer-readable storage medium, which is executable by a processor to perform method 200. A computer-readable storage medium for providing non-transient memory may include, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0109] Modifications to the embodiments of the invention described above may be made without departing from the scope of the invention as defined by the appended claims. Expressions such as “comprising,” “including,” “combined,” “having,” and “are” used to describe and claim the invention should be interpreted in a non-unique manner, allowing for the appearance of items, parts, or elements not explicitly described. Singular references should also be interpreted in relation to the plural. The term “exemplary” as used herein means “as an example, instance, or illustration.” Any “exemplary” embodiment is not necessarily to be construed as preferential or superior to other embodiments, and / or does not exclude combinations of features from other embodiments. The term “optionally” as used herein means “provided in some embodiments and not in others.” It should be understood that certain features of the invention described in the context of a single embodiment for clarity may also be provided in combination in a single embodiment. Conversely, various features of the invention described in the context of a single embodiment for clarity may also be provided individually or in any suitable combination or as embodiments of the invention as described in any other way.

Claims

1. A data storage device (100A, 100B), characterized in that, The data storage device (100A, 100B) includes a memory (102) for storing one or more data elements (106), and the controller (104) for storing at least one of the one or more data elements (106) using deduplication. The data storage device (100A, 100B) is characterized in that the controller (104) is further configured to: Instructions to receive data to be anonymized; Receive data elements to be stored; Parse the data elements to be stored to locate one or more portions of data to be anonymized; Generate copies of the one or more data portions to be anonymized, and generate a corresponding pointer for each of the one or more data portions to be anonymized, and store the corresponding pointers, wherein the corresponding pointers indicate the position of the corresponding data portion to be anonymized in the data element; Delete one or more of the data portions to be anonymized from the data element to be stored, and generate a modified data element to be stored; Generate a copy of the modified data element from the deduplication storage to be used; Store a copy of the modified data element to be stored and a copy of the one or more data portions to be anonymized.

2. The data storage device (100A, 100B) according to claim 1, characterized in that, The controller (104) is also used for: A copy of the modified data element to be stored is stored in the first storage area (108); The one or more portions of data to be anonymized are stored in the second storage area (110).

3. The data storage device (100A, 100B) according to claim 1, characterized in that, The controller (104) is also configured to store the one or more portions of data to be anonymized, wherein the one or more portions of data to be anonymized are attached to a copy of the modified data element to be stored.

4. The data storage device (100A, 100B) according to any one of the preceding claims, characterized in that, The controller (104) is also configured to parse the data element to be stored in order to find one or more data portions to be anonymized by searching for data portions in a data field designated as an anonymized field.

5. The data storage device (100A, 100B) according to any one of claims 1-3, characterized in that, The controller (104) is also configured to parse the data element to be stored in order to locate one or more data portions to be anonymized by searching for data portions having a data format that matches the anonymization format.

6. The data storage device (100A, 100B) according to any one of claims 1-3, characterized in that, The controller (104) is also used to recover data elements in the following manner: Receive a copy of the modified data element to be restored, as well as a copy of one or more anonymous data portions; The modified data element is extracted by utilizing deduplication extraction, and a restored copy of the data element is generated. Insert the one or more anonymous data portions into the restored copy of the data element.

7. A method (200) for a data storage device (100A, 100B), characterized in that, The data storage device (100A, 100B) includes a memory (102) for storing one or more data elements (106), the method includes storing at least one of the one or more data elements (106) using deduplication, and the method (200) is characterized in that the method (200) further includes: Instructions to receive data to be anonymized; Receive data elements to be stored; Parse the data elements to be stored to locate one or more portions of data to be anonymized; Generate copies of the one or more data portions to be anonymized, and generate a corresponding pointer for each of the one or more data portions to be anonymized, and store the corresponding pointers, wherein the corresponding pointers indicate the position of the corresponding data portion to be anonymized in the data element; Delete one or more of the data portions to be anonymized from the data element to be stored, and generate a modified data element to be stored; Generate a copy of the modified data element from the deduplication storage to be used; Store a copy of the modified data element to be stored and a copy of the one or more data portions to be anonymized.

8. The method (200) according to claim 7, characterized in that, The method (200) also includes recovering data elements by: Receive a copy of the modified data element to be restored, as well as a copy of one or more anonymous data portions; The modified data element is extracted by utilizing deduplication extraction, and a restored copy of the data element is generated. Insert the one or more anonymous data portions into the restored copy of the data element.

9. A computer-readable medium carrying computer instructions, characterized in that, When the computer instructions are loaded into and executed by the controller (104) of the data storage device (100A, 100B), the data storage device (100A, 100B) is enabled to perform the method according to claim 7 or 8.

10. A data storage device (100A, 100B), characterized in that, The data storage devices (100A, 100B) include a memory (102) for storing one or more data elements (106), and a deduplication storage software module (114) for storing at least one of the one or more data elements (106) using deduplication. The data storage devices (100A, 100B) are characterized in that they further include: The first software module (116A) is used to receive instructions for data to be anonymized; The second software module (116B) is used to receive data elements to be stored; The third software module (116C) is used to parse the data elements to be stored in order to find one or more data portions to be anonymized; The fourth software module (116D) is used to generate copies of the one or more data portions to be anonymized, and to generate a corresponding pointer for each of the one or more data portions to be anonymized, and to store the corresponding pointer, wherein the corresponding pointer indicates the position of the corresponding data portion to be anonymized in the data element; The fifth software module (116E) is used to delete one or more data portions to be anonymized from the data element to be stored, and generate a modified data element to be stored; The sixth software module (116F) is used to generate a copy of the modified data element to be used for deduplication storage; The seventh software module (116G) is used to store copies of the modified data elements to be stored and copies of the one or more data portions to be anonymized.

11. The data storage device (100A, 100B) according to claim 10, characterized in that, It also includes a data element recovery software module (118) for recovering data elements, the data element recovery software module (118) comprising: The data element receiving software module (120A) is used to receive a copy of the modified data element to be recovered and a copy of one or more anonymous data portions; The deduplication extraction software module (120B) is used to extract the modified data element by utilizing deduplication extraction and generate a restored copy of the data element; Insertion software module (120C) for inserting one or more anonymous data portions into a restored copy of the data element.

Images