Ransomware detection in deduplicated storage systems

Abstract

This disclosure relates to ransomware detection in deduplication storage systems. Example implementations relate to deduplication operations in storage systems. Examples include: receiving data units of a backup stream to be stored in a deduplication storage system; performing a first matching operation for matching data units against a first container index among a plurality of container indexes, wherein the plurality of container indexes include metadata indicating the storage location of data units previously stored in persistent storage; calculating a first ratio, the first ratio indicating the amount of deduplication that occurred during the first matching operation against the first container index; and identifying at least a portion of the backup stream as potentially affected by a ransomware attack in response to determining that the first ratio violates a condition of local ratio history data regarding the first container index.

Description

Background Technology

[0001] Data reduction techniques can be applied to reduce the amount of data stored in a storage system. Example data reduction techniques include deduplication. Deduplication identifies duplicate data units and aims to reduce or eliminate the number of instances of duplicate data units stored in the storage system. Attached Figure Description

[0002] Some implementation methods are described with reference to the following figures.

[0003] Figure 1 This is a schematic diagram of an example storage system according to some implementation methods.

[0004] Figure 2 This is a diagram of an example data structure based on some implementation methods.

[0005] Figures 3A to 3B This is a diagram of an example data structure based on some implementation methods.

[0006] Figure 4 This is a diagram illustrating an example process based on some implementation methods.

[0007] Figure 5 This is a diagram illustrating an example process based on some implementation methods.

[0008] Figure 6 This is a schematic diagram of an example computing device according to some implementation methods.

[0009] Figure 7 This is a diagram of an example machine-readable medium storing instructions according to some implementation methods.

[0010] Figure 8 This is a diagram illustrating an example process based on some implementation methods.

[0011] In all the accompanying drawings, the same reference numerals refer to similar but not necessarily identical elements. The drawings are not necessarily drawn to scale, and the dimensions of some parts may be enlarged to illustrate the examples more clearly. Furthermore, the drawings provide examples and / or embodiments consistent with the description; however, the description is not limited to the examples and / or embodiments provided in the drawings. Detailed Implementation

[0012] In this disclosure, unless the context clearly indicates otherwise, the terms “a,” “an,” or “the” are intended to include the plural form as well. Similarly, when used in this disclosure, the terms “includes / including / comprises / comprising” or “have / having” indicate the presence of the element but do not preclude the presence or addition of other elements.

[0013] In some examples, the storage system can back up a collection of data (referred to herein as a “stream” or “data stream”) in a deduplicated form, thereby reducing the amount of storage space required to store the data stream. The storage system can create “backup items” to represent the data stream in deduplicated form. The storage system can perform a deduplication process, including breaking down the data stream into discrete data units (or “chunks”) and determining the “fingerprints” of these incoming data units (described below). Further, the storage system can compare the fingerprints of the incoming data units with the fingerprints of stored data units, and can thereby determine which incoming data units are copies of previously stored data units (e.g., when the comparison indicates a matching fingerprint). In the case of a data unit being a copy, the storage system can store a reference to the previously stored data unit instead of storing the duplicated incoming data unit.

[0014] As used herein, a "fingerprint" refers to the value obtained by applying a function to the contents of a data unit (where "contents" can include all or a subset of the contents of the data unit). Examples of functions that can be applied include hash functions that generate a hash value based on the contents of an input data unit. Examples of hash functions include cryptographic hash functions such as Secure Hash Algorithm 2 (SHA-2) hash functions (e.g., SHA-224, SHA-256, SHA-384, etc.). In other examples, other types of hash functions or other types of fingerprint functions may be used.

[0015] A “storage system” may include storage devices or arrays of storage devices. A storage system may also include storage controllers that manage access to the storage devices(s). A “data unit” may refer to any portion of data in the storage system that can be individually identified. In some cases, a data unit may refer to a block, a collection of blocks, or any other portion of data. In some examples, the storage system may store data units in persistent storage devices. Persistent storage devices may be implemented using one or more persistent (e.g., non-volatile) storage devices (such as multiple disk-based storage devices (e.g., multiple hard disk drives (HDDs)), multiple solid-state drives (SSDs) (such as multiple flash storage devices), etc., or combinations thereof).

[0016] "Controller" can refer to hardware processing circuitry, which may include any or a combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, a digital signal processor, or other hardware processing circuitry. Alternatively, "controller" can refer to a combination of hardware processing circuitry and machine-readable instructions (software and / or firmware) that can be executed on the hardware processing circuitry.

[0017] In some examples, deduplication storage systems can use metadata to process inbound data streams. For example, such metadata may include a data recipe (also referred to herein as a “manifestation”) specifying the receiving order of particular data units (e.g., within the data stream). Processing each data stream may be referred to herein as a “backup process.” Subsequently, in response to a read request, the deduplication system can use a set of manifests (also referred to herein as “item metadata”) to determine the receiving order of the data units, thereby enabling the regeneration of the original data stream. Accordingly, the set of manifests may be a representation of the original data stream. The manifest may include a series of records, each representing a specific set of data units. Records in the manifest may include one or more fields identifying container indices that include storage information for the data units. For example, a container index may include one or more fields specifying location information of the stored data units (e.g., container, offset, etc.), compression and / or encryption characteristics of the stored data units, etc. The container index may include a reference count indicating the number of manifest references for each data unit.

[0018] In some examples, upon receiving a data unit (e.g., from a data stream), it can be matched against one or more container indices to determine if the same block is already stored in a container of the deduplication storage system. For example, the deduplication storage system can compare the fingerprint of the received data unit with fingerprints in one or more container indices. If no matching fingerprint is found in the searched container indices, the received data unit can be added to the container, and an entry for the received data unit can be added to the container index corresponding to that container. However, if a matching fingerprint is found in the searched container indexes, it can be determined that the same data unit as the received data unit is already stored in the container. In response to this determination, the reference count of the corresponding entry is incremented, and the received data unit is not stored in the container (because it already exists in one of the containers), thus avoiding the storage of duplicate data units in the deduplication storage system. As used herein, the term "matching operation" can refer to the operation used to compare the fingerprints of a set of multiple data units (e.g., from a particular backup data stream) with fingerprints stored in container indices.

[0019] In some examples, storage systems may be vulnerable to ransomware attacks that encrypt the stored data. If a ransom is not paid to the attacker, the data may remain encrypted and unusable. Therefore, in some examples, storage systems may implement protections to detect and / or combat ransomware attacks. For example, the storage system may execute specialized programs that continuously analyze the characteristics of the stored data and determine whether these characteristics match known profiles of data that has been encrypted by ransomware. However, such protections may require significant computational resources (e.g., processing time, memory space, etc.). Furthermore, such protections may not detect ransomware encryption until the data has been stored in persistent storage and earlier (unaffected) stored copies of the data have been overwritten. Therefore, in such examples, ransomware can encrypt large amounts of valuable stored data.

[0020] According to some embodiments of this disclosure, the controller of a deduplication storage system can perform a matching operation against a container index to deduplicat a set of data units from a backup data stream (e.g., by comparing the fingerprints of the set of data units with fingerprints stored in the container index). The controller can calculate a ratio indicating the amount of deduplication that occurred during the matching operation. Further, the controller can determine whether the calculated ratio violates conditions regarding local ratio history data stored in the container index. If so, the controller can identify the set of data units as potentially encrypted by ransomware. In this way, some embodiments can provide detection of ransomware attacks without executing dedicated programs that may consume significant computational resources. Further, some embodiments can provide rapid identification of specific affected data locations and can reduce the amount of valuable data lost due to malicious encryption. Reference is made below. Figures 1 to 8 The disclosed techniques for detecting ransomware will be discussed further.

[0021] Figure 1 -Example Storage System

[0022] Figure 1 An example of a storage system 100 according to some embodiments is shown, including a storage controller 110, a memory 115, and a persistent storage device 140. The persistent storage device 140 may include one or more non-transitory storage media, such as a hard disk drive (HDD), a solid-state drive (SSD), an optical disk, or a combination thereof. The memory 115 may be implemented using a semiconductor memory such as random access memory (RAM). In some examples, the storage controller 110 may be implemented via hardware (e.g., electronic circuitry) or a combination of hardware and programming (e.g., including at least one processor and instructions executable by said at least one processor and stored on at least one machine-readable storage medium).

[0023] like Figure 1 As shown, memory 115 and persistent storage device 140 can store various data structures, including at least backup items 145, list 150, container index 160, and data container 170. In some examples, copies of list 150, container index 160, and data container 170 can be transferred between memory 115 and persistent storage device 140 (e.g., via read and write input / output (I / O) operations).

[0024] In some implementations, storage system 100 may perform deduplication on the stored data. For example, storage controller 110 may receive an inbound data stream 105 comprising multiple data units and may store at least one copy of each data unit in data container 170 (e.g., by appending the data unit to the end of data container 170). In some examples, each data container 170 may be divided into entities 175, wherein each entity 175 comprises multiple stored data units.

[0025] In one or more embodiments, the storage controller 110 may generate a fingerprint for each data unit. For example, the fingerprint may include a complete or partial hash value based on the data unit. To determine whether an incoming data unit is a copy of a stored data unit, the storage controller 110 may compare the fingerprint generated for the incoming data unit with a fingerprint in at least one container index. If a match is identified, the storage controller 110 may determine that the storage system 100 has already stored a copy of the incoming data unit. Instead of storing the duplicate incoming data unit, the storage controller 110 may then store a reference to the previous data unit.

[0026] In some implementations, storage controller 110 may generate backup entries 145 to represent each received data stream 105 in a deduplicated form. Each backup entry 145 may reference multiple lists 150. Lists 150 record the order in which the received data units were received. Further, lists 150 may include pointers or other information indicating a container index 160 associated with each data unit. In some implementations, the associated container index 160 may indicate the storage location of the data unit. For example, the associated container index 160 may include information specifying that the data unit is stored in an entity at a specific offset and that the entity is stored in a data container 170 at a specific offset. Further, container index 160 may include a reference count indicating the number of lists 150 referencing each data unit.

[0027] In some implementations, storage controller 110 may receive a read request to access stored data, and in response may access list 150 to determine the sequence of data units constituting the original data. Storage controller 110 may then use pointer data included in list 150 to identify the container index 160 associated with the data unit. Further, storage controller 110 may use information included in the identified container index 160 (and information included in list 150) to determine the storage location of the data unit (e.g., data container 170, entity 175, offset, etc.), and may then read the data unit from the determined location.

[0028] In some implementations, each container index 160 may include a local ratio history 165, which may be a data structure for storing historical information (also referred to herein as "local ratio history") about the deduplication ratio of that container index 160. For example, the local ratio history 165 may include a list of ratio values ​​indicating the amount of deduplication that occurred during a previous match operation against the container index 160. In another example, the local ratio history 165 may be a rolling average of the ratio values ​​based on the N most recent match operations against the container index 160, where N is a specified integer.

[0029] In some implementations, storage controller 110 may calculate a new ratio value based on the current matching operation against container index 160, and may determine whether the new ratio value violates a condition regarding local ratio history 165 stored in container index 160. For example, storage controller 110 may calculate an average ratio value for local ratio history 165, and may determine whether the new ratio value exceeds that average ratio value. If the new ratio value violates a condition regarding local ratio history 165, storage controller 110 may take action to indicate that the data stream may have been affected by a ransomware attack. Further, in some implementations, storage controller 110 may use container index 160 to identify specific portions of data stream 195 that may have been affected by a ransomware attack. Reference is made below. Figures 4 to 5 Describe an example process for identifying potential ransomware attacks.

[0030] Figure 2 -Example Data Structure

[0031] Now for reference Figure 2The diagram illustrates an example data structure 200 used in deduplication according to some embodiments. As shown, data structure 200 may include backup item metadata 202, inventory records 210, container indexes 220, and data containers 250. In some embodiments, backup item metadata 202 may include multiple inventory records 210. In some examples, backup item metadata 202, inventory records 210, container indexes 220, and data containers 250 may substantially correspond to backup item 145, inventory 150, container index 160, and data container 170, respectively (e.g., ...). Figure 1 Example implementations (as shown). In some examples, data structure 200 may be provided by storage controller 110 (e.g., Figure 1 (As shown) Generate and / or manage.

[0032] Although in order to simplify Figure 2 The illustrations show only one instance of each data structure, but data structure 200 can include multiple instances of backup project metadata 202, each instance including or pointing to one or more inventory records 210. In this example, data structure 200 can include multiple inventory records 210. Inventory records 210 can reference multiple container indices 220, each container index corresponding to one of multiple containers 250. Each container index 220 can include one or more data unit records 230 and one or more entity records 240.

[0033] like Figure 2 As shown, in some examples, manifest record 210 may include various fields such as offset, length, container index, and cell address. In some implementations, each container index 220 may include any number of data cell records 230 and entity records 240. Each data cell record 230 may include various fields such as fingerprint (e.g., hash of the data cell), cell address, entity identifier, cell offset (i.e., the offset of the data cell within the entity), reference count value, and cell length. In some examples, the reference count value may indicate the number of manifest records 210 referencing the data cell record 230. Further, each entity record 240 may include various fields such as entity identifier, entity offset (i.e., the offset of the entity within the container), storage length (i.e., the length of the data cell within the entity), decompression length, checksum value, and compression / encryption information (e.g., compression type, encryption type, etc.). In some implementations, each container 250 may include any number of entities 260, and each entity 260 may include any number of stored data cells.

[0034] In one or more embodiments, data structure 200 can be used to retrieve stored deduplicated data. For example, a read request can specify the offset and length of data in a given file. These request parameters can be matched against the offset and length fields of a specific manifest record 210. The container index and cell address of the specific manifest record 210 can then be matched against a specific data cell record 230 included in container index 220. Further, the entity identifier of the specific data cell record 230 can be matched against the entity identifier of a specific entity record 240. Additionally, one or more other fields of the specific entity record 240 (e.g., entity offset, storage length, checksum, etc.) can be used to identify containers 250 and entities 260, from which data cells can then be read.

[0035] In some implementations, each container index 220 may include an inventory list 222 and a local ratio history 224. The inventory list 222 may be a data structure used to identify each inventory record 210 that references the container index 220. For example, each time the container index 220 is generated or updated to include information about a particular inventory record 210, the inventory list 222 in the container index 220 is updated to store the identifier of that inventory record 210. Further, when the container index 220 is no longer associated with an inventory record 210, the identifier of the inventory record 210 is removed from the inventory list 222.

[0036] In some implementations, local ratio history 224 may substantially correspond to local ratio history 165 (e.g., Figure 1 The example implementation is shown below. The local ratio history 224 can be a data structure used to store historical information (also referred to herein as "local ratio history") about the deduplication ratio of the container index 160. For example, the local ratio history 224 can store a list of deduplication ratios for previous matching operations against the container index 220. In another example, the local ratio history 224 can store a rolling average of previous deduplication ratios. See below for further details. Figures 3A to 3B Some example implementations of the local ratio history 224 are described.

[0037] Figures 3A to 3B -Example Data Structure

[0038] Figure 3A An example ratio list 310 according to some implementations is shown. In some examples, ratio list 310 may substantially correspond to container index 220 (e.g., ...). Figure 2 Example implementation of the local ratio history 224 included in (shown).

[0039] like Figure 3AAs shown, the ratio list 310 can be a list structure that includes multiple stored ratio values, wherein each ratio value is associated with an index of the container storing the ratio list (e.g., ...). Figure 1 Different matching operations are associated with the container index 160 shown. In some embodiments, each stored ratio value may be a ratio of the number of new data units (i.e., data units that do not match previously stored data units) during a particular matching operation (e.g., a past operation for comparing fingerprints of a set of multiple data units from a particular backup data stream with fingerprints in the container index) to the number of matched data units (i.e., data units that match previously stored data units). In some embodiments, ratio list 310 is used to store a maximum number N of the latest ratio values, where N is a positive integer (e.g., N = 100, N = 200, etc.). In other embodiments, ratio list 310 is used to store an unlimited number of ratio values ​​(e.g., starting from a system startup event, reset event, etc.).

[0040] In some implementations, the controller (e.g., Figure 1 The storage controller 110 shown can calculate a new ratio value when performing a new matching operation against a container index (e.g., a new operation to compare the fingerprints of a recently received set of multiple data units with fingerprints in the container index), and can determine whether the new ratio value violates some or all of the conditions regarding the ratio values ​​stored in the ratio list 310. For example, the controller can determine whether the new ratio value exceeds the average of a subset of the ratio list 310, wherein the subset comprises a predefined number M of the most recently stored ratio values, and wherein M is a positive integer (e.g., M = 10, M = 20, etc.). In another example, the controller can determine whether the new ratio value exceeds the average of all ratio values ​​stored in the ratio list 310. In yet another example, the controller can determine whether the new ratio value exceeds the average of the average of a subset (or all) of the stored ratio values ​​by at least a specified percentage or amount (e.g., at least 10% of the average). In yet another example, the controller can determine the rate of change of the stored ratio values ​​over time, and can determine whether the new ratio value represents a rate of change exceeding the historical rate of change (e.g., exceeding at least a specified percentage or amount). Other examples are also possible.

[0041] like Figure 3A As shown, a new ratio value (e.g., calculated based on a new matching operation for the container index) can be inserted into ratio list 310. For example, after determining whether the new ratio value violates a condition regarding a previous ratio value for ratio list 310, the controller can insert the new ratio value into ratio list 310 (e.g., as the latest ratio value).

[0042] Now for reference Figure 3BThe diagram illustrates an example rolling average field 320 according to some implementations. In some examples, the rolling average field 320 may substantially correspond to container index 220 (e.g., ...). Figure 2 The example implementation includes the local ratio history 224 shown in the diagram. In some implementations, the rolling average field 320 may store the value of the rolling average of the ratio values ​​calculated as the latest N matching operations, where N is a positive integer.

[0043] In some implementations, the controller (e.g., Figure 1 The storage controller 110 shown can calculate a new ratio value when performing a new match operation against the container index and can determine whether the new ratio value violates a condition regarding the value of the rolling average field 320. For example, the controller can determine whether the new ratio value exceeds the value of the rolling average field 320. In another example, the controller can determine whether the new ratio value exceeds the value of the rolling average field 320 by at least a specified percentage or amount. Other examples are also possible.

[0044] In some implementations, (e.g., calculated based on a new match operation for the container index) a new ratio value can be used to update the rolling average field 320. For example, after determining whether the new ratio value violates a condition regarding the rolling average stored in the rolling average field 320, the controller can recalculate the rolling average and store the recalculated rolling average in the rolling average field 320 (e.g., by overwriting the previous rolling average).

[0045] Figure 4 - Example procedures for ransomware detection

[0046] Figure 4 An example process 400 for ransomware detection according to some embodiments is shown. For illustration purposes, reference may be made below to examples illustrating some embodiments. Figures 1 to 3B The details of process 400 are described. However, other implementations are also possible. In some examples, storage controller 110 can be used ( Figure 1 Process 400 is executed as shown. Process 400 can be implemented in hardware or a combination of hardware and programming (e.g., machine-readable instructions executable by (multiple) processors). Machine-readable instructions can be stored in a non-transitory computer-readable medium such as an optical storage device, semiconductor storage device, or magnetic storage device. Machine-readable instructions can be executed by a single processor, multiple processors, a single processing engine, multiple processing engines, etc.

[0047] Box 410 may include receiving a stream of data units to be stored in a deduplication storage system. For example, refer to Figure 1The storage controller 110 generates a backup item 145 to represent the data flow 105. Further, the storage controller 110 records the order in which the data units are received in a list 150.

[0048] Box 420 (e.g.) Figure 4 (As shown) may include performing matching operations for matching data units against a container index. For example, refer to Figure 1 Storage controller 110 receives a data stream 105 to be stored in deduplication storage system 100. Storage controller 110 performs a matching operation on the received data units against container index 160. To perform the matching operation, storage controller 110 generates a fingerprint for the received data unit and then compares the generated fingerprint with the fingerprint included in container index 160. If a match is identified, storage controller 110 determines that storage system 100 has already stored a copy of the incoming data unit. In response to this determination, storage controller 110 stores a reference to the previous data unit (e.g., in Listing 150) in a deduplicated form, instead of storing a new copy of the incoming data unit.

[0049] Box 430 (e.g.) Figure 4 (As shown) may include calculating the ratio of the new data unit used for the matching operation to the previously stored data unit. For example, refer to Figure 1 During a matching operation (e.g., during an operation to compare fingerprints of a set of multiple data units in data stream 105 with fingerprints in container index 160), storage controller 110 monitors the number of new data units (i.e., data units that do not match previously stored data units) and the number of matching data units (i.e., data units that match previously stored data units). Upon completion of the matching operation, storage controller 110 calculates a ratio value as the number of new data units divided by the number of matching data units.

[0050] Decision box 440 (e.g.) Figure 4 (As shown) This may include using local ratio history based on the container index to determine whether the calculated ratio violates a condition. If so, process 400 can continue at box 450 (as described below). Otherwise, if it is determined that the calculated ratio does not violate a condition, process 400 can return to box 410 (e.g., continue processing the received data). For example, refer to Figure 1 and Figure 3A Storage controller 110 detects a potential ransomware attack by determining that the calculated ratio exceeds the average of a set of ratio values ​​in ratio list 310 (e.g., ratio values ​​for all storage, ten most recent storage ratio values, twenty most recent storage ratio values, etc.). In another example, refer to... Figure 1 and Figure 3BStorage controller 110 detects a potential ransomware attack by determining that the calculated ratio exceeds the value of the rolling average field 320 by at least a specified percentage (e.g., 10%).

[0051] Frame 450 (e.g.) Figure 4 (As shown) may include identifying backup(s) associated with the container index. Box 460 may include providing indications that the backup(s) may be affected by ransomware. Following box 460, process 400 may be completed. For example, refer to... Figure 1 Storage controller 110 identifies multiple portions of data stream 105 associated with the current container index 160 and initiates an alert indicating that the identified multiple portions are suspected of being encrypted by ransomware. Storage controller 110 may initiate multiple other actions in response to the detection of a potential ransomware attack (e.g., stopping the backup process, initiating a logging process, etc.). See below for reference. Figure 5 Describe an example process for identifying potentially affected specific portions of data stream 105.

[0052] Figure 5 - Example procedure for identifying affected backup parts

[0053] Figure 5 An example process 500 for identifying affected backup portions according to some embodiments is shown. Process 500 can illustrate block 450 (e.g. Figure 4 The example extension is shown below. For illustrative purposes, reference may be made to examples illustrating some implementations. Figures 1 to 3B The details of process 500 are described. However, other implementations are also possible. In some examples, storage controller 110 can be used ( Figure 1 Process 500 is executed as shown. Process 500 can be implemented in hardware or a combination of hardware and programming (e.g., machine-readable instructions executable by (multiple) processors). Machine-readable instructions can be stored in a non-transitory computer-readable medium such as an optical storage device, a semiconductor storage device, or a magnetic storage device. Machine-readable instructions can be executed by a single processor, multiple processors, a single processing engine, multiple processing engines, etc.

[0054] Box 510 may include accessing a list of inventories stored in the container index to identify a set of affected inventories. Box 520 may include accessing said set of affected inventories to identify a set of backup items. Box 530 may include accessing the item metadata of said set of backup items to determine the offset of the affected data unit. Box 540 may include identifying the affected backup portion based on said set of backup items and the determined offset. After box 540, process 500 may be completed.

[0055] For example, refer to Figures 1 to 2Storage controller 110 determines a current deduplication ratio violation condition (e.g., match operation 160 against the first container index) based on local ratio history 224 stored in the first container index 160. In response, storage controller 110 reads list 222 (stored in the first container index 160) to identify a set of lists 150 referenced by the first container index 160. Storage controller 110 loads each identified list 150 into memory 115 and uses the loaded list 150 to identify data portions referencing the first container index 160. For each loaded list 150, storage controller 110 identifies a specific backup item 145 (e.g., using an identifier for list 150) and loads the corresponding backup item metadata 202 into memory 115. Storage controller 110 uses the backup item metadata 202 to determine the offset of the list within backup item 145 and then determines the offset of a data unit within list 150 relative to backup item 145. Storage controller 110 then provides an alert or indication to identify the affected backup portion using the identifier of backup item(s) 145 and the determined offset.

[0056] Figure 6 -Example computing device

[0057] Figure 6 A schematic diagram of an example computing device 600 is shown. In some examples, computing device 600 may substantially correspond to storage system 100 (e.g., Figure 1 (As shown). As illustrated, computing device 600 may include some or all of the following: hardware processor 602, memory 604, and machine-readable storage device 605 including instructions 610-640. Machine-readable storage device 605 may be a non-transitory medium. Instructions 610-640 may be executed by hardware processor 602 or by a processing engine included in hardware processor 602.

[0058] Instruction 610 can be executed to receive data units of a backup stream to be stored in persistent storage of a deduplication storage system. Instruction 620 can be executed to perform a first matching operation for matching data units against a first container index among a plurality of container indexes, wherein the plurality of container indexes include metadata indicating the storage location of data units previously stored in persistent storage. For example, refer to Figure 1Storage controller 110 receives a data stream 105 to be stored in deduplication storage system 100. Storage controller 110 performs a matching operation by generating fingerprints for a set of data units in data stream 105 and comparing the generated fingerprints with fingerprints included in container index 160. If a match is identified, storage controller 110 determines that storage system 100 has already stored a copy of the incoming data unit. In response to this determination, storage controller 110 stores a reference to the previous data unit (e.g., in Listing 150) in a deduplicated form, instead of storing a new copy of the incoming data unit.

[0059] Instruction 630 can be executed to calculate a first ratio, the first ratio indicating the amount of deduplication that occurred during a first matching operation against a first container index. Instruction 640 can be executed to identify at least a portion of the backup stream as potentially affected by a ransomware attack in response to determining that the first ratio violates a condition of local ratio history data regarding the first container index. For example, refer to... Figure 1 The storage controller 110 calculates the ratio value used for the matching operation as the number of new data units divided by the number of matched data units. Further, for example, referring to... Figures 1 to 3A Storage controller 110 detects a potential ransomware attack when it determines that the calculated ratio exceeds the average of a set of ratio values ​​in ratio list 310. Further, storage controller 110 uses metadata (e.g., manifest list 222 in container index 160, one or more manifests 150, one or more backup items 145, etc.) to identify portions of data stream 105 suspected of being encrypted by ransomware. Alerts (e.g., to users, applications, etc.) can be provided to identify affected backup portions (e.g., specific locations within data stream 105).

[0060] Figure 7 -Example machine-readable media

[0061] Figure 7 A machine-readable medium 700 storing instructions 710-740 according to some embodiments is shown. Instructions 710-740 can be executed by a single processor, multiple processors, a single processing engine, multiple processing engines, etc. The machine-readable medium 700 can be a non-transitory storage medium, such as an optical storage medium, a semiconductor storage medium, or a magnetic storage medium.

[0062] Instruction 710 can be executed to receive data units of a backup stream to be stored in a persistent storage device of a deduplication storage system. Instruction 720 can be executed to perform a first matching operation for matching data units against a first container index among a plurality of container indexes, wherein the plurality of container indexes include metadata indicating the storage location of data units previously stored in the persistent storage device.

[0063] Instruction 730 can be executed to calculate a first ratio, the first ratio indicating the amount of deduplication that occurred during a first matching operation against the first container index. Instruction 740 can be executed to identify at least a portion of the backup stream as potentially affected by a ransomware attack in response to determining that the first ratio violates a condition of local ratio history data regarding the first container index.

[0064] Figure 8 - Example procedures for ransomware detection

[0065] Figure 8 An example process 800 for ransomware detection according to some implementations is shown. In some examples, a storage controller 110 can be used. Figure 1 Process 800 is executed as shown. Process 800 can be implemented in hardware or a combination of hardware and programming (e.g., machine-readable instructions executable by (multiple) processors). Machine-readable instructions can be stored in a non-transitory computer-readable medium such as optical storage devices, semiconductor storage devices, or magnetic storage devices. Machine-readable instructions can be executed by a single processor, multiple processors, a single processing engine, multiple processing engines, etc.

[0066] Box 810 may include data units of a backup stream to be stored in the persistent storage of the deduplication storage system, received by the storage controller of the deduplication storage system. Box 820 may include a first matching operation performed by the storage controller for matching data units against a first container index of a plurality of container indexes, wherein the plurality of container indexes include metadata indicating the storage location of data units previously stored in the persistent storage.

[0067] Box 830 may include a first ratio calculated by the storage controller, the first ratio indicating the amount of deduplication that occurred during a first match operation against the first container index. Box 840 may include the storage controller determining whether the first ratio violates conditions of local ratio history data regarding the first container index. Box 850 may include, in response to determining that the first ratio violates conditions of local ratio history data regarding the first container index, the storage controller identifying at least a portion of the backup stream as potentially affected by a ransomware attack. Following box 850, process 800 may be completed.

[0068] According to the embodiments described herein, the controller of a deduplication storage system can perform a matching operation against a container index to deduplicate a set of data units from a backup data stream. The controller can calculate a ratio indicating the amount of deduplication that occurred during the matching operation. Further, the controller can determine whether the calculated ratio violates conditions regarding local ratio history data stored in the container index. If so, the controller can identify the set of data units as potentially encrypted by ransomware. In this way, some embodiments can provide detection of ransomware attacks without executing dedicated programs that may consume significant computational resources. Furthermore, some embodiments can provide rapid identification of specific affected data locations and can reduce the amount of valuable data lost due to malicious encryption. References below... Figures 1 to 8 The disclosed techniques for detecting ransomware will be discussed further.

[0069] It should be noted that, although Figures 1 to 8 Various examples are shown, but the implementation methods are not limited in this respect. For example, see references... Figure 1 It is envisioned that storage system 100 may include additional devices and / or components, fewer components, different components, different arrangements, etc. In another example, it is envisioned that the functionality of the aforementioned storage controller 110 may be included in any other engine or software of storage system 100. Other combinations and / or variations are also possible.

[0070] Data and instructions are stored in a corresponding storage device implemented as one or more computer-readable or machine-readable storage media. Storage media include various forms of non-transitory memory, including: semiconductor memory devices such as dynamic random access memory or static random access memory (DRAM or SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory; magnetic disks such as fixed floppy disks and removable disks; other magnetic media, including magnetic tape; optical media such as optical discs (CDs) or digital video discs (DVDs); or other types of storage devices.

[0071] Note that the instructions discussed above may be provided on a single computer-readable or machine-readable storage medium, or alternatively, on multiple computer-readable or machine-readable storage media distributed across a large system with potentially multiple nodes. Such one or more computer-readable or machine-readable storage media are considered part of an article (or article of manufacture). An article or article of manufacture may refer to any single or multiple manufactured components. The one or more storage media may be located in a machine that executes the machine-readable instructions, or at a remote site from which the machine-readable instructions can be downloaded for execution via a network.

[0072] In the foregoing description, numerous details have been set forth to facilitate understanding of the subject matter disclosed herein. However, embodiments may be practiced without some of these details. Other embodiments may include modifications and variations of the details discussed above. The appended claims are intended to cover such modifications and variations.

Claims

1. A computing device, comprising: processor; Memory; as well as A machine-readable storage device storing instructions that can be executed by the processor to perform the following operations: Receive data units of the backup stream to be stored in the persistent storage device of the deduplication storage system; Perform a first matching operation to match the data unit against a first container index among a plurality of container indexes, wherein the plurality of container indexes include metadata indicating the storage location of the data unit previously stored in the persistent storage device; Calculate a first deduplication ratio, which indicates the amount of deduplication that occurred during the first match operation against the first container index; and Based on the determination that the first deduplication ratio violates local ratio history data regarding the first container index, at least a portion of the backup stream associated with the first container index is identified as potentially affected by a ransomware attack, wherein the local ratio history data represents the deduplication ratio of a previous matching operation against the first container index.

2. The computing device as claimed in claim 1, wherein, The local ratio history data of the first container index includes a list structure for storing multiple stored ratio values, wherein each stored ratio value corresponds to a different matching operation of the first container index.

3. The computing device as claimed in claim 2, wherein, The list structure is used to store a maximum of N latest ratio values, where N is a positive integer.

4. The computing device as claimed in claim 2, wherein, The list structure is used to store an unlimited number of ratio values.

5. The computing device as claimed in claim 2, wherein, Determining that the first deduplication ratio violates the condition includes determining that the first deduplication ratio exceeds the average of a set of stored ratio values ​​in the list structure, wherein the set includes a predefined number M of the latest stored ratio values, and wherein M is a positive integer.

6. The computing device as claimed in claim 1, wherein, The local ratio history data for the first container index includes a field for storing a rolling average of the ratio values ​​of a set of the latest matching operations for the first container index.

7. The computing device of claim 6, wherein, Determining that the first deduplication rate violates the condition includes determining that the first deduplication rate exceeds the stored rolling average by at least a predefined percentage.

8. The computing device of claim 1, further comprising instructions executable by the processor to perform the following operations: The at least portion of the backup stream is identified based on a list of manifests stored in the first container index, wherein, The list of lists is a data structure used to identify each list that references the index of the first container.

9. The computing device of claim 8, further comprising instructions executable by the processor to perform the following operations: Access the list of inventories stored in the first container index to identify a set of affected inventories; Access the list of affected items to identify a set of backup projects; Access the project metadata of the set of backup projects to determine the offset of the affected data units; as well as The at least one affected portion is identified based on the set of backup items and the determined offset.

10. A method for detecting ransomware, comprising: The storage controller of the deduplication storage system receives data units of the backup stream to be stored in the persistent storage device of the deduplication storage system; The storage controller performs a first matching operation to match the data unit against a first container index among a plurality of container indexes, wherein the plurality of container indexes include metadata indicating the storage location of the data unit previously stored in the persistent storage device; The storage controller calculates a first ratio, which indicates the amount of deduplication that occurred during the first match operation for the first container index; The storage controller determines whether the first ratio violates conditions related to local ratio history data for the first container index; and Based on the determination that the first ratio violates local ratio history data regarding the first container index, the storage controller identifies at least a portion of the backup stream associated with the first container index as potentially affected by a ransomware attack, wherein the local ratio history data represents the deduplication ratio of previous matching operations against the first container index.

11. The method of claim 10, wherein, The local ratio history data of the first container index includes a list structure for storing multiple stored ratio values, wherein each stored ratio value corresponds to a different matching operation of the first container index.

12. The method of claim 11, wherein, Determining that the first ratio violates the condition includes determining that the first ratio exceeds the average of a set of stored ratio values ​​in the list structure, wherein the set includes a predefined number M of the latest stored ratio values, and wherein M is a positive integer.

13. The method of claim 10, wherein, The local ratio history data for the first container index includes a field for storing a rolling average of the ratio values ​​of a set of the latest matching operations for the first container index.

14. The method of claim 13, wherein, Determining that the first ratio violates the condition includes determining that the first ratio exceeds the stored rolling average by at least a predefined percentage.

15. The method of claim 10, further comprising: Access the list of lists stored in the first container index to identify a set of affected lists, wherein the list of lists is a data structure used to identify each list that references the first container index; Access the list of affected items to identify a set of backup projects; Access the project metadata of the set of backup projects to determine the offset of the affected data units; and The at least one affected portion is identified based on the set of backup items and the determined offset.

16. A non-transitory machine-readable medium storing instructions that, when executed, cause a processor to perform the following operations: Receive data units of the backup stream to be stored in the persistent storage device of the deduplication storage system; Perform a first matching operation to match the data unit against a first container index among a plurality of container indices, wherein, The plurality of container indexes include metadata indicating the storage location of data units previously stored in the persistent storage device; Calculate a first ratio, which indicates the amount of deduplication that occurred during the first match operation against the first container index; as well as Based on the determination that the first ratio violates local ratio history data regarding the first container index, at least a portion of the backup stream associated with the first container index is identified as potentially affected by a ransomware attack, wherein the local ratio history data represents the deduplication ratio of previous matching operations against the first container index.

17. The non-transitory machine-readable medium of claim 16, wherein, The local ratio history data of the first container index includes a list structure for storing multiple stored ratio values, wherein each stored ratio value corresponds to a different matching operation of the first container index.

18. The non-transitory machine-readable medium of claim 17, wherein, Determining that the first ratio violates the condition includes determining that the first ratio exceeds the average of a set of stored ratio values ​​in the list structure, wherein the set includes a predefined number M of the latest stored ratio values, and wherein M is a positive integer.

19. The non-transitory machine-readable medium of claim 16, wherein, The local ratio history data of the first container index includes a field for storing a rolling average of the ratio values ​​of a set of the latest matching operations of the first container index, and wherein determining that the first ratio violates the condition includes determining that the first ratio exceeds the stored rolling average by at least a predefined percentage.

20. The non-transitory machine-readable medium of claim 16, comprising instructions that, when executed, cause the processor to perform the following operations: Access the list of manifests stored in the first container index to identify a set of affected manifests, wherein, The list of lists is a data structure used to identify each list that references the index of the first container; Access the list of affected items to identify a set of backup projects; Access the project metadata of the set of backup projects to determine the offset of the affected data units; as well as The at least one affected portion is identified based on the set of backup items and the determined offset.

Images