Data leakage detection

Abstract

Systems and methods for data leakage detection based on snapshot analysis are described. The described systems and methods identify multiple snapshots of a data structure, identify multiple leaf nodes of the data structure for each snapshot, generate a vector of data attributes for each leaf node, assign a weight to each vector to produce a set of weighted vectors for each snapshot, calculate a distance metric between each pair of snapshots based on the corresponding set of weighted vectors, and detect anomalous snapshots from the multiple snapshots based on the distance metric.

Description

Background Art

[0001] The following relates generally to data breach detection, and more particularly to data breach detection based on snapshot analysis.

[0002] As individuals and corporate entities become more dependent on data, they also become vulnerable to attacks targeting that data. Cyber ​​resilience is the ability of an individual or organization to continue functioning with minimal disruption in the face of a cyber attack. Cyber ​​resilience can include information security, business continuity, and cyber resilience of an enterprise to ensure that the organization continues to function during cyber attacks and network outages.

[0003] In some cases, cyber attacks may be conducted in a manner that makes them difficult to detect.Therefore, there is a need in the art for systems and methods that provide improved capabilities for detecting complex cyber attacks. Summary of the invention

[0004] Methods, apparatus, and non-transitory computer-readable media for data leakage detection based on snapshot analysis are described. The method, apparatus, and non-transitory computer-readable media may provide for identifying a plurality of snapshots of a data structure, identifying a plurality of leaf nodes of the data structure for each snapshot, generating a vector of data attributes for each leaf node, assigning a weight to each vector to generate a set of weighted vectors for each snapshot, calculating a distance metric between each pair of snapshots based on the corresponding set of weighted vectors, and detecting an abnormal snapshot among the plurality of snapshots based on the distance metric. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Preferred embodiments of the present invention will now be described, by way of example only, with reference to the following drawings:

[0006] Figure 1 An example of an abnormal snapshot detection system according to an embodiment of the present disclosure is shown.

[0007] Figure 2 An example of a server according to an embodiment of the present disclosure is shown.

[0008] Figure 3 An example of a series of snapshots according to an embodiment of the present disclosure is shown.

[0009] Figure 4 An example of a data structure according to an embodiment of the present disclosure is shown.

[0010] Figures 5 to 7 An example of a process for a data leakage detection method according to an embodiment of the present disclosure is shown, in which a snapshot of a backup file is obtained and compared with surrounding files. DETAILED DESCRIPTION

[0011] The present disclosure provides a system and method for data leakage detection. Embodiments of the present disclosure relate to a data leakage detection method, in which a snapshot of a backup file is taken and compared with surrounding files to detect any anomalies in the file structure outside of a normal change pattern.

[0012] Embodiments of the present disclosure use a weighted tree structure to generate discretely distributed file attributes in backup snapshots (i.e., snapshot signatures), the weighted tree structure comprising a set of weight vectors. Then, a distance metric measured between two sets of weight vectors is calculated between two snapshot signatures. Abnormal snapshots are detected by comparing the local density of a snapshot with the density of snapshots of adjacent data groups.

[0013] Cyberattacks are on the rise, with phishing and social engineering, malware, spear phishing, denial of service, and outdated software being the top cyber disruptions threatening organizations today. Key cyber resilience concerns include: lack of ability to remain resilient in the wake of a cyberattack, inadequate planning and preparation as the top barriers to cyber resilience, and ad hoc, non-existent, or inconsistent cybersecurity incident response plans.

[0014] As cyberattacks become more common and costly, cyber resilience challenges become more difficult. Cyber ​​espionage and ransomware become prevalent, with advanced, targeted, and persistent threats targeting many organizations. As networks and infrastructure become more complex, the landscape of attacks continues to expand.

[0015] Attacks such as malware that destroy backup files cause many problems, and the recovery process takes a lot of time (especially when manual actions are required to resolve these issues). In addition, more and more regulations impose requirements on organizations that require expensive compliance programs. For example, data breach notification requirements contribute to challenges in the digital world.

[0016] Various data breach detection methods can be used to determine if data has been compromised. These detection methods can include techniques for detecting anomalous backup snapshots that can indicate a cyber threat or a breach in data security. For example, many data breach detection methods use single vector anomaly detection methods. Density-based anomaly detection methods use vector approximation to determine statistical outliers. These methods compare the density around a point with the density around the local vicinity of the point. This method assumes that the density around a normal data object has a similar density to neighboring data objects.

[0017] Clustering-based anomaly detection methods arrange similar objects into groups. These methods assume that anomaly instances are located in sparse areas, in small clusters, far away from the main cluster center, or they are not assigned to a cluster at all. Support vector machine-based anomaly detection methods can provide nonlinear classification of data through kernel functions.

[0018] However, in many cases, the assumptions of these detection methods may not apply to complex hacking attempts that make complex changes to data. Therefore, the present disclosure provides systems and methods for detecting data leaks that are capable of detecting complex hacking attempts.

[0019] Figure 1 An example of an abnormal snapshot detection system 100 according to an embodiment of the present disclosure is shown. The example shown includes a server 105, a database 110, a network 115, and an attacker terminal 120.

[0020] The server 100 may communicate with the database 110 via the network 115 to provide data leakage detection services. For example, the server 100 may obtain periodic snapshots of the database 110 in order to detect abnormal changes in the data. If the attacker terminal 120 gains access to the database (e.g., via the network 115), it may change some data in the database 110. Then one of the backup snapshots obtained by the server 100 may be identified as abnormal, and a response may be initiated to mitigate the impact of the attack.

[0021] Figure 2 An example of a server 200 according to an embodiment of the present disclosure is shown. The server 200 may be a reference Figure 1 The server 200 may include a processor unit 205 , a memory unit 210 , a snapshot component 215 , a data structure component 220 , a vector component 225 , a weight component 230 , a distance component 235 , and a detection component 240 .

[0022] The processor unit 205 may include an intelligent hardware device (e.g., a general processing component, a digital signal processor (DSP), a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof). In some cases, the processor may be configured to operate the memory array using a memory controller. In other cases, the memory controller may be integrated into the processor. The processor may be configured to execute computer-readable instructions stored in the memory to perform various functions.

[0023] The memory unit 210 may store information of various programs and applications on the computing device. For example, the storage device may include data for running an operating system. The memory may include both volatile memory and non-volatile memory. The volatile memory may be a random access memory (RAM), and the non-volatile memory may include a read-only memory (ROM), a flash memory, an electrically erasable programmable read-only memory (EEPROM), a digital tape, a hard disk drive (HDD), and a solid-state drive (SSD).

[0024] Snapshot component 215 can identify a set of snapshots of the data structure. In some examples, each snapshot in the set of snapshots corresponds to a backup of the data structure at a different point in time.

[0025] The data structure component 220 may identify a data structure for each snapshot, including a set of leaf nodes. For example, the data structure component 220 may identify a tree structure, wherein the leaf nodes of the data structure correspond to the bottom layer of the tree structure. In some examples, the tree structure corresponds to a file directory of the data structure.

[0026] The vector component 225 can generate a vector of data attributes for each leaf node. In some examples, each vector includes a value corresponding to a path depth attribute, a file size, a file count, a file extension attribute, a file modification attribute, or any combination thereof.

[0027] The weight component 230 can assign a weight to each vector to generate a set of weighted vectors for each snapshot. In some examples, the weight assigned to each vector corresponds to the number of files associated with the corresponding leaf node. In some examples, the weight assigned to each of the vectors corresponds to the network security risk associated with the corresponding leaf node.

[0028] The distance component 235 can calculate a distance metric between each pair of snapshots based on the weighted vectors of the corresponding group. For example, the distance component 235 can calculate a set of connection weights, wherein each connection weight includes a first index corresponding to a first leaf node of a first snapshot and a second index corresponding to a second leaf node of a second snapshot, wherein a sum of the connection weights with the first index is equal to the weight assigned to the first leaf node, and a sum of the connection weights with the second index is equal to the weight assigned to the second leaf node, and a sum of the set of connection weights is equal to one.

[0029] The distance component 235 may then multiply each connection weight by the square of the distance between the vector corresponding to the first leaf node and the vector corresponding to the second leaf node to produce a set of weighted distances. The distance component 235 may calculate a distance metric based on the sum of the weighted distances, wherein a set of connection weights is calculated to minimize the sum. In some examples, the distance metric includes an earth mover's distance, a Kantorovich-Mallows distance, a Wasserstein distance, or any combination thereof.

[0030] The detection component 240 detects an abnormal snapshot in a group of snapshots based on a distance metric. The detection component 240 may also calculate a local reachability density of each snapshot based on the calculated distance metric. The detection component 240 may also determine whether the local reachability density of each snapshot is below a threshold based on neighboring snapshots, wherein the abnormal snapshot is identified based on the determination.

[0031] Figure 3 An example of a series of snapshots 300 according to an embodiment of the present disclosure is shown. The example shown includes snapshots 300 and exception snapshots 305. Each snapshot 300 captures the state of the database at a specific moment in time.

[0032] By comparing different snapshots 300, the systems and methods described herein can be used to detect anomalous snapshots 305. When an anomalous snapshot 305 is detected, the system can issue a warning notifying the user that a data breach may have occurred.

[0033] Figure 4 An example of a data structure according to an embodiment of the present disclosure is shown. In the example shown, the data structure corresponds to a directory tree 400, which may include one or more nodes 405 and leaf nodes 410. Each node 405 may be associated with one or more child nodes. The bottom layer of the directory tree 400 includes leaf nodes 410.

[0034] According to an embodiment of the present disclosure, each leaf node can be associated with a vector including various attributes, such as a path depth attribute, a file size, a file count, a file extension attribute, a file modification attribute, or any combination thereof. These vectors change with each snapshot, and the combination of the vectors represents the signature of the snapshot (i.e., the signature of the data structure at a particular moment). By comparing the signatures of statistical anomalies, abnormal snapshots can be detected. Specifically, an embodiment of the present disclosure describes a system for detecting abnormal snapshots based on comparing snapshot signatures, which snapshot signatures include multiple vectors corresponding to leaf nodes.

[0035] Each leaf node may also be associated with a weight based on the number of files it includes (or a normalized number of files, the size of the files, or a similar value). In other examples, leaf nodes 410 may be assigned weights by a user to represent the likelihood that the node will be targeted by a cyber attack. That is, certain directories may be more susceptible to ransomware attacks and may therefore be weighted more heavily.

[0036] Figure 5 An example of a process for a data leakage detection method according to an embodiment of the present disclosure is shown, wherein a snapshot of a backup file is taken and compared with surrounding files. In some examples, these operations can be performed by a processor that executes a set of codes to control the functional elements of the device. In addition or alternatively, the process can be performed using dedicated hardware. Generally, these operations can be performed according to the methods and processes described in the embodiments of the present disclosure. For example, the operation may include various sub-steps, or may be performed in combination with other operations described herein.

[0037] At step 500, the system identifies a set of snapshots of the data structure. In some cases, the operation of this step may refer to the example of reference Figure 2 The described snapshot component performs or is performed by the snapshot component. Each snapshot in the set of snapshots may correspond to a backup of the data structure at a different point in time.

[0038] At step 505, the system identifies a set of leaf nodes of the data structure for each snapshot. In some cases, the operation of this step may refer to Figure 2 The described data structure component performs or is performed by the data structure component. In some embodiments, the system identifies a tree structure on the data structure, wherein the leaf nodes of the data structure correspond to the bottom layer of the tree structure. The tree structure may correspond to a file directory of the data structure.

[0039] In step 510, the system generates a data attribute vector for each leaf node. In some cases, the operation of this step can refer to the reference Figure 2 The described vector components are performed or executed by the vector components. Each vector may include values ​​corresponding to a path depth attribute, a file size, a file count, a file extension attribute, a file modification attribute, or any combination thereof. In some cases, the generic values ​​may be converted to numeric or binary values ​​(e.g., using one hot encoding).

[0040] At step 515, the system assigns weights to each vector to generate a set of weighted vectors for each snapshot. In some cases, the operation of this step may refer to Figure 2The described weight component performs or is performed by the weight component, and the weight assigned to each vector can correspond to a number of files associated with the corresponding leaf node. Additionally or alternatively, the weight assigned to each vector in the vector can also correspond to the network security risk associated with the corresponding leaf node.

[0041] In some embodiments, the system calculates a set of connection weights, wherein each of the connection weights includes a first index corresponding to a first leaf node of a first snapshot and a second index corresponding to a second leaf node of a second snapshot, wherein a sum of the connection weights with the first index is equal to the weight assigned to the first leaf node, and a sum of the connection weights with the second index is equal to the weight assigned to the second leaf node, and the sum of the set of connection weights is equal to one. Additionally, the first leaf node and the vector corresponding to the second leaf node produce a set of weighted distances, and the system then calculates a distance metric based on the sum of the weighted distances, wherein the set of connection weights is calculated to minimize the sum.

[0042] At step 520, the system calculates a distance metric between each pair of snapshots based on the weighted vectors of the corresponding groups. In some cases, the operation of this step may refer to Figure 2 The described distance component is performed or performed by a reference distance component, and the distance metric may include a bulldozer distance, a Kantorovich-Mallows distance, a Wasserstein distance, or any combination thereof.

[0043] In step 525, the system detects abnormal snapshots in a group of snapshots based on the distance metric. In some cases, the operation of this step can refer to the reference Figure 2 The described detection component performs or is performed by the detection component, for example, the system can calculate the local reachability density of each snapshot in the snapshots based on the calculated distance metric; and determine whether the local density of each snapshot in the snapshots is lower than a threshold based on adjacent snapshots, wherein abnormal snapshots are identified based on the determination.

[0044] Figure 6 An example of a process for a data leakage detection method according to an embodiment of the present disclosure is shown, wherein a snapshot of a backup file is taken and compared with surrounding files. In some examples, these operations can be performed by a processor that executes a set of codes to control the functional elements of the device. In addition or alternatively, the process can be performed using dedicated hardware. Generally, these operations can be performed according to the methods and processes described in the embodiments of the present disclosure. For example, the operation may include various sub-steps, or may be performed in combination with other operations described herein.

[0045] In step 600, the system identifies a set of snapshots of the data structure. In some cases, the operation of this step can refer to the reference Figure 2The described snapshot component is performed by or is executed by the snapshot component, and the system provides a method or system for improved detection of anomalous backup snapshots based on discretely distributed data that captures the statistical distribution of changes in user data.

[0046] At step 605, the system identifies a set of leaf nodes of the data structure for each snapshot. In some cases, the operation of this step may refer to the following. Figure 2 The described data structure component performs or is performed by the data structure component. For example, in some cases, the system can identify a set of leaf nodes for each snapshot within a specified time period. In some cases, the system can identify multiple sets of snapshots containing multiple leaf nodes.

[0047] In step 610, the system generates a data attribute vector for each leaf node. In some cases, the operation of this step can refer to the reference Figure 2 The described vector component performs or is performed by the vector component, which generates a weighted tree structure including a set of weighted vectors.

[0048] In step 615, the system assigns weights to each vector to generate a set of weighted vectors for each snapshot. In some cases, the operation of this step may refer to the reference Figure 2 The described weight component is executed or performed by the weight component, and the node weights can be assigned by the user as needed.

[0049] Various methods can be used to determine leaf node weights. In a first example (i.e., Type I), several files at each unique file path can be used. This can produce a vector such as [10, 5, 15, 4, 16, 20, 12, 8, 3, 7]. In a second example (i.e., Type II), the weights can represent cybersecurity risk values. This can produce vectors such as [0.1, 0.05, 0.15, 0.04, 0.16, 0.2, 0.12, 0.08, 0.03, 0.07]. In some examples, a Type II weighting mechanism can produce a normalized set of weights (i.e., where the weights sum to 1).

[0050] At step 620, the system calculates a set of connection weights, wherein each connection weight includes a first index corresponding to a first leaf node of the first snapshot and a second index corresponding to a second leaf node of the second snapshot, wherein a sum of the connection weights with the first index is equal to the weight assigned to the first leaf node, and a sum of the connection weights with the second index is equal to the weight assigned to the second leaf node, and a sum of the set of connection weights is equal to one. In some cases, the operation of this step can refer to reference Figure 2 The described distance component performs or is performed by the distance component.

[0051] The first leaf node and the vector corresponding to the second leaf node generate a set of weighted distances, and a distance metric is calculated based on a sum of the weighted distances, and the set of connection weights may be calculated to minimize the sum.

[0052] At step 625, the system multiplies each connection weight by the squared distance between the vector corresponding to the first leaf node and the vector corresponding to the second leaf node to produce a set of weighted distances. In some cases, the operation of this step can refer to the reference Figure 2 The described distance component performs or is performed by the distance component.

[0053] At step 630, the system calculates a distance metric based on a sum of weighted distances, wherein a set of connection weights is calculated to minimize the sum. In some cases, the operation of this step may refer to reference Figure 2 The described distance component performs or is performed by the distance component. The distance metric (ie, the Mallow / Wasserstein distance) is computed using two snapshot signatures; also referred to as two sets of weight vectors.

[0054] In step 635, the system detects abnormal snapshots in a group of snapshots based on the distance metric. In some cases, the operation of this step can refer to the reference Figure 2 The described detection component performs or is performed by the detection component to detect anomalous snapshots by comparing the local density of the snapshot with the density of its neighbors.

[0055] When the node weights are not normalized (i.e., Type I), the absolute number of files under each file path, the distance between two signatures can be defined according to the optimal routine described by F = (ωi, j):

[0056]

[0057] It is subject to the following conditions:

[0058]

[0059]

[0060]

[0061] ω i，j ≥0, i=1, ..., m1, j=1, ..., m2 (5)

[0062] Find the best After that, the square distance between γ1 and γ2 is defined as:

[0063]

[0064] γ1 and γ2 are backup snapshot signatures. and is the representation vector of leaf nodes i and j, and is the weight, m1 and m2 are the number of leaf nodes. The above distance is called the Earthmoving Distance (EMD).

[0065] When using Type II, the normalized value is calculated by dividing the number of documents in that node by the total number of documents. The squared distance between γ1 and γ2 is defined as:

[0066]

[0067] This equation is subject to the following constraints:

[0068]

[0069]

[0070]

[0071] ω i，j ≥0, i=1, ..., m1, j=1, ..., m2 (11)

[0072] The above distance is called Kantorovich-Mallows distance. Calculating the distance is to optimize the matching weights between the support vectors in the two distributions so that the aggregate distance is minimized. The matching weight ω i，j is restricted to be non-negative, and from any vector The weights emitted are summed until their probability therefore, Set from The amount of influence on the overall distribution distance.

[0073] Assumptions is the mallows distance between signatures γ1 and γ2 using the percentage of the number of documents as weight, and is the mallows distance between signatures γ1 and γ2 using the percentage of file size as weight, and the final distance

[0074] Figure 7An example of a process for a data leakage detection method according to an embodiment of the present disclosure is shown, wherein a snapshot of a backup file is taken and compared with surrounding files. In some examples, these operations can be performed by a processor that executes a set of codes to control the functional elements of the device. In addition or alternatively, the process can be performed using dedicated hardware. Generally, these operations can be performed according to the methods and processes described in the embodiments of the present disclosure. For example, the operation may include various sub-steps, or may be performed in combination with other operations described herein.

[0075] In step 700, the system identifies a set of snapshots of the data structure. In some cases, the operation of this step can refer to the reference Figure 2 The described snapshot component is performed or is performed by the snapshot component.

[0076] At step 705, the system identifies a set of leaf nodes of the data structure for each snapshot. In some cases, the operation of this step may refer to the following. Figure 2 The described data structure components are performed or performed by the data structure components.

[0077] In step 710, the system generates a data attribute vector for each leaf node. In some cases, the operation of this step can refer to the reference Figure 2 The described vector component performs or is performed by the vector component.

[0078] In step 715, the system assigns weights to each vector to generate a set of weighted vectors for each snapshot. In some cases, the operation of this step may refer to the reference Figure 2 The described weight component performs or is performed by the weight component.

[0079] In step 720, the system calculates a distance metric between each pair of snapshots based on the weighted vectors of the corresponding groups. In some cases, the operation of this step can refer to the reference Figure 2 The described distance component performs or is performed by the distance component.

[0080] At step 725, the system calculates the local reachability density of each of the snapshots based on the calculated distance metric. In some cases, the operation of this step may refer to reference Figure 2 The detection component described is performed or is performed by the detection component.

[0081] At step 730, the system determines whether the local reachability density of each snapshot compared to its neighbors is below a threshold, wherein an abnormal snapshot is identified based on the determination. In some cases, the operation of this step may refer to the reference Figure 2 The detection component described is performed or is performed by the detection component.

[0082] In step 735, the system detects abnormal snapshots in a set of snapshots based on the local reachability density. In some cases, the operation of this step can refer to reference Figure 2 The detection component described is performed or is performed by the detection component.

[0083] An example algorithm for identifying anomalies using local reachability density is provided below. The term “distance” refers to the distance defined previously, i.e. The anomaly detection algorithm is as follows. Let k-distance(A) be the distance from snapshot A to its kth nearest neighbor. Denote a set of k nearest neighbors of A as N K (A). Define the reachability distance between snapshot A and snapshot B as reachability-distance k (A,B)=max{k-distance(B), The local reachability density of snapshot A is defined as follows:

[0084]

[0085] It is the inverse of the average reachability distance between snapshot A and its neighbors. The local reachability density is then compared to the local reachability density of the neighbors using the following formula:

[0086]

[0087] It is the average local reachability density of the neighbors divided by the snapshot's own local reachability density. Values ​​around 1.0 indicate that the signature is comparable to its neighbors and is therefore not an outlier. Values ​​below 1.0 indicate denser areas that may be normal, while values ​​significantly greater than 1.0 indicate outliers.

[0088] Therefore, the present disclosure includes the following embodiments.

[0089] A method for data leakage detection based on snapshot analysis is described. The method may include identifying a plurality of snapshots of a data structure, identifying a plurality of leaf nodes of the data structure for each snapshot, generating a vector of data attributes for each leaf node, assigning a weight to each vector to produce a set of weighted vectors for each snapshot, calculating a distance metric between each pair of snapshots based on the corresponding set of weighted vectors, and detecting an anomalous snapshot among the plurality of snapshots based on the distance metric.

[0090] An apparatus for a data leak detection method is described, wherein a snapshot of a backup file is obtained and compared to surrounding files. The apparatus may include a processor, a memory in electronic communication with the processor, and instructions stored in the memory. The instructions are operable to cause the processor to identify a plurality of snapshots of a data structure, identify a plurality of leaf nodes of the data structure for each of the snapshots, generate a vector of data attributes for each of the leaf nodes, assign a weight to each of the vectors to generate a set of weighted vectors for each of the snapshots, calculate a distance metric between each pair of snapshots based on the corresponding set of weighted vectors, and detect an anomalous snapshot in the plurality of snapshots based on the distance metric.

[0091] A non-transitory computer-readable medium storing code for a data leak detection method is described, wherein a snapshot of a backup file is obtained and compared to surrounding files. In some examples, the code includes instructions executable by a processor to: identify a plurality of snapshots of a data structure, identify a plurality of leaf nodes of the data structure for each of the snapshots, generate a vector of data attributes for each of the leaf nodes, assign a weight to each of the vectors to generate a set of weighted vectors for each of the snapshots, calculate a distance metric between each pair of snapshots based on the corresponding set of weighted vectors, and detect an anomalous snapshot among the plurality of snapshots based on the distance metric.

[0092] In some examples, each of the plurality of snapshots corresponds to a backup of the data structure at a different point in time.Some examples of the above methods, apparatus, and non-transitory computer-readable media may also include identifying a tree structure on the data structure, wherein leaf nodes of the data structure correspond to bottom layers of the tree structure.

[0093] In some examples, the tree structure corresponds to a file directory of the data structure. In some examples, each vector includes a value corresponding to a path depth attribute, a file size, a file count, a file extension attribute, a file modification attribute, or any combination thereof. In some examples, a weight assigned to each vector corresponds to a number of files associated with a corresponding leaf node. In some examples, a weight assigned to each of the vectors corresponds to a network security risk associated with a corresponding leaf node.

[0094] Some examples of the above methods, apparatus, and non-transitory computer-readable media may also include calculating a set of connection weights, wherein each of the connection weights includes a first index corresponding to a first leaf node of a first snapshot and a second index corresponding to a second leaf node of a second snapshot, wherein the sum of the connection weights with the first index is equal to the weight assigned to the first leaf node, and the sum of the connection weights with the second index is equal to the weight assigned to the second leaf node, and the sum of the set of connection weights is equal to one. Some examples may also include multiplying each of the connection weights by the squared distance between the vector corresponding to the first leaf node and the vector corresponding to the second leaf node to produce a set of weighted distances. Some examples may also include calculating a distance metric based on the sum of the weighted distances, wherein the set of connection weights is calculated to minimize the sum. In some examples, the distance metric includes a bulldozer distance, a Kantorovich-Mallows distance, a Wasserstein distance, or any combination thereof.

[0095] Some examples of the above methods, apparatuses, and non-transitory computer-readable media may also include calculating a local reachability density for each of the snapshots based on the calculated distance metric. Some examples may also include determining whether the local reachability density for each of the snapshots is below a threshold based on neighboring snapshots, wherein the abnormal snapshot is identified based on the determination.

[0096] The descriptions and drawings described herein represent example configurations and do not represent all implementations within the scope of the claims. For example, operations and steps may be rearranged, combined, or otherwise modified. Moreover, structures and devices may be represented in the form of block diagrams to represent the relationships between components and avoid confusion about the concepts described. Similar components or features may have the same name, but may have different reference numbers corresponding to different drawings.

[0097] Some modifications to the present disclosure may be readily apparent to those skilled in the art, and the principles defined herein may be applied to other variations without departing from the scope of the present invention. Therefore, the present disclosure is not limited to the examples and designs described herein, but is intended to conform to the broadest scope consistent with the principles and novel features disclosed herein.

[0098] The described methods may be implemented or performed by a device including a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general purpose processor may be a microprocessor, a conventional processor, a controller, a microcontroller, or a state machine. The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors and a DSP core, or any other such configuration). Therefore, the functions described herein may be implemented in hardware or software and may be performed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored in a computer readable medium in the form of instructions or codes.

[0099] Computer-readable media include both non-transitory computer storage media and communication media, including any medium that facilitates the transmission of code or data. Non-transitory storage media can be any available medium that can be accessed by a computer. For example, non-transitory computer-readable media can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk (CD) or other optical disk storage, magnetic disk storage, or any other non-transitory medium for carrying or storing data or code.

[0100] In addition, a connecting component may be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (such as infrared, wireless, or microwave signals) is used to transmit code or data from a website, server, or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology is included in the definition of medium. Combinations of media are also included within the scope of computer-readable media.

[0101] In this disclosure and the following claims, the word "or" means an inclusive list, so that a list such as X, Y, or Z means X or Y or Z or XY or XZ or YZ or XYZ. The phrase "based on" is also not used to indicate a closed set of conditions. For example, a step described as "based on condition A" may be based on both condition A and condition B. In other words, the phrase "based on" should be interpreted to mean "based at least in part on".

Claims

1. A method for data leakage detection, comprising: Identify multiple snapshots of a data structure; identifying, for each of the snapshots, a plurality of leaf nodes of the data structure; generating a vector of data attributes for each of the leaf nodes; assigning a weight to each of the vectors to produce a set of weighted vectors for each of the snapshots; calculating a distance metric between each pair of the snapshots based on the weighted vectors of the corresponding groups; detecting abnormal snapshots among the plurality of snapshots based on the distance metric; calculating a set of connection weights, wherein each of the connection weights comprises a first index corresponding to a first leaf node of a first snapshot and a second index corresponding to a second leaf node of a second snapshot, wherein a sum of the connection weights having the first index is equal to a weight assigned to the first leaf node, and a sum of the connection weights having the second index is equal to a weight assigned to the second leaf node, and a sum of the set of connection weights is equal to one; multiplying each of the connection weights by a squared distance between a vector corresponding to the first leaf node and a vector corresponding to the second leaf node to produce a set of weighted distances; and The distance metric is calculated based on a sum of the weighted distances, wherein the set of connection weights is calculated to minimize the sum.

2. The method according to claim 1, wherein: Each of the plurality of snapshots corresponds to a backup of the data structure at a different point in time.

3. The method according to claim 1, further comprising: A tree structure is identified on the data structure, wherein the leaf nodes of the data structure correspond to a bottom layer of the tree structure.

4. The method according to claim 3, wherein: The tree structure corresponds to a file directory of the data structure.

5. The method according to claim 1, wherein: Each of the vectors includes a value corresponding to a path depth attribute, a file size, a file count, a file extension attribute, a file modification attribute, or any combination thereof.

6. The method according to any one of claims 1 to 5, wherein: The weight assigned to each of the vectors corresponds to the number of files associated with the corresponding leaf node.

7. The method according to any one of claims 1 to 5, wherein: The weight assigned to each of the vectors corresponds to a network security risk associated with the corresponding leaf node.

8. The method according to any one of claims 1 to 5, wherein: Distance metrics include bulldozer distance, Kantorovich-Mallows distance, Wasserstein distance, or any combination thereof.

9. The method according to any one of claims 1 to 5, further comprising: calculating a local reachability density for each of the snapshots based on the calculated distance metric; as well as A determination is made based on neighboring snapshots whether the local reachability density of each of the snapshots is below a threshold, wherein the abnormal snapshot is identified based on the determination.

10. An apparatus for a data leakage detection method, wherein a snapshot of a backup file is obtained and compared with surrounding files, the apparatus comprising: a processor and a memory storing instructions and in electronic communication with the processor, the processor being configured to execute the instructions to: Identify multiple snapshots of a data structure; identifying, for each of the snapshots, a plurality of leaf nodes of the data structure; generating a vector of data attributes for each of the leaf nodes; assigning a weight to each of the vectors to produce a set of weighted vectors for each of the snapshots; calculating a distance metric between each pair of the snapshots based on the weighted vectors of the corresponding groups; detecting abnormal snapshots among the plurality of snapshots based on the distance metric; calculating a set of connection weights, wherein each of the connection weights comprises a first index corresponding to a first leaf node of a first snapshot and a second index corresponding to a second leaf node of a second snapshot, wherein a sum of the connection weights having the first index is equal to a weight assigned to the first leaf node, and a sum of the connection weights having the second index is equal to a weight assigned to the second leaf node, and a sum of the set of connection weights is equal to one; multiplying each of the connection weights by a squared distance between a vector corresponding to the first leaf node and a vector corresponding to the second leaf node to produce a set of weighted distances; and The distance metric is calculated based on a sum of the weighted distances, wherein the set of connection weights is calculated to minimize the sum.

11. The device according to claim 10, wherein: Each of the plurality of snapshots corresponds to a backup of the data structure at a different point in time.

12. The apparatus of claim 10, wherein the processor is further configured to execute the instructions to: A tree structure is identified on the data structure, wherein the leaf nodes of the data structure correspond to a bottom layer of the tree structure.

13. The device according to claim 12, wherein: The tree structure corresponds to a file directory of the data structure.

14. The device according to claim 10, wherein: Each of the vectors includes a value corresponding to a path depth attribute, a file size, a file count, a file extension attribute, a file modification attribute, or any combination thereof.

15. The device according to any one of claims 10 to 14, wherein: The weight assigned to each of the vectors corresponds to the number of files associated with the corresponding leaf node.

16. The device according to any one of claims 10 to 14, wherein: The weight assigned to each of the vectors corresponds to a cybersecurity risk associated with the corresponding leaf node.

17. The device according to any one of claims 10 to 14, wherein: Distance metrics include bulldozer distance, Kantorovich-Mallows distance, Wasserstein distance, or any combination thereof.

18. The apparatus according to any one of claims 10 to 14, wherein the processor is further configured to execute the instructions to: calculating a local reachability density for each of the snapshots based on the calculated distance metric; and A determination is made based on neighboring snapshots whether the local reachability density of each of the snapshots is below a threshold, wherein the abnormal snapshot is identified based on the determination.

19. A non-transitory computer readable medium storing code for a data leak detection method in which a snapshot of a backup file is taken and the snapshot is compared with surrounding files, the code comprising instructions executable by a processor to: Identify multiple snapshots of a data structure; identifying, for each of the snapshots, a plurality of leaf nodes of the data structure; generating a vector of data attributes for each of the leaf nodes; assigning a weight to each of the vectors to produce a set of weighted vectors for each of the snapshots; calculating a distance metric between each pair of the snapshots based on the weighted vectors of the corresponding groups; detecting an abnormal snapshot among the plurality of snapshots based on the distance metric; calculating a set of connection weights, wherein each of the connection weights comprises a first index corresponding to a first leaf node of a first snapshot and a second index corresponding to a second leaf node of a second snapshot, wherein a sum of the connection weights having the first index is equal to a weight assigned to the first leaf node, and a sum of the connection weights having the second index is equal to a weight assigned to the second leaf node, and a sum of the set of connection weights is equal to one; multiplying each of the connection weights by a squared distance between a vector corresponding to the first leaf node and a vector corresponding to the second leaf node to produce a set of weighted distances; and The distance metric is calculated based on a sum of the weighted distances, wherein the set of connection weights is calculated to minimize the sum.

20. The non-transitory computer readable medium of claim 19, wherein: Each of the plurality of snapshots corresponds to a backup of the data structure at a different point in time.

21. The non-transitory computer readable medium of claim 19 or 20, the code further comprising instructions executable by the processor to: A tree structure is identified on the data structure, wherein the leaf nodes of the data structure correspond to a bottom layer of the tree structure.

22. The non-transitory computer readable medium of claim 19 or 20, the code further comprising instructions executable by the processor to: calculating a local reachability density for each of the snapshots based on the calculated distance metric; and A determination is made based on neighboring snapshots whether the local reachability density of each of the snapshots is below a threshold, wherein the abnormal snapshot is identified based on the determination.

23. A computer program product comprising program code means adapted to perform the method according to any one of claims 1 to 9 when said program is run on a computer.

Images