Rule-based document cleansing of sensitive data

Abstract

The rule-based attribute mechanism analyzes documents with different data types in different formats through the application of script-based rules that apply labels to documents to identify the type of sensitive data contained in the document. Documents with similar labels are aggregated so that sensitive data is cleansed from the documents, leaving telemetry data available for downstream processing. Cleansing requires different actions such as eliminating sensitive data, obfuscating sensitive data, and converting sensitive data to non-sensitive values.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 62 / 672,071, filed May 16, 2018. Background Art

[0003] Telemetry is the automatic recording and transmission of measurement data from a remote point. Telemetry is often used in software products to study how users engage with the product in real-world environments. Telemetry data provides valuable insights into the features that users use most often, how often users use the product, the settings and options that users select most often, how often the product crashes, the context in which users experience crashes using the product, bugs and issues, and the overall performance of the product. Essentially, telemetry data gives developers insight into how a software product performs in real-world environments. However, telemetry data may occasionally include sensitive information that is not relevant to the performance of the software product. Summary of the invention

[0004] This Summary is provided to introduce a selection of concepts in a simplified form, which will be further described in the Detailed Description below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] Rule-based attribution techniques are used to clean sensitive data from documents including telemetry data. Rule-based attribution techniques analyze documents with different types of data in different formats through the application of script-based rules that apply tags to documents to identify the type of personal data contained in the documents. Documents with similar tags are aggregated so that the identified personal data is cleaned in a similar manner, leaving the telemetry data available for downstream processing. Cleansing requires different actions, such as eliminating sensitive data, obfuscating sensitive data, and converting personal data to a common format.

[0006] These and other features and advantages will be apparent from a reading of the following detailed description and a review of the associated drawings.It is to be understood that both the foregoing general description and the following detailed description are illustrative only and are not restrictive of the claimed aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 An exemplary system that performs rule-based document cleansing of sensitive data is illustrated.

[0008] Figure 2A-2D is a schematic diagram illustrating an example of rule-based document cleansing.

[0009] Figure 3is a flow chart illustrating an exemplary method for using rules to cleanse sensitive data from a document.

[0010] Figure 4 is a flow chart illustrating an exemplary method for performing certain actions to sanitize sensitive data from a document.

[0011] Figure 5 is a block diagram illustrating an operating environment.

[0012] Figure 6 is a schematic diagram illustrating exemplary rules for group users.

[0013] Figure 7 A schematic diagram illustrating exemplary rules for homogenizing event data generated from three different products having different event formats is illustrated. DETAILED DESCRIPTION

[0014] Overview

[0015] Telemetry data is generated at different times and as different events occur during a user's engagement with a software product. To gain insight into a specific issue with a software product, several different pieces of telemetry data from different sources may need to be analyzed to understand the cause and impact of the issue. Telemetry data may exist in different documents, which may be in different formats and contain different fields and features, making it difficult to bring together all the data in a document that is needed to understand the issue. In some cases, telemetry data may include sensitive data that needs to be protected from unauthorized disclosure. Sensitive data can be any type of information, such as, but not limited to, confidential data and / or personal data.

[0016] To overcome this obstacle, the subject matter disclosed herein uses a rule-based attribute technique to identify sensitive data in a document that may need to be filtered, eliminated, or cleaned so that the potential telemetry data is retained for downstream processing. The rules account for the different types of documents in which the telemetry data resides and the different fields and / or features in which the data resides in the document.

[0017] In one aspect, a rule-based attribute mechanism can be used to determine whether personal data is present in telemetry data. The rule-based attribute mechanism analyzes documents with different types of data in different formats through the application of script-based rules that apply tags to documents to identify the type of personal data contained in the documents. Documents with similar tags are aggregated so that the identified personal data is cleaned, leaving the telemetry data available for downstream processing. Cleaning requires different actions, such as eliminating sensitive data, obfuscating sensitive data, and converting personal data to generic or non-personal values.

[0018] Note that we now turn to further discussion of systems, devices, components, and methods utilized in a system for cleansing personal data.

[0019] system

[0020] Figure 1 1 illustrates a block diagram of an exemplary system 100 in which various aspects of the present invention may be implemented. Figure 1 As shown, system 100 includes one or more client devices 102A through 102N (“ 102 ”) communicatively coupled to a telemetry processing center 106 via a network 104 .

[0021] The client device 102 may include one or more client applications 108A-108N ("108") and telemetry modules 110A-110N ("110"). The client application 108 is a software program that executes in the client device 102. There may be multiple client applications 108 in the client device 102. The client application 108 is coupled to a telemetry module 110. The telemetry module 110 may be part of the client application 108 or coupled to the client application 108 as an add-on, extension, plug-in, etc. The telemetry module 110 generates telemetry data by executing the client application 108. The telemetry data may be generated when certain events occur.

[0022] The occurrence of an event causes event data to be generated. The event may be caused by an action performed by the operating system based on the interaction of the user with the operating system, or by the interaction of the user with an application executed under the operating system (e.g., user input, button click, installation of a component or plug-in, uninstallation of a component, etc.). Event data may include system-generated logs generated by resources (such as software products) to solve problems and improve product quality. Event data may include data from crashes, hangs, unresponsive user interfaces, high CPU usage, high memory usage, etc. Event data may include memory dumps, stack traces, and exception information. When an event occurs, the event data is encapsulated into a document and transmitted to the telemetry processing center 106. The document may contain sensitive data.

[0023] The telemetry processing center 106 may be composed of a plurality of servers 107 and includes an attribute module 120, a data cleansing module 124, a telemetry data processing module 128, and a rolling hash storage device 130. The attribute module 120 receives a document 116 and a set of rules 118, the document 116 being generated from the telemetry module 110. The rules 118 indicate the characteristics or fields in the document that are considered sensitive data within different documents, the conditions under which the characteristics / fields become sensitive data, and the labels applied to the document. The attribute module 120 identifies the sensitive data defined by the rules 118 in each document 116 and attributes the document with the appropriate label.

[0024] The tagged document 122 is then processed by a data cleansing module 124, which cleans the sensitive data in an appropriate manner. The data cleansing module 124 can remove the sensitive data, convert the sensitive data to a general form, or obfuscate the sensitive data. The obfuscation can utilize a simple hashing technique or a more complex hashing technique called a rotating hash. The rotating hashing technique utilizes a rotating hash storage device 130. A cleaned document 126 is generated, which can be used by a telemetry data processing module 128. No sensitive data is present in the cleaned document 126. The telemetry data processing module 128 analyzes the telemetry data, which has no sensitive data embedded therein.

[0025] The client device 102 and the server 107 may be any type of electronic device capable of executing programmable instructions, such as, but not limited to, a mobile device, a personal digital assistant, a mobile computing device, a smart phone, a cellular phone, a handheld computer, a server, a server array or server farm, a web server, a network server, an Internet server, a workstation, a minicomputer, a mainframe computer, a supercomputer, a network tool, a web tool, a distributed computing system, an Internet of Things (IOT) device, a multiprocessor system, or a combination thereof. The network 104 may be any type of communication link capable of facilitating communication between the client device 102 and the server 107 using any communication protocol and in any configuration, such as, but not limited to, a wired network, a wireless network, or a combination thereof. It should be understood that the network connections shown are exemplary, and other methods of establishing a communication link between the client device 102 and the server 107 may also be used.

[0026] The client application 108, telemetry module 110, attribute module 120, data cleaning module 124, and telemetry data processing module 128 may be sequences of computer program instructions that, when executed by a processor, cause the processor to perform methods and / or operations according to specified tasks. The client application 108, telemetry module 110, attribute module 120, data cleaning module 124, and telemetry data processing module 128 may be implemented as program code, programs, steps, modules, code segments, program stacks, middleware, firmware, methods, routines, etc. Executable computer program instructions may be implemented according to a predetermined computer language, manner, or syntax to instruct a computer to perform a certain function. Instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0027] Figure 2A-2D An exemplary rule is illustrated. On the one hand, the rule can be implemented as a script file written in a markup language or an object notation scripting language (e.g., XML, JSON, etc.). The rule is applied to each document and the document is modified according to the tag applied to the document. The rule consists of a tag and a condition. The tag identifies the attribute applied to the document, which is used to identify the subsequent cleanup action performed on the sensitive data in the document. The condition indicates the situation in which the tag is applied. Therefore, the rule is composed of the tag to be applied to the document and the condition under which the rule is applied.

[0028] Figure 2A-2D The rules shown in have a common format. A rule contains a tag and a condition under which the tag is generated. A rule is associated with a specific event identified by an event name and an event field. The event name may represent the name of the event consistent with the document name. There may be multiple rules applied to a single document. Although each rule is shown as a separate file, multiple rules may be included in a single file. It should be noted that Figure 2A-2D The format of the rules shown in is exemplary and should not be construed as limiting the present disclosure.

[0029] Figure 2ARepresents exemplary rule 202, which labels the email address in the document with an attribute, which indicates that the email address is to be removed from the document. Line 0 defines the event definition (EventDefinition) spanning lines 0-20. Line 1 defines the event name (EventName), which is designtools / packageanddeploy. Line 2 defines the event field (EventField) to which the rule is applied, i.e., EventName. The rule consists of the tags (Tags) of lines 5-9 and the conditions (Conditions) of lines 10-17. The tag key (Tag Key) in line 6 defines the type of tag, which is SensitiveDataHandling (sensitive data processing) in this example. The value (Value) applied to the tag in line 7 is Drop (remove), which indicates that the identified sensitive data is to be removed or deleted from the document. The conditions in lines 10-17 indicate that the property remotedevice.emailaddress in the document will be tagged if remotedevice.emailaddress is not null. Then remotedevice.emailaddress is removed from the document 204 by the data cleaning module 124.

[0030] It is important to note that the Drop tag can be applied to other fields in the document. For example, the Drop tag can be applied to machine names, user names, and person names (such as Mary Smith).

[0031] Figure 2BRepresents exemplary rule 206, which indicates that a simple hash will be applied to the project identifier in the document. The project identifier is a unique identifier for a project used with an integrated development environment (IDE). Lines 21-41 define the event definition (EventDefinition). Line 22 defines the document name, i.e., workspace / search. Line 23 defines the event field (EventField) to which the rule is applied, i.e., EventName. The rule consists of tags (Tags) of lines 26-30 and conditions (Conditions) of lines 31-38. The tag key (Tag Key) of line 27 defines the type of tag, which is SensitiveDataHandling (sensitive data processing) in this example. The value (Value) applied to the tag in line 28 is SimpleHash (simple hash), indicating that the sensitive data identified in the document will be blurred with a simple hash value. The conditions in lines 31-38 identify that when the feature project id in the document is not null, project id will be tagged. The project id is obfuscated using a simple hashing technique, and the data cleaning module 124 replaces the project id with the hash value 208.

[0032] It should be noted that SimpleHash tags can be applied to other fields in the document. For example, SimpleHash tags can be applied to correlation identifiers. Correlation identifiers (also known as transition identifiers) are unique identifiers applied to requests and messages that provide a reference to a specific transaction or chain of events.

[0033] Figure 2C An exemplary rule 210 showing application of rotating hashing to a MAC address hash in a document. A MAC address is a media access control address, which represents a unique identifier assigned to a network interface controller (NIC). A MAC address hash is a hash value obtained by applying a hash algorithm to a MAC address.

[0034] Lines 42-62 define the event definition. Line 43 defines the event name (EventName) as workspace / search. Line 44 defines the event field (EventField) to which the rule is applied, namely EventName. The rule consists of tags (Tags) in lines 47-51 and conditions (Conditions) in lines 52-59. The tag key (Tag Key) in line 48 defines the type of tag, which is SensitiveDataHandling in this example. The value (Value) of the tag key applied to the tag in line 49 is RollingHash, indicating that the sensitive data identified in the document will be blurred using the rotating hash algorithm. The conditions (Conditions) in lines 52-59 identify that when the feature workspace.macaddresshash (workspace.MAC address hash) in the document is not an empty value, workspace.macaddresshash (workspace.MAC address hash) will be tagged. The workspace.macaddresshash is obfuscated using a more sophisticated hashing technique, and the data sanitization module 124 replaces the sensitive data with the hash value in the document 212 .

[0035] Figure 2DRepresents exemplary rule 214, which indicates that sensitive data is converted into a general value or a non-sensitive value. In this example, the geographic location in the document is converted into a general location. The geographic location represents the geographic coordinates (e.g., latitude and longitude coordinates) of the physical location. Lines 63-83 define event definitions (EventDefinition). Line 64 defines the event name (EventName), which corresponds to workspace / search (workspace / search). Line 65 defines the event field (EventField) to which the rule is applied, i.e. EventName (event name). The rule consists of the tags (Tags) of lines 68-72 and the conditions (Conditions) of lines 73-80. The tag key (Tag Key) of line 69 defines the type of tag that is SensitiveDataHandling (sensitive data processing) in this example. The value (Value) applied to the tag of line 70 is GeoConvertRegion (geographic conversion region), which represents that the sensitive data identified in the document will be converted into a general location, such as country and region names. The conditions in lines 74-80 indicate that workspace.GeoLocation will be tagged if the property workspace.GeoLocation in the document is not null. workspace.GeoLocation is transformed to a general location 216.

[0036] It is important to note that the GeoLocation tag is not the only conversion type tag. Another conversion tag is the IPAddress tag, which indicates that the IP address is to be converted to the name of the IP service provider.

[0037] Note that we now turn to a description of various exemplary methods of using the systems and devices of the present disclosure.

[0038] method

[0039] The operation of these aspects can be further described with reference to various exemplary methods. It is understood that, unless otherwise indicated, the representative methods do not necessarily have to be performed in the order presented or in any particular order. In addition, the various activities of the methods can be performed in a serial or parallel manner, or any combination of serial and parallel operations. In one or more aspects, the method illustrates the operation of the systems and devices disclosed herein.

[0040] Figure 3An exemplary method 300 is illustrated that illustrates the manner in which sensitive data in a document is cleansed in one aspect. Figure 1 and Figure 3 , the attribute module 120 receives a document 116, which may contain sensitive data (block 302). When the document 116 is sent from the client device 102, the document 116 may be received in batches or individually (block 302).

[0041] The property module 120 also receives the rules 118. In one aspect, the rules 118 are implemented in a script file and are written in a declarative language (e.g., XML, JSON). The script file can be easily changed to accommodate additional documents that may have a format in a different mode and / or to accommodate changes to an existing format. (Collectively, block 304).

[0042] The attribute module 120 applies a rule-based script to the document to identify sensitive data and attribute the sensitive data with appropriate labels. The labels represent actions to be applied to fields / features in the document in order to retain the document for further processing. In the case where the sensitive data is personal data, the sensitive data may include user names, machine names, user ids, phone numbers, mailing addresses, work addresses, email addresses, business titles, occupations, geographic locations, and / or other data that is considered personal. Depending on the type of sensitive data and the rules, the type of action to be applied to the sensitive data is labeled. (Collectively, box 306).

[0043] The tagged document is then processed by the data cleansing module 124. The data cleansing module 124 transforms the document into a version that can be used by downstream processing. Depending on the type of sensitive data contained in the document, the sensitive data may be deleted / removed, converted to a non-sensitive format, and / or hashed via a simple hashing technique or a rotating hashing technique. (Collectively, block 308). Once the document is tagged, the telemetry data without the sensitive data is processed (block 310).

[0044] Go to Figure 4, a method 400 for cleaning sensitive data in a document is shown. The data cleaning module 124 reads each tag to determine the cleaning action to be applied to a particular feature or field in the document (box 402). When the tag indicates Drop, the sensitive data is eliminated from the document (box 404). When the tag indicates Convert, a conversion action, such as GeoConvertRegion, the sensitive data is converted to a more general or less sensitive value (box 406). In the case of the GeoConvertRegion tag, the geographic coordinates are converted to a country and region location, and in the case of the IPAddress tag, the IP address is converted to the name of the service provider.

[0045] When the tag indicates SimpleHash, the sensitive data is replaced with a one-way hash value. A one-way hash value is generated from the application of a hash function without a key. The result is a fixed-length hash value that cannot be reversed (block 408).

[0046] For certain types of sensitive data, the sensitive data is obfuscated (blocks 410-418) using a more complex hashing technique known as RollingHash. In the rolling hashing technique, the original value of the sensitive data is obfuscated using a hash function and a first generation seed. The obfuscated value is replaced in the document, and the obfuscated value is stored in a rotating hash table. If another document with the same value for the sensitive data arrives at the telemetry data processing center, another hash value is calculated using a different seed. The sensitive data is then obfuscated using the second hash value.

[0047] The hash function can be any common hash function or cryptographic hash function, such as the Message Digest 5 (MD5) algorithm, the Secure Hash Algorithm (SHA), etc. In one aspect, the SHA-256 function is used using the original value of the sensitive data and a seed. The seed or salt is random data (i.e., a random value) used in the hash function to protect the stored hash value.

[0048] Back to Figure 4, when the tag indicates a rotated hash for the characteristic, a first fuzzy hash value is created for the value using the hash function and the first generation seed (box 410). The first fuzzy hash value is searched in the rotated hash table (box 412). If the rotated hash table has an entry for a fuzzy value for the sensitive data (box 414-yes), the seed is incremented, a second fuzzy value is generated, and is used to replace the value of the sensitive data (box 416). Otherwise, if there is no entry for a fuzzy value for the sensitive data (box 414-no), the value of the sensitive data is replaced with the first fuzzy value, and the first fuzzy value is stored in the rotated hash table (box 418).

[0049] Rule-based labeling can be used for other types of processing besides cleaning personal identifiers or sensitive data. Figure 6 , an exemplary rule 600 for finding groups or user groups 602 with common characteristics is shown. In this example, the rule is used to find event data for users belonging to .NET Core. .NET Core is an open source managed computer software framework for Windows, Linux and Mac operating systems. In example 600, if a document is named vs / solution / project (box 604) and contains a feature named vs.solution.project.targetframework (box 608), the value of which is .NETCoreApp, then the document is tagged as .NetCore (box 606). Documents tagged with the same tag .NetCore can be found and aggregated to study the behavior or characteristics of anonymous users belonging to the group. Similarly, rules can be used to tag features to find groups of anonymous users, who are C# developers, Python developers, etc.

[0050] By way of another example, tags can be used to homogenize data in documents that have different schemas or formats. This is the case when the documents are generated from different products that dictate different formats for their event data. Rule-based tags can be used to align the data in the documents so that the data looks the same to downstream processing. Figure 7 , there are three exemplary rules applicable to three different software products. These rules are used to align session start events from different products 700. Rule 702 pertains to events generated from a first product, rule 704 pertains to events generated from a second product, and rule 706 pertains to events generated from a third product.

[0051] Rule 702 is applied to an event or document named vs / core / sessionstart 708, which is generated during the execution of the first product. The rule tags the document with a SessionStart tag having a value set to True 710. Rule 704 is applied to an event named xs / core / sessionstart 712, whose property name is context.default.xs.core.exename 716 and whose value is Studio.app. The Boolean value of the SessionStart tag is set to True 714. Rule 706 is applied to an event named context / create 718, whose property name is reserved.contextname 722 and whose value is Default. The event has another property 724 named context.default.xs.core.os.productname whose value is macOS. The SessionStart tag is applied to the event 720 with a Boolean value of True. This rule is used so that the information in the event data generated by the three products belonging to SessionStart is homogenized so that they appear to be the same 726.

[0052] Exemplary Operating Environment

[0053] Note that the discussion now turns to an exemplary operating environment. Figure 5 An exemplary operating environment 500 is illustrated in which a computing device 502 is used to clean sensitive data in a document containing telemetry data. However, it should be noted that the aspects disclosed herein are not limited to any particular configuration of a device. The computing device 502 can be configured as part of a cloud service that cleans documents for sensitive data. It should be noted that the operating environment is not limited to any particular configuration and other configurations are possible.

[0054] The computing device 502 can be any type of electronic device, such as but not limited to a mobile device, a personal digital assistant, a mobile computing device, a smart phone, a cellular phone, a handheld computer, a server, a server array or server farm, a Web server, a network server, a blade server, an Internet server, a workstation, a minicomputer, a mainframe computer, a supercomputer, a network tool, a Web tool, a distributed computing system, a multiprocessor system, an IOT device, or a combination thereof. The operating environment 500 can be configured in a network environment, a distributed environment, a multiprocessor environment, or a stand-alone computing device that can access a remote or local storage device.

[0055] The computing device 502 may include one or more processors 506, a communication interface 508, one or more storage devices 510, one or more input / output devices 512, and at least one memory 514. The processor 506 may be any commercially available or custom processor, and may include dual microprocessors, quantum computers, and multi-processor architectures. The communication interface 508 facilitates wired or wireless communication between the computing device 502 and other devices. The storage device 510 may be a computer-readable medium that does not contain a propagated signal (such as a modulated data signal transmitted by a carrier wave). Examples of the storage device 510 include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disk (DVD), or other optical disk storage, cassettes, tapes, disk storage, all of which do not contain a propagated signal (such as a modulated data signal transmitted by a carrier wave). There may be multiple storage devices 510 in the computing device 502. The input / output device 512 may include a keyboard, a mouse, a pen, a voice input device, a touch input device, a display, a speaker, a printer, a camera, etc. and any combination thereof.

[0056] The memory 514 may be any non-transitory computer-readable storage medium that can store executable steps, applications, and data. The computer-readable storage medium is independent of propagated signals, such as modulated data signals transmitted by carrier waves. It may be any type of non-transitory memory device (e.g., random access memory, read-only memory, etc.), magnetic storage device, volatile storage device, non-volatile storage device, optical storage device, DVD, CD, floppy disk drive, etc. that is independent of propagated signals (such as modulated data signals transmitted by carrier waves). The memory 514 may also include one or more external storage devices or remotely located storage devices that are independent of propagated signals, such as modulated data signals transmitted by carrier waves.

[0057] The memory 514 may contain instructions, components, and data. A component is a software program that performs a specific function, also referred to as a module, program, and / or application. The memory 514 may include an operating system 516, one or more documents 518, rules 520, an attribute module 522, tagged documents 524, a data cleaning module 526, a rotating hash storage device 528, cleaned documents 530, a telemetry data processing module 532, other applications, and data 534.

[0058] The computing device 502 may be communicatively coupled to a network 504. The network 504 may be configured as an adhoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), the Internet, a portion of a public switched telephone network (PSTN), a plain old telephone service (POTS) network, a wireless network, network, or any other type of network or combination of networks.

[0059] The network 504 may employ various wired and / or wireless communication protocols and / or technologies. The various generations of communication protocols and / or technologies that may be employed by the network may include, but are not limited to, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access 2000 (CDMA-2000), High Speed ​​Downlink Packet Access (HSDPA), Long Term Evolution (LTE), Universal Mobile Telecommunications System (UMTS), Evolution Data Optimized (Ev-DO), Worldwide Interoperability for Microwave Access (WiMax), Time Division Multiple Access (TDMA), Orthogonal Frequency Division Multiplexing (OFDM), Ultra Wideband (UWB), Wireless Application Protocol (WAP), User Datagram Protocol (UDP), Transmission Control Protocol / Internet Protocol (TCP / IP), any part of the Open Systems Interconnection (OSI) model protocol, Session Initiation Protocol / Real-time Transport Protocol (SIP / RTP), Short Message Service (SMS), Multimedia Message Service (MMS), or any other communication protocol and / or technology.

[0060] in conclusion

[0061] A system having one or more processors and a memory is disclosed. The system has one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors. The one or more programs include the following instructions: receiving a document including telemetry data and sensitive data; applying one or more rules to the document to identify one or more fields in the document as sensitive data, and labeling at least one identified field in the document with a classification represented in the one or more rules; based on the label, cleaning the sensitive data from the document; and processing the telemetry data that does not contain the sensitive data.

[0062] The one or more programs include further instructions to: when the classification indicates removal, clean the sensitive data by deleting the sensitive data from the document; when the classification indicates transformation, clean the sensitive data by replacing the value of the sensitive data with a non-sensitive value; when the classification indicates simple hashing, clean the sensitive data by replacing the value of the sensitive data with a one-way hash value; and when the classification indicates rotated hashing, clean the sensitive data by replacing the value of the sensitive data with an obfuscated value. The one or more rules are contained in a script file that is applied to the document.

[0063] A method is disclosed, the method comprising: receiving a current document at a computing device having at least one processor communicatively coupled to a memory, the current document containing telemetry data and sensitive data; applying a label to the document based on fields in the current document that satisfy rules and conditions for being classified as sensitive data; replacing the sensitive data in the current document with a second fuzzy value when the applied label identifies that the sensitive data was previously replaced by a first fuzzy value in a previous document, the first fuzzy value being different from the second fuzzy value; and analyzing the telemetry data without the sensitive data.

[0064] The method also includes: replacing the identifier with a one-way hash when the applied label identifies the field in the document as an identifier associated with a software product; converting the field in the document to a non-sensitive location when the applied label identifies the field in the document as a geographic location; converting the field in the document to a name of a service provider when the applied label identifies the field in the document as an IP address; removing the email address from the document when the applied label identifies the field in the document as an email address; removing the machine name from the document when the applied label identifies the field in the document as a machine name; obfuscating the value of the project identifier using the hash value when the applied label identifies the field in the document as a project identifier; and obfuscating the value of the related identifier using the hash value when the applied label identifies the field in the document as a related identifier. The sensitive data previously replaced includes a MAC address hash.

[0065] A device having at least one processor and a memory is disclosed. The at least one processor is configured to: label a document with a classification based on characteristics in the document that satisfy rules and conditions, the document including telemetry data and sensitive data; convert the sensitive data to a non-sensitive value when the classification indicates conversion; replace the sensitive data with a hash value from application of a one-way hash function when the classification indicates simple hashing; and analyze the telemetry data without the sensitive data.

[0066] The at least one processor is further configured to replace the sensitive data with a first hash value when the classification indicates a rotated hash; and replace the sensitive data with a second hash value when the sensitive data is received at a subsequent time. The at least one processor is further configured to delete the sensitive data from the document when the classification indicates a removal. The document is tagged by applying a tag document containing a rule. The transformation is applied to the geographic location and the related identifier.

[0067] Although the subject matter has been described with specific language directed to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method for document processing, comprising: receiving, at a computing device having at least one processor communicatively coupled to a memory, a current document containing telemetry data and unsanitized sensitive data; applying a label to the current document to represent an attribute, the attribute identifying a cleansing action to be performed on the uncleaned sensitive data, the label being based on a field in the current document satisfying rules and conditions for being classified as sensitive data; generating a first fuzzy value for the uncleaned sensitive data; Searching the rotation hash table for the first fuzzy value; When the tag identifies a rotated hash cleaning action applied and the search of the rotated hash table finds the first fuzzy value, replacing the uncleaned sensitive data in the current document with a second fuzzy value generated using an incremental seed, the first fuzzy value being different from the second fuzzy value; as well as The telemetry data is analyzed without the uncleaned sensitive data.

2. The method according to claim 1, further comprising: When the applied tag identifies a field in the current document as an identifier associated with a software product, the identifier is replaced with a one-way hash.

3. The method according to claim 1, further comprising: When the applied tag identifies a field in the current document as a geographic location, converting the field to a non-location sensitive field.

4. The method according to claim 1, further comprising: When the applied tag identifies a field in the current document as an IP address, the field is converted into a name of a service provider.

5. The method according to claim 1, further comprising: When the applied tag identifies a field in the current document as an email address, the email address is removed from the current document.

6. The method according to claim 1, further comprising: When the applied tag identifies a field in the current document as a machine name, removing the machine name from the current document.

7. The method according to claim 1, further comprising: When the applied tag identifies a field in the current document as an item identifier, the value of the item identifier is obfuscated with a hash value.

8. The method according to claim 1, further comprising: When the applied tag identifies a field in the current document as a relevant identifier, the value of the relevant identifier is obfuscated with a hash value.

9. The method of claim 1, wherein the sensitive data previously replaced comprises a MAC address hash.

10. A system for document processing, comprising: Processor and memory; The memory includes instructions, and when the instructions are executed on the processor, the processor performs the following actions: obtaining a current document, wherein the current document includes telemetry data and uncleaned sensitive data; applying a label to the current document to represent an attribute, the attribute identifying a cleansing action to be performed on the uncleaned sensitive data, the label being based on a field in the current document satisfying rules and conditions for being classified as sensitive data; generating a first fuzzy value for the uncleaned sensitive data; Searching the rotation hash table for the first fuzzy value; When the tag identification rotated hash cleaning action is applied and the search of the rotated hash table finds the first fuzzy value, replacing the uncleaned sensitive data in the current document with a second fuzzy value generated using an incremental seed, the first fuzzy value being different from the second fuzzy value; and The telemetry data is analyzed without the uncleaned sensitive data.

11. The system of claim 10, wherein the memory comprises further instructions which, when executed on the processor, cause the processor to: When the applied tag identifies a field in the current document as an identifier associated with a software product, the identifier is replaced with a one-way hash.

12. The system of claim 10, wherein the memory includes further instructions that, when executed on the processor, cause the processor to: When the applied tag identifies a field in the current document as a geographic location, converting the field to a non-location sensitive field.

13. The system of claim 10, wherein the memory includes further instructions that, when executed on the processor, cause the processor to: When the applied tag identifies a field in the current document as an IP address, the field is converted into a name of a service provider.

14. The system of claim 10, wherein the memory includes further instructions that, when executed on the processor, cause the processor to: When the applied tag identifies a field in the current document as an email address, the email address is removed from the current document.

15. The system of claim 10, wherein the memory includes further instructions that, when executed on the processor, cause the processor to: When the applied tag identifies a field in the current document as a machine name, removing the machine name from the current document.

16. The system of claim 10, wherein the memory includes further instructions that, when executed on the processor, cause the processor to: When the applied tag identifies a field in the current document as an item identifier, the value of the item identifier is obfuscated with a hash value.

17. The system of claim 10, wherein the memory includes further instructions that, when executed on the processor, cause the processor to: When the applied tag identifies a field in the current document as a relevant identifier, the value of the relevant identifier is obfuscated with a hash value.

18. The system of claim 10, wherein the sensitive data previously replaced comprises a MAC address hash.

Images