Machine learning models based on altered data and systems and methods for training and using the models

Abstract

The data can be abstracted and / or masked prior to providing the data to a machine learning model for training. The machine learning model can provide a confidence level associated with a result. If the confidence level is too high, the machine learning model or an application that includes the machine learning model can refrain from providing the result as an output. In some examples, the machine learning model can provide a "suboptimal" result with an acceptable confidence level. In other examples, an error signal can be provided as an output. According to examples of the present disclosure, the data can be abstracted and / or masked prior to providing the data to a machine learning model for training, and a confidence level of a result of a trained machine learning model can be used to determine when a result should be retained.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 955,297, filed December 30, 2019, and U.S. Application No. 16 / 854,107, filed April 21, 2020. The foregoing applications are incorporated by reference in their entirety for all purposes. BACKGROUND

[0003] Machine learning generally encompasses a variety of methods for utilizing one or more computational models to classify or otherwise infer meaning from data. Examples of models include, but are not limited to, artificial neural networks (deep and shallow, convolutional, recurrent, probabilistic graphical models, Bayesian neural networks, etc.), decision tree-based methods, and support vector machines (including kernelized versions). In some cases, a model can include one or more algorithms and different network architectures. Generally, training the model includes providing a training data set. A computing device analyzes the training data set to develop the model. The trained model can then be used (by the same computing device or another computing device) to analyze new data sets to accomplish one or more tasks, such as providing output in response to a query. The training can be supervised or unsupervised or semi-supervised. In some cases, the model can continue to be trained and / or modified over time based on additional data received after the initial training, which is referred to as a dynamic training model.

[0004] In some applications, the training data and / or additional data can include sensitive information (e.g., personally identifiable information, financial information, proprietary business information). The release of sensitive information can cause harm to individuals or entities (e.g., identity theft). Thus, it can be desirable to prevent the exposure of sensitive information in training data and / or additional data provided to a model. SUMMARY

[0005] According to at least one example of the disclosure, an apparatus can include a processor; and a non-transitory computer-readable medium configured to store input received from another device, wherein the non-transitory computer-readable medium is encoded with instructions that, when executed by the processor, cause the processor to: generate, using a machine learning model, a first value indicative of a result and a second value indicative of a confidence level from the input; compare the second value to a threshold value, wherein the threshold value is indicative of the first value that includes a third value stored from a training data set used to train the machine learning model; and provide an output to other devices based on the comparison.

[0006] According to at least one example of the disclosure, a method can include receiving data input; generating, using a machine learning model, a first value indicative of a result and a second value indicative of a confidence level from the data input; comparing the second value to a threshold value, wherein the threshold value is indicative of the first value, the first value comprising a third value stored from a training data set used to train the machine learning model; and providing an output based on the comparison.

[0007] According to at least one example of the disclosure, an apparatus can include a processor; and a non-transitory computer-readable medium configured to store data, wherein the non-transitory computer-readable medium is encoded with instructions that, when executed by the processor, cause the processor to generate a first value indicative of a ranking, wherein the first value is indicative of a number of times one or more values are included in the data; generate a second value indicative of a classification, wherein the second value is indicative of one or more data types included in the data; change a value of the one or more values included in the data based on the first value and the second value; and generate a training data set comprising the data including the changed value.

[0008] According to at least one example of the disclosure, a method can include receiving data; generating a first value indicative of a ranking, wherein the first value is indicative of a number of times one or more values are included in the data; generating a second value indicative of a classification, wherein the second value is indicative of one or more data types included in the data; changing a value of the one or more values included in the data based on the first value and the second value; and generating a training data set comprising the data including the changed value. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1A Examples of machine learning models trained to provide descriptive textual captions for images are described.

[0010] Figure 1B Examples of machine learning models trained to provide textual output for voice input are described.

[0011] Figure 2 Schematic illustration of a computing device arranged in accordance with examples of the disclosure.

[0012] Figure 3 Functional block diagram of a machine learning application for abstracting and / or masking data in accordance with examples of the disclosure.

[0013] Figure 4 Examples of tokenizing data in accordance with examples of the disclosure are shown.

[0014] Figure 5 Flowchart of a method in accordance with examples of the disclosure.

[0015] Figure 6 This is a functional block diagram of a machine learning application for providing output, based on an example of this disclosure.

[0016] Figure 7 A diagram of a neural network based on an example of this disclosure.

[0017] Figure 8 Examples of the schematic diagram and output probability distribution of a neural network according to the present disclosure.

[0018] Figure 9 A flowchart of a method according to an example of this disclosure.

[0019] Figure 10 A computing system based on an example of this disclosure. Detailed Implementation

[0020] Sensitive data may be contained in the training data of models provided to train machine learning applications and / or in other data provided to dynamically train the model. While the model is trained to classify data or otherwise infer meaning from data based on the training set and / or other data, in some cases, the model may "store" data from the training set, attributable to the training method and / or the content of the training set. In these cases, the model may output the stored data in response to a determined classification or inference based on data input to the model. In some cases, the stored data may be sensitive data that should not be disclosed. In some cases, the stored data may be personal information of the user from whom the training data was obtained, and / or the stored data may provide clues to competitors attempting to reverse engineer the model regarding how the model was trained and / or the content of the training dataset. Therefore, there is a need for machine learning models and / or applications with greater privacy awareness.

[0021] Figure 1A This describes an instance of a machine learning model 102 trained to provide descriptive textual captions for images. The machine learning model 102 is trained to provide textual outputs from image inputs. Training may involve providing a training dataset containing hundreds or thousands of images (e.g., inputs) with textual captions describing elements contained in the images (e.g., the desired result). Based on the training dataset, the machine learning model can learn to recognize various elements in the images and provide text associated with those elements. In some instances, the machine learning model may include a neural network.

[0022] exist Figure 1AIn the first example shown, image 100 is fed to a trained machine learning model 102. The machine learning model 102 analyzes image 100 and provides the descriptive text 104 "Road to the Mountain" as a result. In the second example, image 110 is fed to the machine learning model 102. The model analyzes image 110 and provides the descriptive text 114 "A busy desk with a tablet, glasses, a cup, and a notepad with the password 1p5x3c9r".

[0023] In the first example, machine learning model 102 provides descriptive text 104 that reflects the content of image 100 but likely also describes many other images. In the second example, machine learning model 102 provides descriptive text 114 that reflects the content of image 110, but descriptive text 114 describes image 110 in more detail and is unlikely to describe other images. From descriptive text 114, it can be inferred that the training dataset likely contains a small number of desk images with the other elements described in descriptive text 114. In this example, machine learning model 102 may store training images and / or their associated descriptive text. In some cases, it can be inferred that image 110 is included in the training dataset. This analysis of the results of machine learning model 102 allows the user to determine the characteristics and / or content of the training dataset and / or determine how to train machine learning model 102.

[0024] Furthermore, the explanatory text 114 contains a password. In some instances, the password may already be in the images of the training dataset. In some instances, the password may be sensitive information that is not intended to be presented as the result of the machine learning model 102.

[0025] Figure 1B This describes an instance of a machine learning model 122 trained to provide text output 132 in response to voice input 120. For example... Figure 1BAs shown in the diagram, a user can provide voice input 120 to a computing device, such as “Please provide the best route to Margaret’s house.” Sound waves from the voice input 120 can be received by a speaker included within the computing device, and the computing device can provide a signal (e.g., a digital signal) to a machine learning model 122 in response to the sound waves. This machine learning model can be included on the computing device and / or on a cloud computing system communicating with the computing device. As shown in box 124, the machine learning model 122 can make inferences based on the signal to determine what words were spoken. Once the words are determined as shown in box 126, the machine learning model 122 can infer the user’s intent based on the words shown in box 128. In simpler terms, the machine learning model 122 can determine what the user wants the computing system to do. Based on the inferred intent, the machine learning model 122 can formulate a response (e.g., an output) as shown in box 130. The output can then be provided to the user by the computing device, for example, by displaying it on a screen. In this example, as shown in box 132, the output is the direction to Margaret’s house.

[0026] If Margarita is the name of a publicly accessible business or an individual known to the user, then the output of machine learning model 122 is desirable. For example, the user may have provided her personal address book to the computing device for analysis by machine learning model 122, and Margarita may be a contact in that address book. However, if Margarita is an individual and unknown to the user, then providing the user with directions to Margarita's residence may be undesirable. In these instances, it may be necessary to prevent the computing device from providing results to the user and / or providing alternative results, such as directions to a publicly accessible business with a similar pronounced name (e.g., "Margarita Hut" in this example).

[0027] although Figure 1A The examples provided offer image input and text output, and Figure 1B The example provided offers both voice input and text output, but machine learning storage can occur alongside other data types, such as text for both input and output, voice data for both input and output, voice data for input and text for output, etc. For instance, storage can occur when text is both input and output, such as when a machine learning model suggests words or phrases to a user typing a document (e.g., an email) based at least in part on letters or words the user has already typed. In this example, the user might have typed "Let's meet at Jane's house," and if the machine learning model stores the output based on the input, it could suggest a specific address for the house. In this case, the privacy of the resident at that specific address could be compromised.

[0028] According to examples of this disclosure, data may be abstracted and / or masked before being fed to a machine learning model for training. In some applications, this increases the "privacy awareness" of the machine learning model and reduces or prevents the machine learning model from "storing" sensitive information. According to examples of this disclosure, the machine learning model may provide a confidence level associated with the result. If the confidence level is too high, the machine learning model or the application containing the machine learning model may avoid providing the result as output. In some instances, the result may not be provided when the confidence level of a particular output is too high. In other instances, the machine learning model may provide a "second-best" result with an acceptable confidence level. This "second-best" result may be more privacy-conscious because it is less likely to reveal sensitive information. In still other instances, an error signal may be provided as output. According to examples of this disclosure, data may be abstracted and / or masked before being fed to a machine learning model for training, and the confidence level of the result of training the machine learning model can be used to determine when the result should be retained. Under certain conditions, processing the data used to train machine learning models and / or not providing results from machine learning models can reduce or prevent the exposure of sensitive data and / or the reverse engineering of machine learning models, training methods, and / or training data.

[0029] Figure 2 This is a schematic illustration of a computing device arranged according to an example of the present disclosure. The computing device 200 may include a processor 202, a computer-readable medium (or media) 204, a memory controller 210, a memory 212, and an interface 214. In some embodiments, the computing device 200 may include a display 216.

[0030] Computer-readable medium 204 may be accessible to processor 202. Computer-readable medium 204 may be encoded with executable instructions 208. Executable instructions 208 may be executed by processor 202. In some instances, executable instructions 208 may cause processor 202 to implement a machine learning application comprising one or more machine learning models. The machine learning application may implement various functions, such as generating training datasets, training machine learning models, and / or applying trained machine learning models to received data to produce results. Alternatively or additionally, in some instances, the machine learning application or a portion thereof may be implemented in hardware contained within computer-readable medium 204 and / or processor 202, such as application-specific integrated circuits (ASICs) and / or field-programmable gate arrays (FPGAs).

[0031] Computer-readable medium 204 may store data 206. In some instances, data 206 may contain one or more training datasets, such as training dataset 218. In some instances, training dataset 218 may be received from another computing device (e.g., edge device 222, cloud computing device). In other instances, training dataset 218 may be generated by computing device 200. In some instances, the training dataset may be used to train one or more machine learning models. In some instances, data 206 may contain data used in the machine learning model (e.g., weights, connections between nodes). In some instances, data 206 may contain other data, such as new data 220. In some instances, other data may be analyzed by the trained machine learning model to make inferences (e.g., providing results / outputs based on the data). In some instances, data 206 may contain outputs generated by one or more machine learning models implemented by computing device 200. Computer-readable medium 204 may be implemented using any medium containing non-transitory computer-readable media. Examples include memory, random access memory (RAM), read-only memory (ROM), volatile or non-volatile memory, hard disk drives, solid-state drives, or other storage devices. Although Figure 2 A single media may be displayed, but multiple media may be used to implement computer-readable media 204.

[0032] In some instances, processor 202 may be implemented using one or more central processing units (CPUs), graphics processing units (GPUs), ASICs, FPGAs, or other processor circuitry systems. In some instances, processor 202 may execute some or all of the executable instructions 208. In some instances, processor 202 may communicate with memory 212 via memory controller 210. In some instances, memory 212 may be volatile memory, such as dynamic random access memory (DRAM). In some instances, memory 212 may provide information to and / or receive information from processor 202 and / or computer-readable media 204 via memory controller 210. Although a single memory 212 and a single memory controller 210 are shown, any number may be used. In some instances, memory controller 210 may be integrated with processor 202.

[0033] In some instances, interface 214 may provide a communication interface to another device (e.g., edge device 222), a user, and / or a network (e.g., LAN, WAN, Internet). Interface 214 may be implemented using wired and / or wireless interfaces (e.g., Wi-Fi, Bluetooth, HDMI, USB, etc.). In some instances, interface 214 may include user interface components that can receive input from a user. Examples of user interface components include a keyboard, mouse, touchpad, touchscreen, and microphone. In some instances, interface 214 may transmit information that may include user input, data 206, training dataset 218, and / or new data 220 between an external device (e.g., edge device 222) and one or more components of computing device 200 (e.g., processor 202 and computer-readable media 204).

[0034] In some instances, computing device 200 may communicate with display 216 as a separate component (e.g., using a wired and / or wireless connection), or display 216 may be integrated with computing device. In some instances, display 216 may display data 206, such as the output generated by one or more machine learning models implemented by computing device 200. Any number or variety of displays may be present, including one or more LED, LCD, plasma, or other display devices.

[0035] In some instances, training dataset 218 and / or new data 220 may be provided to computing device 200 via interface 214. Optionally, in some instances, some or all of the training dataset 218 and / or new data 220 may be provided to computing device 200 by edge device 222. In some instances, computing device 200 may provide results (e.g., inferences made by a machine learning application) to edge device 222. In some instances, edge device 222 may also be a computing device containing components similar to those shown in computing device 200. In some instances, edge device 222 may be a mobile device such as a smartphone or tablet computer. In some instances, edge device 222 may be a desktop computer or other fixed device. In some instances, edge device 222 and computing device 200 may be included in a computing system such as a cloud computing system. In this example, computing device 200 may be a cloud computing device. In some instances, computing device 200 may be included in a server.

[0036] In some instances, computing device 200 may process data (e.g., data 206, training dataset 218, and / or new data 220) to mask and / or abstract sensitive information. The processed data can be used to generate training sets for training machine learning models (e.g., neural networks, support vector machines, decision trees). In some instances, the machine learning model may be trained by computing device 200. In some instances, the trained machine learning model may be implemented by computing device 200 and / or computing device 200 may implement one or more other trained machine learning models.

[0037] In some instances, computing device 200 may implement a machine learning model that provides results (also known as inferences) based on inputs (e.g., data such as new data 220) and a confidence level associated with the results. The machine learning model and / or other components of computing device 200 may provide outputs based on the confidence level associated with the results. For example, if the confidence level is equal to or higher than a threshold, it may indicate that the machine learning model has "stored" results from the training dataset. In this case, the output may not contain results. In some instances, computing device 200 may output different results (e.g., results with a second highest confidence level) from a machine learning model with an acceptable confidence level (e.g., equal to or lower than a threshold) and provide this result as output. In some instances, the output may contain error signals.

[0038] Figure 3 This is a functional block diagram of a machine learning application 300 for abstracting and / or masking data according to an example of this disclosure. In some instances, the machine learning application 300 may be implemented by computer-readable instructions. In some instances, the machine learning application 300 may be implemented by hardware (e.g., FPGA and / or ASIC). In some instances, the machine learning application 300 may be implemented by a combination of computer-readable instructions and hardware. In some instances, the machine learning application 300 may be implemented by... Figure 2 The computing device 200 shown in the figure is implemented.

[0039] Machine learning application 300 may include training dataset 302. Training dataset 302 may include one or more inputs (X) 304, each associated with a corresponding result (Y) 306. In some instances, training dataset 302 may exist pre-existing. In other instances, machine learning application 300 may generate training dataset 302 from received data 322. In some instances, machine learning application 300 may generate training data by tokenizing received data 322, which is related to... Figure 4For more detailed description. In some instances, data 322 may be received from computer-readable media contained within a computing device implementing a machine learning application. In some instances, data 322 may be received from an application 320 implemented by another computing device (e.g., edge device 222).

[0040] Machine learning application 300 can process training dataset 302 to abstract and / or mask sensitive data and produce a modified training dataset 310. As used herein, abstracting data means replacing specific values ​​of a data type with generic values. For example, a data type might be a proper name (e.g., John Smith, Sarah Jones). All proper names in the original data can be replaced with generic values ​​(e.g., name, proper name) indicating the presence of the proper name in the original data. In another instance, a data type might be a specific date (e.g., December 25, 1978). All specific dates can be replaced with generic values ​​indicating the existence of the date or a relative date (e.g., date, today's date). As used herein, masking data means removing specific values ​​of a data type. When removing specific values, they may or may not be replaced with an indication of the removed value (e.g., XXX).

[0041] The abstraction and / or masking 308 of the training dataset 302 may include classifying and / or ranking the data in the training dataset 302. Classifying the data means analyzing the data and identifying one or more data types contained within it. For example, data may be tokenized, and each token of the data may be analyzed to determine the data type contained within that token. A data type refers to the kind of information contained in the data (e.g., date, account, quantity, pixel intensity, diagnosis). Some data types may be sensitive data (e.g., proprietary names, addresses, account numbers). Ranking the data means analyzing the data and determining the frequency with which a particular data type and / or value exists in the data. For example, the ranking may determine the number of times the value appears "benign" in the data and / or in data classified as having a "diagnostic" data type.

[0042] In some instances, whether the values ​​of the data in training dataset 302 are abstracted or masked may be based at least in part on the classification and / or ranking of the values. In some instances, if the value is classified as non-sensitive data, then the value may not be abstracted or masked regardless of the ranking. In some instances, if the value is classified as sensitive data, but the ranking indicates that the value appears multiple times in the data (e.g., a threshold number of occurrences or a percentage of values ​​representing data types above a threshold), then the value may be abstracted. In some instances, if the value is classified as sensitive data and the ranking indicates that the value is rare (e.g., a number of occurrences below a threshold or a percentage of values ​​representing data types below a threshold), then the value may be masked.

[0043] In some instances, abstraction and / or 308 masking can be performed by rule-based systems (e.g., all numeric strings of a certain length are account numbers). In other instances, abstraction and / or 308 masking can be performed by machine learning models trained to identify the data types (including sensitive data) in the training dataset.

[0044] The data in the training dataset 302, processed by abstraction and / or masking 308, can be used to generate a modified training dataset 310. The modified training dataset 310 may contain one or more modified inputs (X') 314 and corresponding modified results (Y') 312. The modified training dataset 310 may abstract or remove some or all of the sensitive data from the training dataset 302. The modified training dataset 310 can be used to train a machine learning model 316. In some instances, using the modified training dataset 310 can reduce or eliminate the risk that the machine learning model 316 may "store" sensitive data that can then be provided as a result.

[0045] like Figure 3 As shown in the figure, once trained, the machine learning model (f())316 can receive new input data (X). new )324 and provides results based on new input data 324 326, making In some instances (e.g.) Figure 3 In the example shown, new input data 324 can be provided from an application 320 that can be implemented on a separate device, and the machine learning model 316 can provide results to the application 320. In some embodiments, when the machine learning model 316 is trained dynamically, the new input data 324 and the results 326 may be contained in another training dataset 302 that is abstracted and / or masked before being used to train the machine learning model 316.

[0046] Figure 4This demonstrates an example of tokenizing data according to the present disclosure. In some instances, tokenization may be performed by a machine learning application, such as machine learning application 300. The original data 400 is the text string "Let's organize a meeting on September 15, 2020". The original data 400 is parsed into segments called tokens 404, which, in some instances, may be analyzed individually by a machine learning model. In some instances (e.g.) Figure 4 In the example shown, the original data 400 can be tokenized so that elements of the original data 400 are repeated on different tokens 404. For example, the word "meeting" appears... Figure 4 There are three different tokens 404 in the input 406. The tokens 404 are organized such that the token 404 of the input 406 is associated with the token 404 of the desired result 408. All sets of input 406 and result 408 pairs can be used as training dataset 402 to train a machine learning model.

[0047] Figure 4 The examples provided illustrate tokenization using text data. In some instances, k-grams can be used to generate tokens from text data, but other methods can also be used. Furthermore, Figure 4 The examples provided are illustrative only, and this disclosure is not limited to the text data shown or the specific tokenization methods.

[0048] Figure 5 This is a flowchart of a method 500 according to an example of the present disclosure. In some instances, all or part of method 500 may be computed by a computing device (e.g., Figure 2 The computing device 200 shown in the figure executes the method. In some instances, all or part of the method 500 may be executed by a machine learning application (e.g., Figure 3 The machine learning application 300 shown in the figure is executed, and in some instances the machine learning application may be implemented by a computing device such as computing device 200.

[0049] At box 502, "Receive Data" can be performed. In some instances, the data can be received from an interface (e.g., interface 214). In some instances, the data can contain text, image, and / or audio data. In some instances, data can be received from an edge device (e.g., edge device 222).

[0050] At box 504, you can perform "Rank the data". In some instances, ranking can indicate the number of times one or more values ​​are included in the data. At box 506, you can perform "Classify the data". In some instances, classification can indicate one or more data types included in the data. In some instances, classification can be rule-based. In some instances, classification can be performed by a machine learning model, such as a neural network. In some instances, box 506 can be performed before box 504. In some instances, boxes 504 and 506 can be performed simultaneously.

[0051] At box 508, you can perform a "Change Value". In some instances, you can change the value of one or more values ​​contained in the data. In some instances, you can abstract or mask the value. In some instances, how and / or whether to change the value can be based at least in part on the value's ranking and classification. In some instances, changing the value may include masking the value when the classification indicates that the data type of the value is sensitive data and the ranking indicates that the value is contained in the data an amount equal to or less than a threshold. In some instances, changing the value may include abstracting the value when the classification indicates that the data type of the value is sensitive data and the ranking indicates that the value is contained in the data an amount equal to or greater than a threshold. In some instances, sensitive data may include proprietary names, dates, addresses, passwords, dates of birth, account numbers, and / or usernames.

[0052] At box 510, you can perform "Feeding data to a machine learning model". In some instances, the data provided to the machine learning model may contain changed values. That is, the data provided to the machine learning model can be modified from the data initially received at box 502. In some instances, the data can be used as a training dataset to train the machine learning model. Optionally, at box 512, you can perform "Training a machine learning model". The machine learning model can be trained using the training dataset.

[0053] Optionally, in some instances, "parse the data into one or more tokens" can be performed at box 514. In some instances, individual tokens may be included in at least a portion of the data received at box 502. In some instances (e.g.) Figure 5 In the example shown, parsing can be performed before ranking and / or classifying the data.

[0054] Figure 6This is a functional block diagram of a machine learning application 600 for providing output according to an example of this disclosure. In some instances, the machine learning application 600 may be implemented by computer-readable instructions. In some instances, the machine learning application 600 may be implemented by hardware (e.g., FPGA and / or ASIC). In some instances, the machine learning application 600 may be implemented by a combination of computer-readable instructions and hardware. In some instances, the machine learning application 600 may be implemented by... Figure 2 The computing device 200 shown is implemented. In some instances, the machine learning application 600 can be coupled with... Figure 2 The machine learning application 300 shown herein is used in combination and / or included in the machine learning application 300. For example, machine learning model 602 may be included in machine learning model 316 or machine learning model 316 may be included in machine learning model 602.

[0055] In some instances, machine learning application 600 may include machine learning model 602, which may be trained to produce a result (Y) 604 (e.g., inference) based on data (X) 622 provided as input to machine learning model 602. Machine learning model 602 may produce a confidence level (C) 606 associated with the result 604. Confidence level 606 may represent the degree of certainty (e.g., probability) that machine learning application 600 has provided the correct or desired result 604 based on data 622. (Reference) Figure 7 and 8 The determination of confidence level 606 is described in more detail. Typically, providing results with low confidence levels is undesirable. However, absolute or near-absolute certainty is rare in machine learning models. Therefore, the confidence level corresponding to this certainty indicates that the machine learning model 602 stores data from the training dataset ( Figure 6 Results (not shown in the original text) are used to train the machine learning model 602 on the training dataset. In some applications, results with high confidence levels may be more likely to contain sensitive data and / or information that could expose the machine learning model and / or the training dataset. Therefore, if the confidence level 606 is high, it may be necessary to avoid providing results 604.

[0056] In some instances, confidence level 606 can be analyzed, as shown in box 608. In some instances, box 608 may include a comparator that compares confidence level 606 to one or more thresholds. In some instances, confidence level 606 may be compared to a threshold that confirms that result 604 does not contain stored results from the training dataset. In some instances, the threshold may represent a high degree of certainty or probability that result 604 is correct or the desired result based on data 622. For example, in some instances, the threshold may be 0.99 or 1.00.

[0057] Optionally, in some instances, another threshold may confirm that the confidence level 606 is high enough to provide a correct result 604 with an acceptable level of reliability. What threshold corresponds to an acceptable level of reliability may vary depending on the application. For example, in some applications, a threshold of 0.51 may be an acceptable confidence level. In other applications, a threshold of 0.60 may be an acceptable confidence level. In still other applications, thresholds of 0.80, 0.90, or 0.95 may be acceptable confidence levels. In some applications, no threshold level may be used, and the classification with the highest probability (and / or the highest probability after removing any classifications with probabilities greater than the upper threshold) may be returned as the result 604.

[0058] Based on the analysis of confidence level 606, machine learning application 600 can provide output 624. In some instances, output 624 may include result 604, as indicated by box 610, if the analysis of confidence level 606 determines that result 604 is not a stored result (e.g., confidence level 606 is equal to or below a threshold). In some instances, output 624 may not include result 604, as indicated by box 612. In some instances, output 624 may include an error signal, as indicated by box 612. The error signal may indicate that a result cannot be provided for input data 622. Optionally, in some instances, an error signal may be provided when confidence level 606 is equal to or below a threshold indicating that result 604 is unreliable (e.g., has a low probability of being correct and / or the desired output for data 622).

[0059] Optionally, in some instances, if confidence level 606 indicates that result 604 is a stored result, then machine learning application 600 may generate another result (Y') from machine learning model 602 with a confidence level indicating that the result is not stored. That is, the confidence level of the new result Y' may be lower than the confidence level 606 associated with the original result 604. In some instances, result Y' may represent a "second-best" result. Result Y' may then be included in output 624, as indicated by box 614.

[0060] In some instances, data 622 may be provided by a separate application 620, which may be contained on a computing device separate from the computing device implementing the machine learning application 600. For example, application 620 may reside on an edge device (e.g., edge device 222). In some instances, output 624 may be provided to application 620.

[0061] In some applications, the concern regarding including the stored results in output 624 may vary depending on the source of data 622, the source of the training dataset used to train machine learning model 602, what output 624 is provided to, and / or the user of machine learning application 600. For example, if an administrator is using machine learning application 600, a high threshold (e.g., 1.0) may be set to determine whether to store the results. An example of an administrator could be a software engineer at a company that owns machine learning application 600 and is testing it. In another instance, if the user accessing machine learning application 600 (e.g., a user of application 620) is also a source of the training dataset, the threshold may also be set high. For example, when training intelligent writing machine learning model 602 only on the user's own emails. In another instance, if the user is not an administrator and machine learning model 602 is not trained separately on data from the user, the threshold may be set low (e.g., 0.97, 0.98, 0.99).

[0062] Figure 7 This is a diagram of a neural network 700 according to an example of the present disclosure. In some instances, the neural network 700 may be included in a machine learning model (e.g., machine learning model 316 and / or machine learning model 602). In some instances, the neural network 700 may be deterministic. The neural network 700 may include input nodes 702. In some instances, the input nodes 702 may be organized into layers. The input nodes 702 may be coupled to one or more layers of hidden units 706 via weights 704. In some instances, the hidden units 706 may perform operations on one or more inputs from the input nodes 702, at least in part, based on the associated weights 704. The output of the hidden units 706 may be provided to an output layer 708, which may return a confidence value, that is, a value associated with a confidence level in the result inferred by the neural network 700. The output layer 708 may compute a result provided to a result node 710. The associated confidence value (e.g., confidence level).

[0063] In some instances, the output layer 708 can use a flexible maximum function to calculate the confidence values ​​of the classification or regression output. The flexible maximum function can be expressed as:

[0064]

[0065] The maximum flexibility The confidence value used as the output. The output is n, and n is the number of outputs. However, in other instances, the confidence level can be computed using variants of the flexible maximum equation (e.g., argmax) or other equations or dedicated extra layers.

[0066] Figure 8 This is a diagram of a neural network 800 according to an example of this disclosure. In some instances, the neural network 800 may be included in a machine learning model (e.g., machine learning model 316 and / or machine learning model 602). In some instances, the neural network 800 may be stochastic (e.g., Bayesian representation). Similar to neural network 700, neural network 800 may include input nodes 802. In some instances, the input nodes 802 may be organized into layers. The input nodes 802 may be coupled to one or more layers of hidden units 806 via weights 804. In some instances, the hidden units 806 may perform operations on one or more inputs from the input nodes 802, at least in part, based on the associated weights 804. The output of the hidden units 806 may be provided to a result node 810. However, unlike neural network 700, the result at the result node 810 is not a single value, but a distribution of the output Y 808. In some instances, the distribution of the output Y can be used to estimate the confidence level from a probability distribution, which is represented as:

[0067] p(y (n) |x (n) ,θ)

[0068] Where θ represents the weights of neural network 800 and p is the conditional probability distribution on the output layer, from which the confidence level is derived. In other instances, other distributions or analysis of the distribution of output 808 may be used to determine the confidence level. Provided for illustrative purposes only. Figure 7 and 8 The techniques for determining confidence levels shown herein are not limited to the examples provided.

[0069] Figure 9 This is a flowchart of a method 900 according to an example of the present disclosure. In some instances, all or part of method 900 may be computed by a computing device (e.g., Figure 2 The computing device 200 shown in the diagram executes the method. In some instances, all or part of the method 900 can be executed by a machine learning application, such as... Figure 6 The machine learning applications shown in the document number 600 and / or Figure 3 The machine learning application 300 shown in the figure can be implemented by a computing device, such as computing device 200, in some instances.

[0070] At box 902, "receiving data input" can be performed. In some instances, the data input can be received by an interface (e.g., interface 214). In some instances, the data input can contain text, image, and / or audio data. In some instances, the data input can be received from an edge device (e.g., edge device 222).

[0071] At box 904, the action "Analyze the data input using a machine learning model to produce results and confidence levels" can be performed. In some instances, the machine learning model can be a neural network. In some instances, the neural network can be deterministic. In some instances, it can be based at least in part on a flexible maximum algorithm (e.g., Figure 7 The algorithm mentioned above generates confidence levels. In some instances, the neural network can be random. In some instances, confidence levels can be generated at least in part based on the distribution of the results.

[0072] At block 906, the action "comparing a confidence level to a threshold" can be performed. In some instances, this comparison can be performed by a comparator. At block 908, the action "providing output based on the comparison" can be performed. In some instances, the output can be provided from a computing device, such as computing device 200, to an edge device, such as edge device 222.

[0073] In some instances, the output includes an error signal when the confidence level is equal to or higher than the threshold. In some instances, the output includes a result when the confidence level is equal to or lower than the threshold. In some instances, the threshold is 0.99. In some instances, the threshold is at least partially based on the type of user of the machine learning model. User types may include, for example, regular users and administrators. In some instances, the threshold is at least partially based on the source of the training dataset used to train the machine learning model and the user of the machine learning model. In some instances, the threshold is higher when the source of the training dataset is users than when the source of the training dataset is not users.

[0074] Optionally, when the confidence level is equal to or higher than the threshold, in some instances, boxes 908 and 910 can be executed to provide a more privacy-conscious result. At box 908, the function "Analyze the data input using a machine learning model to produce a second result" can be executed. In some instances, the second result may have a second confidence level below the threshold. At box 910, the function "Provide a second result as output" can be executed.

[0075] In some instances, method 900 may be executed during and / or after method 500.

[0076] Figure 10 This is a computing system 1000 according to an example of the present disclosure. The computing system 1000 may include one or more edge devices 1012, such as wearable devices (e.g., smartwatches) 1002 and / or mobile devices (e.g., smartphones, tablets) 1004. The wearable devices 1002 and / or mobile devices 1004 may be operated by a user 1001. The computing system 1000 may further include a cloud computing system 1006, which may include one or more computing devices (e.g., computing device 200).

[0077] In some instances, edge device 1012 may implement one or more machine learning applications, such as applications 300, 320, 600, and / or 620, or portions thereof. For example, edge device 1012 may implement machine learning applications that abstract and / or mask the data collected by edge device 1012. For example, wearable device 1002 may collect health data (e.g., user location, heart rate, miles per hour, exercise duration), and machine learning applications implemented by wearable device 1002 may abstract and / or mask certain values ​​in the health data (e.g., precise location).

[0078] In some instances, cloud computing system 1006 may implement one or more machine learning applications, such as applications 300, 320, 600, and / or 620, or portions thereof. For example, cloud computing system 1006 may include a training application 1008 that generates training datasets and / or trains machine learning models. In some instances, abstracted and / or masked data may then be provided from edge device 1012 to training application 1008. Training application 1008 can use the abstracted and / or masked data from edge device 1012 to train the machine learning model. In this instance, because abstraction and / or masking are performed on edge device 1012, little or no sensitive data is transmitted by edge device 1012 and / or received by cloud computing system 1006. This can provide additional security for sensitive information in some applications.

[0079] In some instances, cloud computing system 1006 may include machine learning application 1010 that produces results based on input provided from edge device 1012. In some instances, machine learning application 1010 may implement a machine learning application, such as machine learning application 600, that suppresses the storage of results. In some instances, only unstored results (e.g., results with a confidence level equal to or below a threshold) are provided from cloud computing system 1006 to edge device 1012. In some applications, this reduces the risk that cloud computing system 1006 may release sensitive data and / or other information that could allow reverse engineering of machine learning application 1010.

[0080] The devices, systems, and methods disclosed herein enable more privacy-aware operations in machine learning models, applications, and / or systems. The devices, systems, and methods described herein can abstract and / or mask values ​​in data before feeding data to a machine learning model used for training. This can reduce or prevent machine learning models from storing sensitive information in some applications. Furthermore, the devices, systems, and methods of this disclosure can analyze the confidence level associated with the results from the machine learning model. If the confidence level is too high, the results may not be provided as output. Under certain conditions, abstracting and / or masking the data used to train the machine learning model and / or not providing results from the machine learning model can reduce or prevent the exposure of sensitive data and / or reverse engineering of the machine learning model, training methods, and / or training data. In some applications, this can improve the protection of privacy for individuals and / or entities.

[0081] The foregoing description of certain embodiments is merely exemplary in nature and is in no way intended to limit the scope of this disclosure or its application or use. In this detailed description of embodiments of the devices, systems, and methods of the present invention, reference is made to the accompanying drawings, which form part of the invention, and illustratively illustrate specific embodiments in which the described devices, systems, and methods may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the devices, systems, and methods disclosed herein, and it should be understood that other embodiments may be utilized, and structural and logical changes may be made without departing from the spirit and scope of this disclosure. Furthermore, for clarity, certain features will not be discussed in detail when they are obvious to those skilled in the art, so as not to obscure the description of embodiments of this disclosure. Therefore, the discussions herein should not be construed in a limiting sense, and the scope of this disclosure is defined by the appended claims. As used herein, the term "device" may refer to a circuit, apparatus, system, component, or combination thereof. For example, a device may be a computing device, processor, memory, memory device, mobile device, edge device, server, and / or cloud computing system.

[0082] Of course, it should be understood that any of the examples, embodiments, or processes described herein may be combined with or separated from one or more other examples, embodiments, and / or processes and / or performed in a separate apparatus or apparatus portion of a system, apparatus, or method according to the invention.

[0083] Finally, the foregoing discussion is intended to be illustrative only and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Therefore, while various embodiments of this disclosure have been described in particular detail, it should be understood that numerous modifications and alternative embodiments can be devised by those skilled in the art without departing from the broader and contemporaneous spirit and scope of this disclosure as set forth in the appended claims. Consequently, the specification and drawings should be viewed in an illustrative manner and are not intended to limit the scope of the appended claims.

Claims

1. A memory device comprising: a processor; and a non-transitory computer-readable medium configured to store input received from another device, wherein the non-transitory computer-readable medium is encoded with instructions that, when executed by the processor, cause the processor to: generate, using a machine learning model, a first value indicative of a result and a second value indicative of a confidence level of the result from the input; compare the second value to a first threshold, wherein the first threshold is indicative of the first value that includes a third value stored from a training data set used to train the machine learning model, wherein the stored third value includes sensitive information; compare the second value to a second threshold, wherein the second threshold is indicative of an acceptable reliability level of the result; and provide an output to the other device based on the comparison to the first threshold, wherein the output excludes at least a portion of the sensitive information.

2. The memory device of claim 1, wherein the output includes the result when the second value is equal to or less than the first threshold.

3. The memory device of claim 1, wherein the output includes an error signal when the second value is equal to or greater than the first threshold.

4. The memory device of claim 1, wherein the other device is a mobile device.

5. The memory device of claim 1, wherein the machine learning model is trained using a modified training data set, wherein the modified training data set includes at least one of a masked value or an abstracted value.

6. The memory device of claim 5, wherein at least some of the data from the modified training data set is received from the other device.

7. A method for operating a memory device comprising: receiving a data input; generating, using a machine learning model, a first value indicative of a result and a second value indicative of a confidence level of the result from the data input; comparing the second value to a first threshold, wherein the first threshold is indicative of the first value that includes a third value stored from a training data set used to train the machine learning model, wherein the stored third value includes sensitive information; comparing the second value to a second threshold, wherein the second threshold is indicative of an acceptable reliability level of the result; and providing an output based on the comparison to the first threshold, wherein the output excludes at least a portion of the sensitive information.

8. The method of claim 7, wherein the output includes an error signal when the second value is equal to or higher than the first threshold.

9. The method of claim 7, further comprising: when the second value is equal to or higher than the first threshold: analyzing the data input with the machine learning model to generate a fourth value indicative of a second result and a fifth value indicative of a second confidence level, wherein the fifth value is lower than the first threshold; and providing the fourth value as the output.

10. The method of claim 7, wherein the output includes the first value when the second value is equal to or below the first threshold.

11. The method of claim 7, wherein the second value is generated based at least in part on a flexible maximum algorithm.

12. The method of claim 7, wherein the second value is generated based at least in part on a distribution of the first value.

13. The method of claim 7, further comprising training the machine learning model using a modified training data set, wherein the modified training data set includes at least one of a masked value or an abstracted value.

14. A memory device comprising: a processor; and a non-transitory computer-readable medium configured to store data, wherein the non-transitory computer-readable medium is encoded with instructions that, when executed by the processor, cause the processor to: generate a first value indicative of a rank, wherein the first value indicates a number of times one or more values are included in the data; generate a second value indicative of a classification, wherein the second value indicates one or more data types included in the data; change a value of the one or more values included in the data based on the first value and the second value, wherein the first value, the second value, or a combination thereof indicates sensitive information of the value; generate a training data set comprising the data including the changed value, wherein the changed value excludes at least a portion of the sensitive information of the value; and train a machine learning model with the training data set.

15. The memory device of claim 14, wherein the instructions implement a second machine learning model to generate the second value.

16. The memory device of claim 15, wherein the second machine learning model is a neural network.

17. The memory device of claim 14, wherein the instructions implement a rule-based classification system to generate the second value.

18. The memory device of claim 14, wherein the value of the one or more values is changed to an abstracted value when the second value indicates the data type of the value is sensitive data and the first value indicates the number of times the value is included in the data is equal to or above a threshold.

19. The memory device of claim 14, wherein the value of the one or more values is changed to a masked value when the second value indicates the data type of the value is sensitive data and the first value indicates the number of times the value is included in the data is equal to or below a threshold.

20. A training method for operating a memory device, comprising: receiving data; generating a first value indicative of a rank, wherein the first value indicates a number of times one or more values are included in the data; generating a second value indicative of a classification, wherein the second value indicates one or more data types included in the data; changing a value of the one or more values included in the data based on the first value and the second value, wherein the first value, the second value, or a combination thereof indicates sensitive information of the value; generating a training dataset comprising the data including the altered values, wherein the altered values exclude at least a portion of the sensitive information of the values; and training a machine learning model with the training dataset.

21. The training method of claim 20, wherein altering the values comprises masking the values when the second value indicates that the data type of the values is sensitive data and the first value indicates that the number of times the values are included in the data is at or below a threshold.

22. The training method of claim 21, wherein the sensitive data includes at least one of a proprietary name, a date, an address, a password, a birth date, an account number, or a username.

23. The training method of claim 20, wherein altering the values comprises abstracting the values when the second value indicates that the data type of the values is sensitive data and the first value indicates that the number of times the values are included in the data is at or above a threshold.

24. The training method of claim 20, wherein the second value is generated by a rule-based classification.

25. The training method of claim 20, wherein the second value is generated by a machine learning model.

Images