Encoded host-to-dla traffic

Abstract

Devices and methods can relate to encoding traffic between a host and a DLA. Traffic between a host and a DLA can be encoded with an autoencoder. Encoding traffic between a host and a DLA changes a bandwidth of the traffic. Changing the bandwidth of the traffic prevents a correlation between the bandwidth and an input from which the traffic is generated.

Description

Technical Field

[0001] This disclosure generally relates to memory, and more specifically, to apparatus and methods associated with encoding traffic between a host and a deep learning accelerator (DLA). Background Technology

[0002] Memory devices are typically provided as internal semiconductor integrated circuits in computers or other electronic devices. Many different types of memory exist, including volatile and non-volatile memory. Volatile memory requires power to maintain its data and includes Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), and Synchronous Dynamic Random Access Memory (SDRAM), among others. Non-volatile memory provides persistent data by retaining stored data when no power is supplied and includes NAND flash memory, NOR flash memory, Read-Only Memory (ROM), Electrically Erasable Programmable Memory (EEPROM), Erasable Programmable Memory (EPROM), and Resistive Variable Memory, such as Phase-Change Random Access Memory (PCRAM), Resistive Random Access Memory (RRAM), and Magnetoresistive Random Access Memory (MRAM), among others.

[0003] Memory also serves as a volatile and non-volatile data storage device for a wide range of electronic applications, including but not limited to personal computers, portable memory sticks, digital cameras, cellular phones, portable music players (e.g., MP3 players), movie players, and other electronic devices. Memory cells can be arranged in an array, wherein the array is used in a memory device. Summary of the Invention

[0004] On one hand, this disclosure provides an apparatus for encoding services between a host and a deep learning accelerator, comprising: a memory cell array; the deep learning accelerator (DLA) coupled to the array; and a controller coupled to the array and the DLA, wherein the controller is configured to: receive encoded data from the host; store the encoded data in the array; and wherein the DLA is configured to: access the encoded data from the array in response to receiving a signal from the controller; decode the encoded data using an autoencoder, wherein the encoded data is decoded by a decoder of the autoencoder implemented by the DLA to produce decoded data, and wherein the decoder of the autoencoder is implemented as an artificial neural network (ANN); and perform a plurality of operations on the decoded data.

[0005] On the other hand, this disclosure further provides a method for encoding services between a host and a deep learning accelerator (DLA), comprising: receiving, at the deep learning accelerator (DLA) in the memory device, signaling from the host indicating first data including hyperparameters for configuring an artificial neural network (ANN), wherein the ANN is implemented by the DLA; receiving, at the DLA in the memory device, encoded second data from the host, wherein the second data is generated by a plurality of sensors and encoded by an encoder of an autoencoder implemented by the host; decoding the encoded second data to produce decoded second data, wherein the decoding includes utilizing a decoder of the autoencoder and wherein the decoder is implemented by the DLA; processing the decoded second data using the ANN; and transmitting the encoded output of the ANN to the host.

[0006] In another aspect, this disclosure further provides a system for encoding services between a host and a DLA, comprising: the host; a memory device including a deep learning accelerator (DLA) and a memory cell array, wherein the DLA includes a decoder of an autoencoder and an artificial neural network (ANN); wherein the host is configured to: encode a first set of data to generate encoded data, wherein the first set of data is encoded by an encoder of the autoencoder, wherein the encoder is implemented by the host; and provide the encoded data from the host to the DLA in the memory device such that the DLA uses the ANN to process a second set of data corresponding to the first set of data; wherein the DLA is configured to: decode the encoded data using the decoder of the autoencoder to generate the second set of data corresponding to the first set of data; and process the second set of data using the ANN. Attached Figure Description

[0007] Figure 1 This is a block diagram of a device in the form of a computing system including a memory device, according to several embodiments of the present disclosure.

[0008] Figure 2 A block diagram illustrating an automatic encoder according to several embodiments of the present disclosure.

[0009] Figure 3A A block diagram illustrating the services between a host and a DLA according to several embodiments of the present disclosure.

[0010] Figure 3B A block diagram illustrating the services between a host and a DLA according to several embodiments of the present disclosure.

[0011] Figure 4 A block diagram illustrating the services between a host and a DLA according to several embodiments of the present disclosure.

[0012] Figure 5 Example flowcharts illustrating methods for encoding services between a host and a DLA according to several embodiments of the present disclosure.

[0013] Figure 6 An example machine illustrating a computer system may execute a set of instructions within said computer system to cause said machine to perform the various methods discussed herein. Detailed Implementation

[0014] This disclosure includes apparatus and methods related to encoding traffic between a host and a deep learning accelerator (DLA). The DLA may be implemented in a memory device.

[0015] As used in this paper, artificial neural networks (ANNs) provide learning by forming probabilistic weight associations between inputs and outputs. These probabilistic weight associations can be provided by multiple nodes comprising the ANN. These nodes, along with weights, biases, and activation functions, are used to produce the ANN's output based on its inputs. Multiple nodes of an ANN can be grouped to form layers of the ANN.

[0016] For example, inexpensive and energy-efficient artificial intelligence (AI) accelerators for deep learning accelerators (DLAs) can be implemented at the edge of a memory device and are referred to herein as “DLAs” or “edge DLAs”. As used herein, the edge of a memory device can refer to a region of the memory device other than the memory device region occupied by the memory array and / or sensing circuitry system. For example, the edge of a memory device can refer to the output and / or input paths of the memory device, where the output and / or input paths are used to store and / or retrieve data from the memory array of the memory device.

[0017] As used in this article, AI refers to the ability of a device to improve itself through “learning,” such as by storing patterns and / or instances that can be used to take action later. Deep learning refers to the ability of a device to learn from data provided as instances. Deep learning can be a subset of AI. Neural networks and other types of networks can be classified as deep learning. The low-power and inexpensive design of DLA can be implemented in Internet of Things (IoT) devices or storage devices and other types of devices. Edge DLA can process and make intelligent decisions at runtime. Storage devices containing edge DLA can also be deployed in remote locations without cloud or offloading capabilities.

[0018] A DLA implemented in a memory device can receive signals from and cause signals to be transmitted to the host, which is coupled to the memory device via an interface. However, the signals transmitted between the DLA and the host may be observed by a third party. As used herein, the signals transmitted between the DLA and the host may be referred to as services. These signals may include data files and commands, as well as other types of services that can be communicated between the host and the DLA.

[0019] For example, safety-critical applications used in autonomous vehicles are targets of side-channel attacks. As used herein, side-channel attacks exploit vulnerabilities to extract privileged data from memory devices that would otherwise be unavailable. For example, in the context of applications used in autonomous vehicles, privileged data might include the number of nearby vehicles, the current road scene (e.g., an image), the locations of various objects near the vehicle, and other instances of privileged data.

[0020] Charter data can be correlated with resource utilization. For example, charter data can be correlated with the bandwidth of node interconnects. As used in this paper, node interconnects describe how nodes connect to different nodes. For example, node interconnects can describe how nodes in one layer connect to nodes in different layers.

[0021] In various examples, the bandwidth utilization of a bus connecting the DLA to a host (e.g., a host processor), such as a PCIe bus, can depend on the data moving across that bus. When operations are performed on the DLA, the data transmitted via that bus (e.g., signals) can be correlated with the input data provided to the DLA. A secondary model can utilize the relationship between bandwidth utilization and input data to predict input data characteristics and / or predict the output provided by the DLA. Therefore, the input data, bandwidth utilization, and output data can be proprietary data.

[0022] For example, licensed data can be used to influence the autonomous decision-making of self-driving software in a timely manner. To overcome the security concerns of side-channel attacks, the information available and / or visible to the observer may be limited.

[0023] This disclosure addresses the above and other shortcomings. These shortcomings can be overcome by providing input data via a deep learning-based encoding and / or obfuscation method through the interface connecting the DLA to the host. Deep learning-based encoding and / or obfuscation can be performed in real time. Compression caused by encoding and / or obfuscation may affect services provided through the interface by coupling the memory device to the host. As used herein, obfuscation can describe changes to services to alter the size of the service between the host and the memory device. For example, obfuscation can add bits to the service. Changes to the service provided through the interface can refute attempts to identify input and / or output data from the interface service. Given the power consumption and latency issues involved in encoding and / or obfuscating interface services, the level of encoding and / or obfuscation can be configured.

[0024] The figures in this document follow a numbering convention, where the first few digits correspond to the figure number and the remaining digits identify the elements or components in the figure. Similar elements or components between different figures can be identified by using similar digits. For example, 115-1 can be referenced. Figure 1 The element "15" in the text, and similar elements can be referenced as Figure 2 See section 215-1. Similar elements within the figure may be referenced using hyphens and additional numbers or letters. For example, see [reference 1]. Figure 1 Components 115-1 and 115-2 are shown in the figures. It will be understood that components shown in the various embodiments herein may be added, interchanged, and / or eliminated to provide several additional embodiments of this disclosure. Furthermore, it will be understood that the scale and relative dimensions of the components provided in the figures are intended to illustrate certain embodiments of the invention and should not be construed as limiting.

[0025] Figure 1 This is a block diagram of a device in the form of a computing system 100 including a memory device 103, according to several embodiments of the present disclosure. As used herein, for example, the memory device 103, the memory array 110, and / or the host 102 may also be individually considered as a “device”.

[0026] In this example, computing system 100 includes a host 102 coupled to memory device 103 via interface 104. Computing system 100 can be a personal laptop, desktop computer, digital camera, mobile phone, memory card reader, or Internet of Things (IoT) enabled device, as well as various other types of systems. Host 102 may include several processing resources (e.g., one or more processors, microprocessors, or some other type of control circuitry) capable of accessing memory device 103, said processing resources being referred to as processor 116. Computing system 100 may include a separate integrated circuit, or both host 102 and memory device 103 may be on the same integrated circuit. For example, host 102 may be a system controller for a memory system including multiple memory devices 103, wherein said system controller provides access to the respective memory devices 103 through another processing resource, such as a central processing unit (CPU).

[0027] exist Figure 1In the example shown, host 102 is responsible for executing an operating system (OS) and / or various applications that can be loaded onto it (e.g., from memory device 103 via control circuitry 105). The OS and / or various applications can be loaded from memory device 103 by providing access commands from host 102 to memory device 103 to access data including the OS and / or various applications. Host 102 can also access data used by the OS and / or various applications to retrieve said data used in the execution of the OS and / or various applications by providing access commands to memory device 103.

[0028] For clarity, computing system 100 has been simplified to focus on features particularly relevant to this disclosure. For example, memory array 110 may be a DRAM array, SRAM array, STT RAM array, PCRAM array, TRAM array, RRAM array, NAND flash memory array, NOR flash memory array, and / or 3D cross-dot array. Memory array 110 may include memory cells arranged in rows coupled by access lines (which may be referred to herein as word lines or select lines) and columns coupled by sense lines (which may be referred to herein as digital lines or data lines). Although memory array 110 is shown as a single memory array, memory array 110 may represent multiple memory arrays arranged in the memory bank of memory device 103.

[0029] Memory device 103 includes an address circuitry 106 for latching address signals provided via interface 104. The interface may include, for example, a physical interface employing a suitable protocol (e.g., a data bus, address bus, and command bus, or a combined data / address / command bus). This protocol may be custom or proprietary, or interface 104 may employ a standardized protocol, such as Peripheral Component Interconnect Fast (PCIe), Gen-Z Interconnect, Cache Coherent Interconnect for Accelerators (CCIX), etc. Address signals are received and decoded by row decoder 108 and column decoder 112 to access memory array 110. Data can be read from memory array 110 by sensing voltage and / or current changes on a sensing line using sensing circuitry 111. Sensing circuitry 111 may be coupled to memory array 110. Each memory array and its corresponding sensing circuitry may constitute a storage bank of memory device 103. Sensing circuitry 111 may include, for example, sensing amplifiers capable of reading and latching pages (e.g., rows) of data from memory array 110. I / O circuitry 107 can be used for bidirectional data communication with host 102 via interface 104. Read / write circuitry 113 is used to write data to or read data from memory array 110. As an example, read / write circuitry 113 may include various drivers, latching circuitry, etc.

[0030] Control circuitry system 105 decodes signals provided by host 102. These signals may be commands provided by host 102. These signals may include chip enable signals, write enable signals, and address latch signals for controlling operations performed on memory array 110, including data read operations, data write operations, and data erase operations. In various embodiments, control circuitry system 105 is responsible for executing instructions from host 102. Control circuitry system 105 may include state machines, sequencers, and / or some other type of control circuitry implemented in hardware, firmware, or software, or any combination of these. In some instances, host 102 may be a controller external to memory device 103. For example, host 102 may be a memory controller coupled to the processing resources of a computing device. Data may be provided to and / or from memory array 110 via data lines that couple memory array 110 to I / O circuitry system 107.

[0031] The memory device (e.g., a memory subsystem) may also include a DLA 114. The DLA may be implemented at the edge of the memory device. For example, the DLA 114 may be implemented externally to the memory array 110. The DLA 114 may be coupled to an output path that couples the memory array 110 to the I / O circuitry system 107.

[0032] DLA 114 can also be coupled to control circuitry 105. Control circuitry 105 can control DLA 114. For example, control circuitry 105 can provide signaling to row decoder 108 and column decoder 112 to cause data to be transferred from memory array 110 to DLA 114 to provide input to DLA 114. Control circuitry 105 can also cause the output of DLA 114 to be provided to I / O circuitry 107 and / or stored back into memory array 110.

[0033] The host 102 can host encoder 115-1 and / or decoder 116-1. The DLA 114 can host encoder 115-2, decoder 116-2 and ANN 117. The DLA 114 can be controlled by the control circuit system 105 to execute ANN 117, encoder 115-2 and / or decoder 116-2.

[0034] As used herein, encoders 115-1, 115-2 and decoders 116-1, 116-2 include hardware and / or firmware for implementing ANNs for encoding or decoding input, respectively. Encoder 115-1 and decoder 116-2 may include a first auto encoder. Encoder 115-2 and decoder 116-1 may include a second auto encoder. The first auto encoder can be used to encode and decode traffic provided from host 102 to DLA 114. The second auto encoder can be used to encode and decode traffic provided from DLA 114 to host 102. ANN 117 may include hardware and / or firmware for implementing ANNs to perform operations on decoded data.

[0035] As used herein, decoding or encoding describes the process of converting data from one format to another. As used herein, the terms decoding and encoding utilize cryptography to enable data to be encoded from plaintext to encoded formats and to be decoded from encoded formats back to plaintext. Plaintext format describes the format in which data is generated or prepared for processing.

[0036] In various instances, host 102 can access data. For example, the data may be generated by multiple sensors of the vehicle. For example, the vehicle's cameras may generate multiple images. Host 102 may encode the data using encoder 115-1. Encoder 115-1 can generate encoded data. The encoded data may be provided to memory device 103 via interface 104. Memory device 103 may provide the encoded data to DLA 114.

[0037] DLA 114 can use decoder 116-2 to decode the encoded data. Decoder 116-2 can produce decoded data. In various examples, the data and the decoded data can be similar but not identical. The decoded data can retain many of the essential characteristics of the data, but can have a different size than the data. For example, the decoded data can be smaller than the data.

[0038] DLA 114 may provide decoded data to ANN 117. ANN 117 may, for example, process the decoded data to produce an output. In the case of an image, the output may be a classification. ANN 117 may perform operations on decoded data requested by host 102. As used herein, performing operations in the context of ANN 117 may involve propagating the decoded data through multiple nodes to produce an output.

[0039] The output generated by ANN 117 can be provided to encoder 115-2 to produce encoded output. The encoded output can be provided to host 102 via interface 104. In various examples, the output generated by ANN 117 can be provided directly to host 102 without being encoded.

[0040] Host 102 can use decoder 116-1 to decode the encoded output to produce a decoded output. In various examples, host 102 can use the decoded output to perform further operations.

[0041] In various examples, DLA 114 may cause the encoded output, or said output, to be stored in memory array 110. Host 102 may retrieve the encoded output, or said output, from memory array 110.

[0042] In several instances, host 102 may configure DLA 114, encoder 115-2, decoder 116-2, and / or ANN 117. For example, host 102 may provide multiple hyperparameters to memory device 103 to configure DLA 114, which includes encoder 115-2, decoder 116-2, and / or ANN 117. These hyperparameters may include the number of layers, the number of nodes, the number of nodes per layer, multiple weights, multiple biases and / or activation functions, and other possible parameters that can be used to configure the ANN.

[0043] Host 102 may encode the hyperparameters before providing them to DLA 114. In other instances, host 102 may provide the hyperparameters to DLA 114 without encoding them. If the hyperparameters are encoded, host 102 may use encoder 115-1 to generate encoded hyperparameters. Host 102 may provide the encoded hyperparameters to DLA 114. DLA 114 may use decoder 116-2 to decode the encoded hyperparameters. Decoder 116-2 may use default hyperparameters to decode the encoded hyperparameters. Decoder 116-2 may generate decoded hyperparameters. DLA 114 may use the decoded hyperparameters to configure decoder 116-2, encoder 115-2, and / or ANN 117.

[0044] The host 102 is provided with a unique key to configure encoder 115-2 and / or decoder 116-2. The key used to encode or decode interface traffic may be the hyperparameters themselves or may be based on the hyperparameters used to configure encoders 115-1, 115-2 and decoders 116-1, 116-2.

[0045] although Figure 1The example shown provides encoder 115-2 and decoder 116-2 as implemented in DLA 114, but encoder 115-2 and decoder 116-2 may be implemented in different parts of memory device 103. For example, encoder 115-2 and decoder 116-2 may be implemented in control circuit system 105 and / or I / O circuit system 107 and other locations within memory device 103.

[0046] The example described herein provides a DLA 114 to perform multiple actions, such as decoding data using decoder 116-2. Any action described as being performed by DLA 114 can actually be performed by control circuitry 105. For example, control circuitry 105 may receive a command from host 102 to decode encoded data received via interface 104. Control circuitry 105 may control DLA 114 such that DLA 114 uses decoder 116-2 to decode the encoded data.

[0047] In various examples, host 102 may provide commands, along with the encoded data, to cause the encoded data to be decoded. Host 102 may also provide commands, along with the encoded hyperparameters, to cause DLA 114 to decode the hyperparameters. Host 102 may further provide commands to cause DLA 114 to perform operations using ANN 117 and the decoded data. Host 102 may further provide commands to cause DLA 114 to configure encoder 115-2, decoder 116-2, and / or ANN 117 using the hyperparameters.

[0048] Figure 2 Block diagrams illustrating automatic encoders 220-1 and 220-2 according to several embodiments of the present disclosure. Figure 2 Autoencoders 220-1 and 220-2 are illustrated as neural networks. As used herein, autoencoders 220-1 and 220-2 are referred to herein as autoencoder 220. Autoencoder 220-1 includes encoder network 215-1 and decoder network 216-1. Input can be provided to encoder network 215-1. Encoder network 215-1 can produce encoded data 223-1 (e.g., encoded information). Decoder network 216-1 can receive encoded data 223-1 and can produce output 222-1. Autoencoder 220-2 can receive input 221-2. Autoencoder 220-2 can include decoder network 216-2, encoded data 223-2 (e.g., encoded information), and encoder network 215-2. Autoencoder 220-2 can produce output 222-2.

[0049] As used herein, encoder networks 215-1 and 215-2 may be referred to as encoders 215-1 and 215-2, respectively. Decoder networks 216-1 and 216-2 may be referred to as decoders 216-1 and 216-2, respectively. Encoder networks 215-1 and 215-2 and decoder networks 216-1 and 216-2 may be composed of artificial neural networks such that autoencoders 220-1 and 220-2 are composed of artificial neural networks. As used herein, autoencoders 220-1 and 220-2 consist of multiple layers. Each of the multiple layers consists of multiple nodes. Nodes in each of the layers are coupled to other nodes in other layers. Each of autoencoders 220-1 and 220-2 may be implemented by a host and a memory device. For example, encoder network 215-1 of autoencoder 220-1 may be implemented by a host. Decoder network 216-1 of autoencoder 220-1 may be implemented by a memory device.

[0050] The encoder network 215-2 of the automatic encoder 221-2 can be implemented by a memory device. The decoder network 216-2 of the automatic encoder 221-2 can be implemented by a memory device. The decoder network 216-1 and the encoder network 215-2 can be implemented by... Figure 1 The implementation of DLA 114.

[0051] The host computer can access input 221-1. The host computer can provide input 221-1 to encoder network 215-1. Input 221-1 can be referred to as data. The host computer can implement encoder network 215-1 by executing multiple computer-executable instructions (e.g., software) representing encoder network 215-1. The host computer can process input 221-1 using the multiple computer-executable instructions. Encoder network 215-1 can produce output. The output can be encoded data 223-1. Therefore, the output of encoder network 215-1 can be referred to as encoded output.

[0052] Encoded data 223-1 can be transmitted via an interface that couples the host to a memory device. The memory device can receive encoded data 223-1. Encoded data 223-1 can be provided to the DLA. Encoded data 223-1 can be processed by a decoder network 216-1. Decoder network 216-1 can be implemented as firmware. Decoder network 216-1 can decode encoded data 223-1 to produce output 222-1. Output 222-1 can be referred to as decoded data.

[0053] The data including input 221-1 and the decoded data including output 222-1 may be substantially similar but not identical. For example, the decoded data may be smaller in size than the data. The size difference between the decoded data and the data may be a result of the encoding and decoding processes, in which information loss occurs. Information loss may be reflected in the size of the decoded data. The decoded data may retain many characteristics of the data. The encoder network 215-1 and the decoder network 216-1 may be configured such that the decoded data retains the desired characteristics of the data.

[0054] In various examples, the hyperparameters of encoder network 215-1 can be considered as a first key and the hyperparameters of decoder network 216-1 can be considered as a second key. Different hyperparameters can be used to configure encoder network 215-1 and decoder network 216-1 so that different keys can be used to encode and decode input 221-1 and encoded data 223-1. In various examples, the hyperparameters of encoder network 215-1 and decoder network 216-1 can be the same so that encoder network 215-1 and decoder network 216-1 share a key.

[0055] The memory device can process output 222-1. For example, the DLA can provide output 222-1 as input to an ANN separate from the decoder network 216-1 and encoder network 215-2. The output from the ANN can be provided as input 221-2 to the encoder network 215-2. The encoder network 215-2 implemented in the DLA of the memory array can process input 221-2 to generate encoded data 223-2. The encoded data 223-2 can be provided to the host via an interface that couples the host to the memory device. The host can receive the encoded data 223-2. The host can use the decoder network 216-2 to decode the encoded data 223-2. The decoder network 216-2 can generate output 222-2 from the encoded data 223-2. Output 222-2 can be decoded data.

[0056] In various examples, the memory device may cause the encoded data 223-2 to be stored in the memory array. The host may retrieve the encoded data 223-2 from the memory array by causing it to be retrieved via an interface that couples the host to the memory device. The host may decode the encoded data 223-2 as previously described.

[0057] The memory device can also cause the ANN's output (e.g., inputs 221-2) to be stored in the memory array. The host can retrieve the ANN's output from the memory array by causing the ANN's output to be retrieved via an interface that couples the host to the memory device. The host can utilize the ANN's output without needing to decode it.

[0058] Figure 3A A block diagram illustrating services between host 302-1 and DLA 314-1 according to several embodiments of the present disclosure is provided. Host 302-1 and DLA 314-1 provide a first instance of encoding and decoding data.

[0059] Host 302-1 and DLA 314-1 demonstrate symmetric encoding and decoding. As used herein, symmetric encoding describes the use of the same key to encode and decode traffic between host 302-1 and DLA 314-1. The use of the same key is illustrated by the shading of encoder 315-1 and decoder 316-1. Encoder 315-1 and decoder 316-1 are shown with the same shading to indicate that encoder 315-1 and decoder 316-1 share the same key used for encoding and decoding.

[0060] The keys used for encoding and decoding may consist of the weights, biases, and / or activation functions of the encoder 315-1 and decoder 316-1, as well as other hyperparameters.

[0061] Although the keys for encoder 315-1 and decoder 316-1 are described as the same key, the implementation of said key may differ. For example, the weights of the first layer of encoder 315-1, which may include the key of encoder 315-1, may be different from the weights of the first layer of decoder 316-1, which may include the key of decoder 315-1. However, the weights of the first layer of encoder 315-1 may be the same as the weights of the last layer of decoder 316-1, and the weights of the last layer of encoder 315-1 may be the same as the weights of the first layer of decoder 316-1.

[0062] The key used by encoder 315-1 and decoder 316-1 can be the same key used by encoder 315-2 and decoder 316-2. For example, the key used by encoder 315-1 can be the same key used by encoder 315-2. The key used by decoder 316-1 can be the same key used by decoder 316-2. Using the same key in encoders 315-1, 315-2 and decoders 316-1, 316-2 provides asymmetric encoding / decoding between an autoencoder containing encoder 315-1 and decoder 316-1 and an autoencoder containing encoder 315-2 and decoder 316-2.

[0063] The host 302-1 can configure encoders 315-1, 315-2 and decoders 316-1, 316-2 as symmetric by programming encoders 315-1, 315-2 and decoders 316-1, 316 with the same hyperparameters including the same key.

[0064] Figure 3BA block diagram illustrating the services between host 302-2 and DLA 314-2 according to several embodiments of the present disclosure is provided. Host 302-2 and DLA 314-2 provide a second instance of encoding and decoding data.

[0065] Host 302-2 and DLA 314-2 demonstrate asymmetric encoding and decoding. As used herein, symmetric encoding describes the use of different keys to encode and decode traffic between host 302-2 and DLA 314-2. Encoder 315-3 and decoder 316-3 are shown with the same shading to indicate that encoder 315-3 and decoder 316-3 share the same key used for encoding and decoding. Encoder 315-4 and decoder 316-4 are shown with different shading than those used for encoder 315-3 and decoder 316-3, but with the same shading to indicate that different keys are used for encoders 315-3, 315-4 and decoders 316-3, 316-4. The keys used for encoding and decoding may consist of the weights, biases, and / or activation functions of encoders 315-3, 315-4 and decoders 316-3, 316-4, as well as other hyperparameters.

[0066] The host 302-2 can configure encoders 315-3, 315-4 and decoders 316-3, 316-4 as asymmetric by programming encoders 315-3, 315-4 and decoders 316-3, 316 with different hyperparameters, including the number of layers, the number of nodes in each layer, multiple weights, multiple biases and activation functions for DLA.

[0067] Figure 4 Block diagrams illustrating services between a host and a DLA according to several embodiments of the present disclosure are provided. In various instances, input 421 can be processed such that the input is a packet stream comprising packets 442-1, 442-2, ..., 442-N, collectively referred to as packets 442.

[0068] Each of the groups 442 can be encoded by an encoder 415. For example, the encoder 415 of an auto encoder can encode group 442-1 to produce encoded group 423-1, encode group 442-2 to produce encoded group 423-2, ..., and encode group 443-N to produce encoded group 423-N. Encoded groups 423-1, 423-2, ..., 423-N can be referred to as encoded group 423. The encoded groups can be provided via an interface. Encoded group 423 can be decoded by a decoder 416 of an auto encoder. For example, the decoder can decode encoded group 423-1 to produce decoded group 443-1, decode encoded group 423-2 to produce decoded group 443-2, ..., and decode encoded group 423-N to produce decoded group 443-N.

[0069] In various examples, the host and / or memory device may insert obfuscation mark 441 into the encoded packet 423. For example, an automatic encoder may insert obfuscation mark 441. The host may configure decoder 416 to remove obfuscation mark 441 from the encoded packet 423 before decoding the encoded packet 423. In various examples, the scheme for inserting obfuscation mark 441 may be known to the host and memory device such that obfuscation mark 441 can change the size of traffic between the host and the memory device without changing the size of the decoded packet 443. The host may configure encoder 415 and / or decoder 416 to identify obfuscation mark 441 or the placement of obfuscation mark 441 within the order of the encoded packet 423.

[0070] Figure 5 This illustration provides example flowcharts of a method 580 for compiling instructions to implement a bypass path in an artificial neural network according to several embodiments of the present disclosure. Method 580 may be executed by processing logic, which may include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, method 580 is controlled by a control circuitry (e.g., a controller) 105, by a host 102, and / or by... Figure 1 The DLA 114 is executed. Although shown in a specific sequence or order, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be executed in different orders, and some processes may be executed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0071] At block 581, signaling indicating first data including hyperparameters can be received from the host at the DLA of the memory device to configure the ANN, wherein the ANN is implemented by the DLA. The hyperparameters of the ANN are accessible. For example, the hyperparameters may be retrieved from the memory array or received from a different device. The hyperparameters may be provided from the host to the DLA of the memory device to configure the ANN.

[0072] At box 582, encoded second data can be received from the host at the DLA of the memory device. The second data may be generated by multiple sensors. The second data may be encoded by an encoder of an automatic encoder implemented by the host. The second data generated by multiple sensors can be encoded to produce encoded data.

[0073] At box 583, the encoded second data can be decoded to produce decoded second data. Decoding may include a decoder utilizing an autoencoder. The decoder may be implemented by a DLA.

[0074] At box 584, the ANN can be used to process the decoded data. At box 585, the encoded output of the ANN can be transmitted to the host. The encoded output of the ANN can be received at the host. Different decoders from different autoencoders can be used to decode the encoded output. Different decoders can be implemented by the host.

[0075] The hyperparameters may include the number of layers, the number of nodes in each layer, multiple weights, multiple biases, and an activation function for the DLA. The hyperparameters may be encoded before being provided to the host. The hyperparameters may be encoded using an autoencoder. Signaling indicating the receipt of first data including the hyperparameters may further include receiving the encoded hyperparameters.

[0076] The encoded output of the ANN can be transmitted to the host computer so that the host computer can decode the encoded output. The encoded output can be decoded using different decoders from different autoencoders, with the different decoders implemented by the host computer. That is, a first decoder, different from the second decoder, can be implemented by a memory device, while the second decoder is implemented by the host computer.

[0077] Encoded data can be provided via an interface that couples the host to the memory device. This interface can be, for example, a PCIe interface or other suitable interfaces.

[0078] The host encoder and the DLA decoder can share a first set of hyperparameters. That is, the host encoder and the DLA decoder can share the same key. Different encoders of the DLA and different decoders of the host share a second set of hyperparameters. That is, different encoders of the DLA and different decoders of the host can also share the same key, where the second set of hyperparameters is the same key. The first set of hyperparameters and the second set of hyperparameters are the same set of hyperparameters, which makes the encoder and the different encoders share the same key, and the decoder and the different decoders share the same key.

[0079] The host encoder and the DLA decoder can share a first set of hyperparameters. That is, the host encoder and the DLA decoder can share the same key. Different encoders of the DLA and different decoders of the host share a second set of hyperparameters. That is, different encoders of the DLA and different decoders of the host can also share the same key, where the second set of hyperparameters is the same key. However, the first set of hyperparameters and the second set of hyperparameters can be different hyperparameters such that the encoder and the different encoders do not share the same key, and the decoder and the different decoders do not share the same key.

[0080] In several embodiments, encoded data can be received from a host and a controller in the memory device. The controller can store the encoded data in an array of memory devices. The controller can control a DLA to access the encoded data from the array in response to a signal received from the controller. The encoded data can be decoded using an autoencoder implemented by the DLA. The encoded data is decoded by a decoder of the autoencoder to produce decoded data. The decoder of the autoencoder is implemented as an ANN. The ANN can be implemented by the DLA. Thus, the decoder and / or encoder can be configured by configuring the DLA to execute a specific ANN with hyperparameters. Multiple operations can be performed on the decoded data.

[0081] The DLA can utilize a decoder, including a first ANN, to decode encoded data. The DLA can also utilize a second ANN to perform multiple operations on the decoded data. The first ANN can be independent of the second ANN. The first ANN can be implemented using first firmware, while the second ANN can be implemented using second firmware. The DLA can be further configured to perform multiple operations on the decoded data to produce output. The output can be provided to a host.

[0082] The DLA can encode the output to produce an encoded output before it is provided to the encoding host. A controller for the memory device can provide the encoded output to the host. The output can be encoded using encoders from different autoencoders. The encoders from different autoencoders can be implemented in the DLA.

[0083] In various examples, the first set of data can be encoded to produce encoded data. The first set of data can be encoded by the encoder of the host's autoencoder. The encoder can be implemented by the host.

[0084] Encoded data can be provided from the host to the DLA of the memory device, causing the DLA to process the encoded data using an ANN. Control signals can be provided to the memory device to cause the memory device to process the encoded data. The DLA can decode the encoded data using a decoder of an autoencoder. The decoder can be implemented in the DLA. The decoder can decode the encoded data to generate a second set of data corresponding to the first set of data. The second set of data can be processed using an ANN. Using encoders of different autoencoders in the memory device, the output of the ANN can be encoded to produce encoded output. The encoded output can be stored in the memory device array or provided to the host.

[0085] Several obfuscation flags can be added to encoded data. Adding obfuscation flags can change the bandwidth of traffic between the host and the storage device. The host can provide the obfuscation flags to the storage device to allow the storage device to recognize the obfuscation flags. For example, the host can provide the obfuscation flags before providing encoded data including the obfuscation flags. Providing the obfuscation flags to the storage device allows the storage device to identify the obfuscation flags in the encoded data and remove the obfuscation flags from the encoded data before decoding the encoded data.

[0086] The aforementioned obfuscation flags can be used to decode the encoded data. For example, the obfuscation flags can only be removed from the data after the encoded data has been decoded. DLA can use the aforementioned obfuscation flags to further decode the encoded data.

[0087] The DLA can remove the plurality of obfuscation flags from the encoded data before using the decoder of the memory device to decode the encoded data. The host can add bits to the first set of data before encoding it. The memory device can remove bits from the first set of data after decoding it. The bits removed from the first set of data may correspond to the obfuscation flags. In various examples, the host can be configured to retrieve the encoded output from the memory array and decode the encoded output using different decoders from different autoencoders implemented by the host.

[0088] Figure 6 An example machine illustrating computer system 690 is described, within which a set of instructions can be executed to cause the machine to perform the various methods discussed herein. In various embodiments, computer system 690 may correspond to a system that may include, be coupled to, or utilize a memory subsystem (e.g., Figure 1The memory device 103) or the controller that can be used to execute (e.g., Figure 1 The system operating the controller circuit system 105) (e.g., Figure 1 (Computing system 100). In alternative embodiments, the machine may be connected (e.g., networked) to other machines in a LAN, intranet, extranet, and / or the Internet. The machine may operate as a server or client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment.

[0089] The machine may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), cellular phone, network infrastructure, server, network router, switch, or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) specifying the action to be taken by that machine. Furthermore, while a single machine is described, the term "machine" should also be understood to include any collection of machines that individually or jointly execute a set (or more sets) of instructions to perform any or more of the methods discussed herein.

[0090] Example computer system 690 includes processing devices 691 that communicate with each other via bus 696, main memory 693 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), static memory 697 (e.g., flash memory, static random access memory (SRAM), etc.) and data storage system 698.

[0091] Processing device 691 represents one or more general-purpose processing devices, such as microprocessors, central processing units, etc. More specifically, the processing device may be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, or a processor implementing other instruction sets, or a processor implementing combinations of instruction sets. Processing device 691 may also be one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, etc. Processing device 691 is configured to execute instructions 692 for performing the operations and steps discussed herein. Computer system 690 may further include a network interface device 694 for communicating via network 695.

[0092] Data storage system 698 may include machine-readable storage medium 699 (also referred to as computer-readable medium) thereon storing one or more sets of instructions 692 or software embodying any one or more of the methods or functions described herein. Instructions 692 may also reside wholly or at least partially in main memory 693 and / or processing device 691 during execution by computer system 690, which also constitute machine-readable storage medium.

[0093] In one embodiment, instruction 692 includes instructions for implementing the corresponding Figure 1 The machine-readable storage medium 699 provides functional instructions for the host 102 and / or memory device 103. While the machine-readable storage medium 699 is shown as a single medium in the exemplary embodiment, the term "machine-readable storage medium" should be understood as a single medium or multiple media containing one or more sets of instructions. The term "machine-readable storage medium" should also be understood as any medium capable of storing or encoding a set of instructions that are executable by a machine and cause the machine to perform any one or more of the methods of this disclosure. Therefore, the term "machine-readable storage medium" should be understood to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0094] As used herein, “several things” can refer to one or more of such things. For example, “several memory devices” can refer to one or more memory devices. “Multiple things” means two or more. Additionally, as used herein, and particularly with respect to reference numerals in the drawings, designators (e.g., “N”) indicate that several specific features so specified may be included in several embodiments of this disclosure.

[0095] The figures in this document follow a numbering convention, where the first few digits correspond to the figure number and the remaining digits identify elements or components in the figures. Similar elements or components between different figures can be identified by using similar digits. As will be understood, elements shown in the various embodiments herein may be added, interchanged, and / or removed to provide several additional embodiments of this disclosure. Furthermore, the scale and relative dimensions of the elements provided in the figures are intended to illustrate various embodiments of this disclosure and are not intended to be limiting.

[0096] Although specific embodiments have been illustrated and described herein, those skilled in the art will understand that arrangements calculated to achieve the same results may replace the specific embodiments shown. This disclosure is intended to cover adaptations or variations of various embodiments of this disclosure. It should be understood that the above description is illustrative and not restrictive. After reviewing the above description, combinations of the above embodiments and other embodiments not specifically described herein will be apparent to those skilled in the art. The scope of the various embodiments of this disclosure includes other applications in which the above structures and methods are used. Therefore, the scope of the various embodiments of this disclosure should be determined by reference to the appended claims and the full scope of their equivalents.

[0097] In the foregoing detailed embodiments, various features are grouped into a single embodiment for the purpose of simplifying this disclosure. This disclosure method should not be construed as reflecting an intention that the disclosed embodiments of this disclosure must use more features than are expressly stated in each claim. Rather, as reflected in the appended claims, the subject matter of the invention lies in fewer than all features of a single disclosed embodiment. Therefore, the appended claims are thus incorporated into the detailed embodiments, wherein each claim is considered an independent, separate embodiment.

Claims

1. An apparatus for encoding traffic between a host computer and a deep learning accelerator, comprising: Memory cell array; The deep learning accelerator DLA is coupled to the memory cell array; and A controller, coupled to the memory cell array and the DLA, wherein the controller is configured to: Receive encoded data from the host; The encoded data is stored in the memory cell array; and The DLA is configured to: In response to receiving a signal from the controller, the encoded data is accessed from the memory cell array; The encoded data is decoded using an autoencoder, wherein the encoded data is decoded by a decoder of the autoencoder implemented by the DLA to produce decoded data, and wherein the decoder of the autoencoder is implemented as an artificial neural network (ANN); and Perform multiple operations on the decoded data.

2. The device of claim 1, wherein the DLA is further configured to utilize the decoder comprising a first artificial neural network (ANN) to decode the encoded data.

3. The device of claim 2, wherein the DLA is further configured to utilize a second ANN to perform multiple operations on the decoded data.

4. The device of claim 1, wherein the DLA is further configured to perform a plurality of operations on the decoded data to generate an output, and wherein the controller is further configured to provide the output to the host.

5. The device of claim 4, wherein the DLA is further configured to: The output is encoded using encoders of different automatic encoders to produce an encoded output, wherein the encoders are implemented in a DLA, and wherein the controller is further configured to provide the encoded output to the host.

6. A method for encoding traffic between a host computer and a deep learning accelerator (DLA), comprising: At the DLA in the memory device, signaling is received from the host indicating first data including hyperparameters for configuring an artificial neural network (ANN), wherein the ANN is implemented by the DLA; Encoded second data is received from the host at the DLA of the memory device, wherein the second data is generated by a plurality of sensors and encoded by an encoder of an autoencoder implemented by the host. The encoded second data is decoded to produce decoded second data, wherein the decoding includes a decoder utilizing the autoencoder and wherein the decoder is implemented by the DLA; The decoded second data is processed using the ANN; and The encoded output of the ANN is transmitted to the host.

7. The method of claim 6, wherein receiving signaling indicating the first data including the hyperparameters further comprises receiving the first data including the number of layers, the number of nodes in each layer, a plurality of weights, a plurality of biases, and an activation function for the DLA.

8. The method of claim 6, wherein receiving signaling indicating the first data including the hyperparameters further comprises receiving coded hyperparameters.

9. The method of claim 8, further comprising transmitting the encoded output of the ANN to the host such that the host decodes the encoded output using different decoders of different autoencoders implemented by the host.

10. The method of claim 6, further comprising receiving the encoded second data via an interface that couples the host to the memory device.

11. The method of claim 9, wherein the encoder of the host and the decoder of the DLA share a first plurality of hyperparameters and different encoders of the DLA and different decoders of the host share a second plurality of hyperparameters, wherein the first plurality of hyperparameters and the second plurality of hyperparameters are the same plurality of hyperparameters.

12. The method of claim 9, wherein the encoder of the host and the decoder of the DLA share a first plurality of hyperparameters and different encoders of the DLA and different decoders of the host share a second plurality of hyperparameters, wherein the first plurality of hyperparameters and the second plurality of hyperparameters are different.

13. A system for encoding services between a host and a DLA, comprising: The host; A memory device comprising a deep learning accelerator (DLA) and a memory cell array, wherein the DLA includes a decoder of an autoencoder and an artificial neural network (ANN). The host is configured to: Encoding the first set of data to produce encoded data, wherein the first set of data is encoded by the encoder of the autoencoder, wherein the encoder is implemented by the host; and The DLA provides the encoded data from the host to the memory device such that the DLA uses the ANN to process a second set of data corresponding to the first set of data; The DLA is configured to: The decoder of the automatic encoder is used to decode the encoded data to generate a second set of data corresponding to the first set of data; and The ANN is used to process the second set of data.

14. The system of claim 13, wherein the DLA is further configured to encode the output of the ANN using different encoders of different autoencoders to produce an encoded output, wherein the different encoders are implemented by the DLA.

15. The system of claim 14, further comprising a controller for the memory device, the controller being configured to store the encoded output in the memory cell array of the memory device.

16. The system of claim 13, wherein the host is further configured to: Several obfuscation flags are added to the encoded data; and The plurality of obfuscation flags are provided to the memory device to allow the memory device to recognize the plurality of obfuscation flags.

17. The system of claim 16, wherein the DLA is configured to use the plurality of obfuscation flags to decode the encoded data.

18. The system of claim 16, wherein the DLA is further configured to remove the plurality of obfuscation flags from the encoded data before the decoder utilizing the DLA is used to decode the encoded data.

19. The system of claim 16, wherein the host is further configured to add bits to the first set of data before encoding the first set of data, and wherein the memory device is further configured to remove bits from the first set of data after decoding the first set of data.

20. The system of claim 13, wherein the host is further configured to retrieve the encoded output from the memory cell array and to decode the encoded output using different decoders of different autoencoders implemented by the host.

Images