AI processing supplementing memory

Abstract

Devices and methods may relate to complementing AI processing in memory. Accelerators and / or hosts may perform AI processing. Some tasks involving AI processing may be performed by memory devices instead of accelerators and / or hosts. Memory devices may perform AI processing in conjunction with hosts and / or accelerators to increase the efficiency of hosts and / or accelerators.

Description

Technology Field

[0001] The present disclosure generally relates to memory, and in particular to an apparatus and method related to complementing artificial intelligence (AI) processing in memory. Background Technology

[0002] Memory devices are generally provided as semiconductors or integrated circuits inside computers or other electronic devices. There are various types of memory, including volatile and non-volatile memory. Volatile memory may require power to retain data and includes Random-Access Memory (RAM), Dynamic Random Access Memory (DRAM), and Synchronous Dynamic Random Access Memory (SDRAM). Non-volatile memory can provide permanent data by retaining stored data when power is not supplied, and may include, in particular, NAND flash memory, NOR flash memory, Read Only Memory (ROM), Electrically Erasable Programmable ROM (EEPROM), Erasable Programmable ROM (EPROM), and variable resistance memory, such as Phase Change Random Access Memory (PCRAM), Resistive Random Access Memory (RRAM), and Magnetoresistive Random Access Memory (MRAM).

[0003] Memory is also utilized as a storage for volatile and non-volatile data for a wide range of electronic applications, including, but not limited to, personal computers, portable memory sticks, digital cameras, mobile phones, portable music players such as MP3 players, movie players, and other electronic devices. Memory cells can be arranged in arrays, and arrays are used in memory devices.

[0004] Various computing systems include multiple processing resources connected to memory (e.g., memory systems) accessed in connection with the execution of instruction sets (e.g., programs, applications, etc.). Processing resources can execute instructions to perform artificial intelligence (AI). Processing resources may be dedicated to performing AI. AI may include learning and / or problem-solving. For example, AI may be characterized by the ability to solve and learn problems so that the success rate of problem-solving increases over time and / or across identified examples. AI may also include the ability to recognize the environment corresponding to the problem being solved. For example, if an image is the environment, AI can be used to identify features of the image, and the success and failure of identification can be used to increase the success rate of feature identification. Brief explanation of the drawing

[0005] FIG. 1 is a block diagram of a device in the form of a computing system including a memory device according to a plurality of embodiments of the present disclosure. FIG. 2 is a block diagram of a device in the form of a computing system including a memory device and a processing resource according to a plurality of embodiments of the present disclosure. FIG. 3 is a block diagram of an exemplary memory device comprising a plurality of layers according to a plurality of embodiments of the present disclosure. FIG. 4 is a block diagram of a memory device and a plurality of processing resources outside the memory device according to a plurality of embodiments of the present disclosure. FIG. 5 illustrates an exemplary flowchart of a method for supplementing AI processing in memory according to a plurality of embodiments of the present disclosure. FIG. 6 illustrates an exemplary machine of a computer system in which an instruction set can be executed to enable the machine to perform the various methodologies discussed herein. Specific details for implementing the invention

[0006] The present disclosure includes devices and methods related to complementing artificial intelligence (AI) processing in memory. AI may include machine learning. Machine learning may include performing tasks without using explicit instructions, but instead relying on patterns and / or inference derived from a dataset. As used herein, references to AI may also include references to machine learning. Machine learning may include training neural networks, using trained neural networks for inference, or both.

[0007] Processing resources can be configured to perform tasks aligned with AI. For example, processing resources can execute instructions for email filtering, computer vision, data mining, and / or natural language, among other areas where AI can be implemented. Processing resources can be used generally, such as processing resources that are part of a host. In various cases, processing resources can be used exclusively for AI. For example, processing resources may include AI chips, which can also be referred to as AI accelerators and / or AI processing resources. AI chips can also be called accelerators.

[0008] AI chips may be processing resources for AI applications, including neural networks (e.g., artificial neural networks), machine vision, and / or machine learning. AI chips may be used in mobile devices, Internet of Things (IoT) devices, and other devices that perform data-intensive or sensor-based tasks.

[0009] In some cases, AI chips can perform tasks that can be offloaded to other processing resources. Offloading tasks to other processing resources can increase the efficiency of the AI ​​chip or decrease its complexity. In an example where processing resources coexist with the data used by the AI ​​chip in a memory device, the efficiency of the AI ​​chip can increase due to the latency corresponding to moving data from the memory device to the processing resources.

[0010] Offloading tasks from processing resources used in AI applications can be referred to as complementing AI processing in memory. In some examples, assisting an AI chip can also be considered complementing AI processing. For instance, a memory device can complement AI processing by preparing data for use by the AI ​​chip and / or data generated by the AI ​​chip for storage in the memory device. A memory device can also complement AI processing by processing data generated by the AI ​​chip and designated to be delivered to different processing resources, such as the AI ​​chip and / or a host.

[0011] In some examples, AI processing may be supplemented by performing tasks that can be performed by the AI ​​chip other than data preparation. For example, a memory device may perform machine learning by utilizing a neural network implemented by the memory device. Data used in the neural network may be generated by the AI ​​chip, and the results of the neural network may be utilized in AI processing by the AI ​​chip. As used herein, AI processing may include the execution of operations used in AI. Machine learning tasks may include tasks used in machine learning.

[0012] As used herein, "multiple" may refer to one or more of such things. For example, multiple memory devices may refer to one or more memory devices. "Multiple" something means two or more. Additionally, a designator such as "N" used herein, particularly in relation to reference numbers in the drawings, indicates that multiple specific features so designated may be included in multiple embodiments of the present disclosure.

[0013] The drawings in this specification follow a numbering rule in which a first number or number corresponds to a drawing number and the remaining numbers identify elements or components of the drawings. Similar elements or components between different drawings may be identified using similar numbers. As is understood, elements illustrated in various embodiments of this specification may be added, exchanged, and / or removed to provide various additional embodiments of this disclosure. Furthermore, the proportions and relative scales of the components provided in the drawings are intended to illustrate various embodiments of the invention and should not be used in a limiting sense.

[0014] FIG. 1 is a block diagram of an apparatus in the form of a computing system (100) including a memory device (103) according to a plurality of embodiments of the present disclosure. As used herein, for example, the memory device (103), memory array (110), and / or host (102) may also be individually considered as an "apparatus."

[0015] In the present example, the system (100) includes a host (102) connected to a memory device (103) via an interface (104). The computing system (100) may be a personal laptop computer, a desktop computer, a digital camera, a mobile phone, a memory card reader, or an Internet of Things (IoT) capable device, among various other types of systems. The host (102) may include a number of processing resources (e.g., one or more processors, microprocessors, or some other type of control circuit) capable of accessing the memory (120). The system (100) may include individual integrated circuits, or both the host (102) and the memory device (103) may be on the same integrated circuit. For example, the host (102) may be a system controller of a memory system comprising multiple memory devices (103), and the system controller (102) provides access to each memory device (103) by other processing resources such as a central processing unit (CPU). The host (102) may also be an AI chip configured for AI processing.

[0016] In the example illustrated in FIG. 1, the host (102) is responsible for running an operating system (OS) and / or various applications (e.g., AI processes) that can be loaded (e.g., from a memory device (103) via a control circuit (105)). The OS and / or various applications can be loaded from the memory device (103) by providing an access command from the host (102) to the memory device (103) to access data including the OS and / or various applications. The host (102) can also access said data utilized by the OS and / or various applications by providing an access command to the memory device (103) to retrieve data utilized for the execution of the OS and / or various applications.

[0017] For clarity, the system (100) has been simplified to focus on features particularly relevant to the present disclosure. The memory array (110) may be, for example, a DRAM array, an SRAM array, an STT RAM array, a PCRAM array, a TRAM array, an RRAM array, a NAND flash array, and / or a NOR flash array. The array (110) may include memory cells arranged in rows connected by access lines (which may be referred to herein as word lines or select lines) and columns connected by detection lines (which may be referred to herein as digit lines or data lines). Although a single array (110) is illustrated in FIG. 1, embodiments are not limited thereto. For example, a memory device (103) may include a plurality of arrays (110) (e.g., a plurality of DRAM cell banks).

[0018] The memory device (103) includes an address circuit (106) for latching an address signal provided through an interface (104). The interface may include, for example, a physical interface utilizing an appropriate protocol (e.g., a data bus, an address bus, and a command bus, or a combined data / address / command bus). Such a protocol may be custom-made or proprietary, or the interface (104) may use a standardized protocol such as PCIe (Peripheral Component Interconnect Express), CXL (Compute Express Link), Gen-Z, CCIX, etc. The address signal is received and decoded by a row decoder (108) and a column decoder (112) to access the memory array (110). Data may be read from the memory array (110) by using a sensing circuit (111) to detect changes in voltage and / or current on a sensing line. The sensing circuit (111) may include a sensing amplifier capable of reading and latching a page (e.g., a row) of data from, for example, the memory array (110). The I / O circuit (107) may be used to communicate bidirectionally with the host (102) through the interface (104). The read / write circuit (113) is used to write data to the memory array (110) or to read data from the memory array (110). For example, the circuit (113) may include various drivers, latch circuits, etc.

[0019] The control circuit (105) decodes signals provided by the host (102). The signals may be commands provided by the host (102). These signals may include chip enable signals, write enable signals, and address latch signals used to control operations performed on the memory array (110), and these operations include data read operations, data write operations, and data erase operations. In various embodiments, the control circuit (105) is responsible for executing commands from the host (102). The control circuit (105) may include some other type of control circuit that can be implemented in the form of a state machine, a sequencer, and / or hardware, firmware, software, or a combination of these three. In some examples, the host (102) may be a controller outside the memory device (103). For example, the host (102) may be a memory controller connected to the processing resources of the computing device.

[0020] The example of FIG. 1 illustrates processing resources (114 and 115). Processing resources (114) may be connected to a control circuit (105) and a data line (116). Processing resources (114 and 115) may perform operations on data read from the memory array (110) before providing data through the interface (104). Although both processing resources (114 and 115) are illustrated in FIG. 1, the memory device (103) may be implemented with one or more of the processing resources (114 and / or 115). For example, in at least one embodiment, the memory device (103) may include a processing resource (115) configured as a complementary metal-oxide-semiconductor (CMOS) under the memory array (110), but does not include a processing resource (114). As another example, in at least one embodiment, the memory device (103) may include a processing resource (114) composed of individual chips on the memory device (103), but does not include a processing resource (115).

[0021] In various examples, a processing resource (114) may access data from data lines (116), process data, and provide data through data lines (116). A data line (116) may be coupled to a latch used to latch the data provided through the data line (116). Data may be read from a latch connected to the data line (116) to retrieve data for processing. In response to data processing, the processing resource (114) may update the latch to store the updated data. A charge corresponding to the updated data may be moved back from the latch to the data line (116).

[0022] Each latch may include, for example, a plurality of AND gates and a plurality of NOR gates. The first AND gate and the second AND gate may receive a signal indicating whether the processing resource (114) needs to access data. The first AND gate may receive a signal from the processing resource (114) containing updated data. The second AND gate may also receive a signal from the processing resource (114) containing the inverse of the updated data.

[0023] The first NOR gate can receive the output of the first AND gate and the output of the second NOR gate. The first NOR gate can be coupled to a data line, such as a data line, and the output from the second NOR gate can be merged to provide the input to the first NOR gate. A processing resource (114) can also be connected to the second NOR gate so that the output of the second NOR gate is provided to the processing resource (114) to allow reading of the value stored in the latch. The output of the first NOR gate can be provided as an input to the second NOR gate. The second NOR gate may also receive the output of the second AND gate as an input.

[0024] In various examples, the processing resource (114) can access data from the detection amplifiers. The processing resource (114) can perform AI processing on the data to generate processed data. The processing resource (114) also activates a data line with the processed data.

[0025] The processing resource (115) may also be used to perform AI processing. The processing resource (115) (e.g., CMOS under array) may include a number of logic blocks configured to perform various functions using data values ​​stored in, for example, a memory array (110). The processing resource (115) is also connected to a sensing amplifier (111) and / or a data line, so that the processing resource (115) can provide data to the sensing amplifier (111) and / or the data line (116).

[0026] The processing resources (114 and / or 115) may be configured to perform AI processing on data stored in the memory array (110). The processing resources (114 and / or 115) may be configured to process data streams. For example, the processing resources (114 and / or 115) may be configured to process data having a size equal to the width of the data bus (e.g., interface (104)). The processing resources (114 and / or 115) may also process data having a size greater than the width of the data bus and provide the data through the interface (104) in chunks having a size equal to the width of the data bus. Stream processing may also include performing operations on the data read from the memory array (110) before providing the data through the data bus (e.g., interface (104)).

[0027] The processing resource (114 and / or 115) can process data before storing data in the memory array (110) and / or after reading data from the memory array (110). For example, the processing resource (114) can process data read from the memory array (110), and the processing resource (115) can process data before storing data in the memory array (110) or in response thereto. The processing resource (114 and / or 115) can perform AI processing to prepare data for processing by the AI ​​chip and / or prepare data to be received by a processing resource other than the AI ​​chip, such as the host (102).

[0028] FIG. 2 is a block diagram of a device in the form of a computing system including a memory device (203) and a processing resource (222) according to a plurality of embodiments of the present disclosure. The memory device (203) shows a stream processor (214). The memory device (203) is similar to the memory device (103), and the stream processor (214) is similar to the processing resource (114 and / or 115) of FIG. 1.

[0029] The processing resource (222) may be, for example, an AI chip. The processing resource (222) may also be a host, a graphics processing unit (GPU), and / or other processing resources. In the example described herein, the processing resource (222) is described as an AI chip. The processing resource (222) can perform AI processing. To process data (220) stored in the memory device (203) at the processing resource (222), the processing resource (222) may retrieve data (220) from the memory device (203). Data stored in the memory array of the memory device (203) may be described as raw data, which is not ready for processing by the processing resource (222). The raw data (220) may be retrieved from the memory array and provided to the stream processor (214). The stream processor (214) may perform multiple operations characteristic of AI processing before providing data to the processing resource (222).

[0030] The stream processor (214) provides stream processing when interacting with the processing resource (222). "Stream" describes the interaction between the stream processor (214) and the processing resource (222). Stream processing involves the processing of data (220) in a data path between memory and the processing resource (222). Stream processing can be initiated at the stream processor (214) and terminated at the processing resource (222). Stream processing can be initiated at the processing resource (222) and terminated at the stream processor (214). Stream processing can be performed simultaneously at the processing resource (222) and the stream processor (214). For example, the stream processor (214) can process data to generate a first data set. The stream processor (214) can provide the first data set to the processing resource (222). Upon receiving the first data set from the processing resource (222), the processing resource (222) can generate a second data set while the stream processor (214) generates a third data set. Similarly, the processing resource (222) can provide the first data set to the stream processor (214). When the stream processor (214) receives the first data set, the stream processor (214) can generate a second data set while the processing resource (222) generates a third data set.

[0031] In some examples, the processing resource (222) may perform AI processing to identify features of an image, context of a word, and / or classification of the data set, among other machine learning applications performed on the first data set. The identified features of the image, context of the word, and / or classification of the data set may be provided to the stream processor (214) as a second data set. The second data set may be a first format unique to the processing resource (222), but may not be unique to a different processing resource, such as a host and / or GPU. The stream processor (214) may convert the data in the first format to a second format before storing the data in a memory array of the memory device (203). Converting the data to a second format before storing the data in the memory array may make the data available to the different processing resource in a format compatible with the application executed by the processing resource and / or the different processing resource.

[0032] The stream processor (214) may also process the data (220) before providing the data (220) to the processing resource (222). For example, the stream processor (214) may receive raw data of a first format and process the data to generate data of a second format. The first format may not be compatible with the processing resource (222) and / or the application executed by the processing resource (222). The data (220) may be stored in a first format that may include a raw format. The raw format may include unformatted data. AI processing performed by the stream processor may identify and / or generate rules that can be used to format the data.

[0033] The stream processor (214) may provide identified rules regarding the provision of raw data to the processing resource (222). For example, the stream processor (214) may identify weights of a network that can be used to format the data. The weights may be generated from the data and / or training data. The weights of the network and data may be provided to the processing resource (222). The processing resource (222) may utilize the weights to configure the network hosted by the processing resource (222) and / or may use the weights as a starting point for the processing resource (222) to further refine the network weights.

[0034] In various examples, the data can describe an image. For example, the data can describe the pixels of an image, among other types of image descriptions that can be represented by the data (220). The data (220) may be in YCC format before being provided to the stream processor (214). In the YCC format, Y is luminance, Y' is luminance, and C B is blue difference, C R YC is the red difference B C R or Y'C B C R It is referred to in the industry in the form. The YCC color space is defined by the mathematical coordinate transformation of the associated RGB (Red Green Blue) color space. Data in the YCC format may be more efficient than the RGB format because there is more redundancy than in the RGB format. A stream processor (214) can convert the data (220) into the RGB format. The processed data may be provided to a processing resource (222) for further AI processing, which may include the identification of features of the image represented by the data.

[0035] In some examples, data (220) may be provided to a stream processor (214). The stream processor (214) may perform AI processing on the data (220) to identify, for example, features of an image represented by the data. The stream processor (214) may provide a description of the features or a description of the features and data to a processing resource (222). The processing resource (222) may perform AI processing on the results of the stream processor (214). For example, the processing resource (222) may categorize the features identified by the stream processor (214). When the stream processor (214) identifies features of a face image, the processing resource (222) may perform image recognition by identifying the face based on the features identified by the stream processor (214). In various examples, the stream processor (214) may identify the face based on the features identified by the stream processing resource (222).

[0036] The stream processor may be activated by the control circuit of the memory device (203). In some examples, the stream processor may be activated by the host. The control circuit may determine whether to provide raw data (220) to the processing resource (222) or to provide processed data to the processing resource (222). The control circuit may control multiple multiplexers to provide raw data (220) or processed data. In some examples, the control circuit may determine the work performed by the stream processor (214). For example, the control circuit may control a network implemented by the stream processor (214). The control circuit may receive a control command from the processing resource (222) describing the type of AI processing to be performed by the stream processor (214). The control command may be associated with an access command. For example, the processing resource (222) may request from the control circuit that features be identified in the data accessed by a subsequent access command. The processing resource (222) may also request through the control command that the data be converted from a first format to a second format. For example, the control command can identify the second format.

[0037] FIG. 3 is a block diagram of an exemplary memory device (303) comprising a plurality of layers according to a plurality of embodiments of the present disclosure. The memory device (303) may be similar to the memory device (103) previously described in relation to FIG. 1.

[0038] The memory device (303) is a three-dimensional (3D) memory device comprising multiple layers stacked together. For example, a first layer (310) of the memory device (303) (e.g., a memory array shown in FIG. 3) is connected to a second layer (315) of the memory device (303) (e.g., a CMOS under array as shown in FIG. 3). Although the first layer (310) is shown as being on the second layer (315), the first layer (310) and the second layer (315) may be designed to include a number of different orientations so that the first layer (310) is coupled to the second layer (315). The example described herein is not limited to a specific orientation between the first layer (310) and the second layer (325). The first layer (310) of the memory device (303) may include an array of memory cells. Although the embodiments are not limited thereto, the memory cells of the array may include DRAM memory cells.

[0039] The second layer (315) may include a plurality of logical blocks configured to perform various functions, for example, using data values ​​stored in an array of memory cells. The second layer (315) may be described as a processing resource (315). The processing resource (315) may be composed of a plurality of processing resources (332). The second layer may also include a row driver (347) and / or a column driver (348). Although nine processing resources (332) are shown in FIG. 3, the processing resource (315) may include more or fewer processing resources (332) than shown herein.

[0040] The second layer (315) may be one of a plurality of logic blocks included within the memory device (303). A processing resource (332) may be configured to perform AI processing. For example, the processing resource (332) may be configured as a network (e.g., a neural network). Each processing resource (332) may be a node of the neural network. Each processing resource may be connected to a different memory cell of the memory array (310) capable of storing the network's weights and inputs to the network. Processing resources (332) may be interconnected so that the output of some processing resources (332) can be received as an input by other processing resources (332). The result of the AI ​​processing performed by the processing resources (332) may be stored back in the memory array (310) and / or latched by a sensing amplifier.

[0041] The second layer (315) may further include a row driver (347) and a column driver (348) that can be implemented to activate the row(s) and / or column(s) of a memory array (e.g., memory array (310)). As described herein, the row driver (347) and the column driver (348) may receive an address signal decoded by a row decoder and a column decoder, respectively, which are controllable by a control circuit such as the control circuit (105) of FIG. 1.

[0042] FIG. 4 is a block diagram of a memory device (403) and a plurality of processing resources (442-1, 442-2) outside the memory device (403) according to a plurality of embodiments of the present disclosure. The memory device (403) is similar to the memory device (103) of FIG. 1. The processing resources (442-1, 442-2) may be similar to the host (102) and processing resources (222) of FIG. 1 and FIG. 2, respectively.

[0043] In various examples, the processing resource (442-2) can perform AI processing and store the results of the AI ​​processing in the memory device (403). The processing resource (442-1) can provide access commands to the memory device (403). The memory device can retrieve the requested data and format the data in a format compatible with the processing resource (442-1) and / or the application hosted by the processing resource (442-1). The memory device (403) can provide the formatted data to the processing resource (442-1).

[0044] The formatting of data stored by the processing resource (442-2) may be an extension of AI processing and may be considered as AI processing. In some examples, the AI ​​processing performed by the processing resource (442-2) may be incomplete and / or may be utilized to perform additional AI processing by the memory device (403). For example, the processing resource (442-2) may identify features of an image and provide data defining the identified features to the memory device (403). The memory device (403) may receive data identifying features of an image and may also receive the actual image. The memory device (403) may perform further AI processing to perform face recognition on the features of the image using the image and / or data describing the image. The memory device (403) may store identifiers corresponding to the image and / or features of the image. The memory device (403) may also store data describing the reliability of the identification of the image and may store additional possible identifiers corresponding to the image.

[0045] The memory device (403) may provide data corresponding to an identifier of an image, confidence in the identification of the image, and / or possible identifiers corresponding to the image to the processing resource (442-1) and / or the processing resource (442-2). In some examples, the processing resource (442-2) may continue to perform AI processing on the data describing the identification of the image, confidence in the identification, and / or possible identifications. For example, the processing resource (442-2) may use the identification to scan additional images to determine whether additional images provide a match for the identification generated by the memory device (403). The memory device (403) and the processing resource (442-2) may function as a unit to perform multifaceted AI processing. The memory device (403) and / or the processing resource (442-2) may independently and / or collectively control the AI ​​processing performed in the memory device (403) and the processing resource (442-2).

[0046] In various examples, the processing resource (442-1) may store data in a memory device that can be used to perform AI processing by the processing resource (442-2). However, the data may be in a format incompatible with the processing resource (442-2) and / or the application hosted by the processing resource (442-2). The memory device (403) may format the data into a format compatible with the processing resource (442-2) before storing the data and / or after storing the data in the memory array of the memory device (403).

[0047] The memory device (403) can extend the functionality of the processing resource (442-2) by extending the interface of the processing resource (442-2). For example, the processing resource (442-1) and / or the application hosted by the processing resource (442-1) can be simplified to use a single interface to the additional processing resource and / or the processing resource (442-2) that performs AI processing, regardless of whether the single interface is compatible with the processing resource (442-2) and / or the additional processing resource. The memory device (403) can receive data in a format utilized by the processing resource (442-1) and can reformat the data into a format compatible with the processing resource (442-2) and / or the additional processing resource.

[0048] FIG. 5 illustrates an exemplary flowchart of a method for supplementing AI processing in memory according to a plurality of embodiments of the present disclosure. In FIG. 550, an access command and a data processing request may be received from a host device or an accelerator. The access command and / or request may be provided individually and / or simultaneously. For example, the request may be a command separate from the access command, or the request may be provided as the same command as the access command.

[0049] In 552, in response to the receipt of an access command and a data processing request, a memory cell having an address corresponding to the access command may be activated. The memory cell may contain data that can be used for AI processing. For example, the data may include images, natural language and / or IP addresses, among other possible datasets that can be used for training or classification.

[0050] In 554, data can be processed using processing resources under the memory array. The memory array can be used to store data. Data can be processed according to a data processing request. The control circuit of the memory device can activate processing resources and memory cells based on the reception of a request.

[0051] In 556, multiple data lines may be activated to retrieve data from a processing resource. A processing resource hosted by a memory device and used for AI processing may be connected to a sensing amplifier. Given that each processing resource may be coupled to multiple memory cells that are sequentially coupled to different sensing lines of the memory array, each processing resource may be coupled to a part of the sensing amplifier of the memory array, or each processing resource may be connected to multiple sensing amplifiers.

[0052] The control circuit can activate the sensing circuit to latch data generated by the processing resource. The control circuit can also store the data generated by the processing resource back into the memory cell and activate the sensing amplifier to latch the processed data stored in the memory cell. The control circuit can activate the data line to transmit the data stored in the sensing amplifier to the data line.

[0053] A request for processing data stored in a memory array may include identifiers for a plurality of tasks to prepare data for machine learning tasks to be performed by a host device or accelerator. Tasks may be utilized for machine learning as well as for data preparation for machine learning. As used herein, tasks for AI processing may include standard operations such as AND and OR operations, and may include operations used in neural networks. For example, load weighting for a network may be considered a task. Tasks used for network configuration may be AI processing. For example, configuring a network to include selecting the amount of nodes and / or changing the topology of the network may be considered a task.

[0054] A data processing request may include identifiers for multiple tasks to perform machine learning tasks by processing resources instead of the host and / or accelerator. If the host and / or accelerator (e.g., an AI chip) has performed a task or an equivalent task to perform AI processing, the task may be AI processing. Offloading tasks from the AI ​​chip to processing resources in a memory device may involve performing tasks in the memory device that would otherwise be performed on the AI ​​chip. In some examples, the host and / or accelerator have the capability to perform tasks offloaded to the memory device, but choose to offload tasks to allow for greater throughput and / or efficiency in performing the tasks.

[0055] In various examples, a memory device configured to perform AI processing may include a memory array, data lines, and a control circuit coupled to the memory array. The control circuit may be configured to receive an access command and retrieve data from the memory array in response to the reception of the access command. The control circuit may also be configured to perform multiple operations on the data. The control circuit may perform operations through a processing resource hosted by the memory device. The processing resource may be any of the processing resources hosted by the memory device described herein. The control circuit may update the data line as a result of the multiple operations by moving data from the processing resource to the data line.

[0056] Operations may be performed on the data to convert the data from a first format to a second format. Converting the data may include, for example, decompressing the data in YCC format to produce data in RGB format.

[0057] In some examples, the request may identify tasks to be performed as requested by the host and / or accelerator. The request may also identify the format in which data must be provided without identifying specific tasks to be performed. In some examples, identifying tasks to be performed by the memory device may include identifying the order in which the tasks are to be performed. Multiple tasks may include data pre-processing before providing data to the host and / or accelerator.

[0058] A data line may be connected to multiple latches. The latches may be used to latch the charge provided to the data line. A control circuit may update the data line by updating the latches connected to the data line. Updating the data line may involve transferring a charge representing data to the data line.

[0059] The processing resources of the memory device can perform operations on a stream of data stored in a memory array. For example, the size of the data on which the operation is performed may be equal to the width of the bus (e.g., data bus) used to provide data to the host and / or accelerator. The size of the data on which the operation is performed may include the width interval of the bus (e.g., data bus). The size of the data on which the operation is performed may be multiple widths of the bus. The data may be divided into multiple partitions equal to the width of the bus and provided to the host and / or accelerator.

[0060] In various examples, access commands may be received by a memory device. The memory device may also receive network settings from a host and / or accelerator to process data identified by the access command. The memory device may organize multiple processing resources into a network by utilizing the memory cells of a memory array storing data and the network settings. The memory device may process data identified by the access command by utilizing the organized network. The network may include an artificial neural network or other types of networks that can be used for AI processing.

[0061] Network configuration may include the weights of the neural network and the number of nodes in the neural network. Network configuration may include the network type, such as a neural network. Network configuration may also include the operations used by the nodes to combine the inputs received by the nodes with the weights corresponding to each node. Network configuration may also include a network topology that describes the active communication paths between processing resources.

[0062] FIG. 6 illustrates an exemplary machine of a computer system (640) capable of executing a set of instructions to enable the machine to perform the various methodologies discussed herein. In various embodiments, the computer system (640) may correspond to a system (e.g., system (100) of FIG. 1) that includes, is coupled to, or utilizes a memory subsystem (e.g., memory device (103) of FIG. 1), or may be used to perform the work of a controller (e.g., control circuit (105) of FIG. 1). In alternative embodiments, the machine may be connected (e.g., networked) to other machines on a LAN, intranet, extranet, and / or the Internet. The machine may operate as a server or client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment.

[0063] A machine may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web device, server, network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or non-sequential) that specifies the actions to be taken by the machine. Additionally, although a single machine is exemplified, the term “machine” should be considered to include any set of machines that execute a set of instructions (or multiple sets) individually or jointly to perform any one or more of the methodologies discussed herein.

[0064] An exemplary computer system (640) includes a processing device (602) communicating with each other via a bus (630), a main memory (604) (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), e.g., synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory (606) (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system (618).

[0065] The processing unit (602) represents one or more general-purpose processing units, such as a microprocessor, a central processing unit, etc. More specifically, the processing unit may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing another instruction set, or a processor implementing a combination of instruction sets. The processing unit (602) may also be one or more special-purpose processing units, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. The processing unit (602) is configured to execute instructions (626) for performing the tasks and steps discussed herein. The computer system (640) may further include a network interface unit (608) for communicating through a network (620).

[0066] The data storage system (618) may include a machine-readable storage medium (624) (also known as a computer-readable medium) that stores one or more sets of software or instructions (626) that implement one or more of the methodologies or functions described herein. The instructions (626) may also exist wholly or at least partially within the main memory (604) and / or processing unit (602) while being executed by the computer system (640), and the main memory (604) and processing unit (602) also constitute the machine-readable storage medium.

[0067] In one embodiment, the instruction (626) includes instructions for implementing functions corresponding to the host (102) and / or memory device (103) of FIG. 1. Although the machine-readable storage medium (624) is depicted as a single medium in exemplary embodiments, the term “machine-readable storage medium” should be considered to include a single medium or multiple media storing one or more instruction sets. The term “machine-readable storage medium” is considered to include any medium capable of storing or encoding an instruction set for execution by a machine and enabling the machine to perform one or more of the methodologies of the present invention. Accordingly, the term “machine-readable storage medium” should be considered to include, but not be limited to, solid-state memory, optical media, and magnetic media.

[0068] Although specific embodiments have been illustrated and described herein, those skilled in the art will understand that an arrangement calculated to achieve the same result may replace the specific embodiments illustrated. The present disclosure is intended to include adaptations or variations of various embodiments of the present disclosure. It should be understood that the foregoing description is illustrative and not limiting. Combinations of the above embodiments and other embodiments not specifically described herein will be apparent to those skilled in the art upon reviewing the foregoing description. The scope of the various embodiments of the present disclosure includes other applications in which the structures and methods are used. Accordingly, the scope of the various embodiments of the present disclosure should be determined by reference to the appended claims, together with the full scope of equivalents to which such claims are granted.

[0069] In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of simplifying the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the disclosed embodiments of this disclosure must use more features than explicitly cited in each claim. Rather, as reflected in the following claims, the subject matter is less than all the features of a single disclosed embodiment. Accordingly, the following claims are incorporated into the detailed description, and each claim stands as a separate embodiment in itself.

Claims

Claim 1 A device comprising: a memory array; an input / output circuit; a processing resource; data lines connected between the memory array and the input / output circuit and between the processing resource and the input / output circuit; and a control circuit connected to the memory array and the processing resource, wherein the control circuit is configured to: transmit unprocessed data from the memory array to the input / output circuit in response to a first access command without a request for preprocessing of data; cause the processing resource to perform operations on data accessed from the memory array in response to a second access command with a request for preprocessing of data; store preprocessed data in latches connected to the processing resource; and transmit the preprocessed data from the processing resource to the input / output circuit. Claim 2 In claim 1, the processing resource comprises a device including a complementary metal oxide semiconductor below the memory array configured as a plurality of logic blocks. Claim 3 In claim 1, the processing resource comprises a chip on the device separately from the memory array. Claim 4 In claim 1, the memory array is a device comprising a dynamic random access memory array. Claim 5 A device according to claim 1, wherein the control circuit configured to cause the processing resource to perform the operation is further configured to convert the data from one format to another format. Claim 6 In paragraph 5, the control circuit configured to convert data is a device further configured to decompress data from YCC format to RGB format. Claim 7 A device according to any one of claims 1 to 6, wherein the access command includes a request to perform an operation requested by a host device or an accelerator. Claim 8 In paragraph 7, the above operation is a device that includes preprocessing of the data before providing the data to the host device or accelerator. Claim 9 In any one of paragraphs 1 to 6, the size of the data is a device including the bus width. Claim 10 A device according to any one of claims 1 to 6, wherein the size of the data includes a plurality of bus widths. Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images