Media type selection for image data

Abstract

The present disclosure describes systems, devices, and methods related to media type selection for image data. A memory system may include multiple types of memory media (e.g., volatile and / or non-volatile) and may write data to the memory media types. Image data input may be written (e.g., stored) in memory media having characteristics of a particular type. For example, the selection of the memory media may be based on one or more attributes of the image data. In an example, a method may include receiving image data from a first image sensor of a plurality of image sensors via a memory system including multiple memory media types, identifying one or more attributes of the image data, and writing the image data to a first memory media type of the plurality of memory media types based at least in part on the one or more attributes of the image data.

Description

Technical Field

[0001] The present disclosure relates generally to semiconductor memories and methods, and more particularly, to apparatus, systems, and methods related to media type selection for image data. Background Art

[0002] Memory devices are typically provided as internal semiconductor integrated circuits in computers or other electronic systems. There are many different types of memory, including volatile and non-volatile memory. Volatile memory may require power to maintain its data (e.g., host data, error data, etc.) and includes random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), synchronous dynamic random access memory (SDRAM), and thyristor random access memory (TRAM), among others. Non-volatile memory can provide permanent data by retaining stored data when power is not supplied, and can include NAND flash memory, NOR flash memory, and resistance variable memory such as phase change random access memory (PCRAM), resistance random access memory (RRAM), and magnetoresistive random access memory (MRAM) such as spin torque transfer random access memory (STTRAM), among others.

[0003] A memory device may be coupled to a host (e.g., a host computing device) to store data, commands, and / or instructions for use by the host during operation of a computer or electronic system. For example, during operation of a computing or other electronic system, data, commands, and / or instructions may be transferred between the host and the memory device. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 is a functional block diagram of a computing system including a device including a memory system according to various embodiments of the present disclosure.

[0005] Figure 2 is a functional block diagram of a computing system including multiple storage media types according to various embodiments of the present disclosure.

[0006] Figure 3 is a flowchart representing an example method for media type selection for image data according to various embodiments of the present disclosure. DETAILED DESCRIPTION

[0007] The present disclosure includes apparatus, systems, and methods related to selecting a storage media type for image data. Example methods include receiving image data from a first image sensor among a plurality of image sensors (e.g., a camera) via a memory system including a plurality of storage media types, identifying one or more attributes of the image data (e.g., a picture), and writing the image data to a first storage media type among the plurality of storage media types based at least in part on the one or more attributes of the image data.

[0008] In some examples, one or more attributes of the image data may include pixel quality and / or density of the image data. Pixel quality may be based on the lens type and / or image data processing capabilities of the image sensor. For example, the density of the image data may be based on spot size, pixel size, and / or pixel depth.

[0009] In various embodiments, the multiple memory types may include DRAM, storage class memory (SCM), and / or NAND. In some examples, image data may be written to DRAM in response to image data including a small diameter spot size, a high number of pixels, and / or a high number of bits, while image data having a large diameter spot size, a low number of pixels, and / or a low number of bits may be stored in SCM and / or NAND.

[0010] Selecting a storage media type for image data based on one or more attributes of the image data can make more efficient use of memory. Media types can have different characteristics, including volatility, non-volatility, power usage, read / write latency, footprint, resource usage, and / or cost. For example, high-density memory like DRAM can be expensive, and image data containing large spot sizes may not require high-density memory and can instead be effectively and efficiently stored in SCM and / or NAND.

[0011] As used herein, "a number of" something can refer to one or more of such things. For example, a number of memory devices can refer to one or more memory devices. A "plurality" of something means two or more than two. Additionally, as used herein, designators such as "N" (particularly with respect to element numbers in the drawings) indicate that a number of the particular feature so designated can be included with several embodiments of the present disclosure.

[0012] The figures herein follow a numbering convention in which the first digit or digits correspond to the figure number of the figure and the remaining digits identify an element or component in the figure. Similar elements or components between different figures may be identified by using similar digits. For example, reference number 102 may refer to Figure 1 Component "2" in the , and similar components can be found in Figure 2In some cases, multiple similar but functionally and / or structurally distinguishable elements or components in the same figure or in different figures may be referenced sequentially with the same element number (e.g., Figure 2 230-1, 230-2, and 230-N in FIG. ). As will be appreciated, elements shown in the various embodiments herein may be added, exchanged, and / or eliminated to provide multiple additional embodiments of the present disclosure. Additionally, the proportions and relative scales of the elements provided in the figures are intended to illustrate various embodiments of the present disclosure and are not intended to be limiting.

[0013] Figure 1 1 is a functional block diagram of a computing system 100 including an apparatus, including a memory system 104, according to various embodiments of the present disclosure. As used herein, "apparatus" may refer to, but is not limited to, any of a variety of structures or combinations of structures, such as a circuit or circuitry, one or more dies, one or more modules, one or more devices, or one or more systems. Memory system 104 may include a host interface 108, a controller 110, such as a processor, control circuitry, hardware, firmware, and / or software, and a plurality of storage media devices, each of which includes control circuitry.

[0014] Figure 1 A non-limiting example of multiple memory media types is illustrated in the form of DRAM 112 including control circuitry 113, SCM 114 including control circuitry 115, and NAND 116 including control circuitry 117. Although three memory media types (e.g., DRAM 112, SCM 114, and NAND 116) are illustrated, embodiments are not limited thereto, and there may be more or less than three memory media types. Furthermore, the memory media types are not limited to Figure 1 While only three are specifically illustrated in FIG1 (e.g., DRAM 112, SCM 114, and NAND 116), other types of volatile and / or non-volatile memory media are contemplated. In various embodiments, controller 110, memory media DRAM 112, SCM 114, and NAND 116, and / or host interface 108 may be physically located on a single die or within a single package, such as in a managed memory application. Furthermore, in various embodiments, memory, such as memory media DRAM 112, SCM 114, and NAND 116, may be included on a single memory system 104.

[0015] As in Figure 1, controller 110 can be coupled to host interface 108 and memory media DRAM 112, SCM 114, and NAND 116 via one or more channels and can be used to transfer data between memory system 104 and host 102 having host controller 109. Host interface 108 can be in the form of a standardized interface. For example, when memory system 104 is used for data storage in computing system 100, interface 108 can be a Serial Advanced Technology Bus Attached (SATA), Peripheral Component Interconnect Express (PCIe), or Universal Serial Bus (USB), a Double Data Rate (DDR) interface, and other connectors and interfaces. However, in general, interface 108 can provide an interface for passing control, address, data, and other signals between memory system 104 and host 102 having a compatible receiver for host interface 108.

[0016] Host 102 may be a host system, such as a personal laptop computer, a head-mounted display, a vehicle, a desktop computer, a digital camera, a mobile phone, an Internet of Things (IoT)-enabled device or a memory card reader, a graphics processing unit (e.g., a video card), and various other types of hosts. Host 102 may include a system motherboard and / or a backplane, and may include multiple memory access devices, such as multiple processing resources (e.g., one or more processors, microprocessors, or some other type of control circuitry). Those skilled in the art will appreciate that "processor" may refer to one or more processors, such as a parallel processing system, multiple coprocessors, etc. Host 102 may be coupled to host interface 108 of memory system 104 via communication channel 103.

[0017] As used herein, an "IoT-enabled device" may refer to a device embedded with electronics, software, sensors, actuators, and / or network connectivity that enables such a device to connect to a network and / or exchange data. Examples of IoT-enabled devices include mobile phones, smartphones, tablets, phablets, computing devices, implantable devices, vehicles, home appliances, smart home devices, monitoring devices, wearable devices, devices that support smart shopping systems, and other cyber-physical systems.

[0018] In some embodiments, host 102 may be responsible for executing an operating system for computing system 100, including memory system 104. Thus, in some embodiments, host 102 may be responsible for controlling the operation of memory system 104. For example, host 102 may execute instructions (e.g., in the form of an operating system) that manage the hardware of computing system 100, such as scheduling tasks, executing applications, controlling peripheral devices, etc.

[0019] The computing system 100 may include a separate integrated circuit or host 102, a memory system 104, a host interface 108, a controller 110, and / or memory media DRAM 112, SCM 114, and / or NAND 116 that may be on the same integrated circuit. For example, the computing system 100 may be a server system and / or a high performance computing (HPC) system and / or a portion thereof. Figure 1 The examples shown in the drawings illustrate a system having a Von Neumann architecture, but embodiments of the present disclosure may be implemented in a non-Von Neumann architecture, which may not include one or more components typically associated with a Von Neumann architecture (e.g., a CPU, an ALU, etc.).

[0020] Although not shown in the examples to avoid obscuring the present disclosure, Figure 1 Although not shown in FIG, the memory system 104 may be communicatively coupled (e.g., connected) to a sensor, which may be communicatively coupled to the host 102. The term "coupled" means connected directly or indirectly, and may include wireless connections unless otherwise specified. As used herein, the term "image sensor" refers to a device that can generate and transmit image data and / or receive image data. Some examples of image sensors may include camera devices, video devices, and other devices. The image sensor may transmit data for storage in the memory system 104. For example, the controller 110 may be coupled to multiple memory media types (e.g., memory media DRAM 112, SCM 114, and NAND 116) to receive data from multiple image sensors.

[0021] Controller 110 (and / or host controller 109) may receive image data multiple times from a single image sensor or from multiple image sensors. The image sensor may be a different type of camera and / or may be used with different types of lenses. The image sensor may be included in a camera or a detector. For example, the image sensor may be included in a visible light camera, a compact camera, a digital single-lens reflex camera (DSLR), or an infrared detector, among other types of cameras and detectors.

[0022] In various embodiments, the controller 110 may receive image data from one or more other devices. Figure 1 For example, the one or more other devices may include a memory device. The memory device may be included in the host 102, the memory system 104, and / or external to the host 102 and the memory system 104.

[0023] Controller 110 may identify information regarding one or more attributes of the image data. For example, controller 110 may identify the image sensor that emitted the image data and the pixel quality and / or density of the image data, including the type of image sensor and / or the type of lens used with the image sensor. In some examples, controller 110 may identify the polarization state of the image data and / or the wavelength of the image data. Controller 110 may select a storage media type from a plurality of storage media types (e.g., storage media DRAM 112, SCM 114, and NAND 116) based at least in part on the identified information regarding the one or more attributes and write the image data to the selected storage media type. Furthermore, the storage media types (e.g., storage media DRAM 112, SCM 114, and NAND 116) may be communicatively coupled to one another so that data can be transferred between the storage media.

[0024] Figure 2 is a functional block diagram of a computing system 201 including multiple storage media types 212 , 214 , and 216 in accordance with various embodiments of the present disclosure.

[0025] The computing system 201 may include a plurality of image sensors 230-1, 230-2, ..., 230-N and a host 202 including a host controller 209 (which may be similar to a host controller 209). Figure 1 Although not shown in the example to avoid obstructing the present disclosure, the host 102 and host controller 109 are described. Figure 2 , but the computing system 201 may include a controller (e.g., in conjunction with Figure 1 Controller 110 as described above).

[0026] Host 202 may be communicatively coupled to image sensor 230 via a physical connection (e.g., via wiring, circuitry, etc.) or remote coupling (e.g., via wireless signals, near field communication, Bluetooth, Bluetooth Low Energy, RFID, etc.). Host 202 may be communicatively coupled to one or more memory media types 212, 214, and 216. Figure 2 Non-limiting examples of multiple memory media types are illustrated in the form of DRAM 212 including control circuitry 213 , SCM 214 including control circuitry 215 , and NAND 216 including control circuitry 217 .

[0027] Figure 2The embodiment described in

[0045] describes an example in which an image sensor 230 transmits data to a host 202 having a host controller 209. The host controller 209 may receive data from at least one of the image sensors 209, identify one or more attributes about the image data, and select one or more types of memory media (e.g., DRAM 212, SCM 214, and / or NAND 216) to write the image data based on the identified attributes. For example, the host controller 209 may receive image data from a first image sensor 230-1 among a plurality of image sensors 230, identify information about one or more attributes of the image data from the first image sensor 230-1, and select one or more types of memory media (e.g., DRAM 212, SCM 214, and / or NAND 216) to write the image data.

[0028] Attributes of the image data received from an image sensor in image sensor 230 may include pixel quality and / or density of the image data. Pixel quality may be based on lens type and / or image data processing capabilities.

[0029] Lens types can have different focal lengths. For example, a wide-angle lens may have a focal length of 24 mm to 35 mm for capturing interiors, architecture, and landscapes, while a short telephoto lens may have a focal length of 85 mm to 135 mm for capturing portraits. In some examples, image data from a lens with a high focal length may require a higher-density storage media type than image data from a lens with a low focal length. For example, image data from a wide-angle lens may be stored in NAND 216 or SCM 214, while image data from a short telephoto lens may be stored in DRAM 212.

[0030] Image data processing performance can be used to determine pixel quality. Image data processing performance can be based on the colors in the image data. Image data with high frequencies and multiple colors may require high-density memory, while image data with low frequencies and multiple colors may require low-density memory. For example, image data containing 1,000 different colors may be written to a high-density memory media type (e.g., DRAM 212), while image data containing 100 different colors may be written to a low-density memory media type (e.g., NAND 216 and / or SCM 214).

[0031] The density of the image data may be based on the spot size, pixel size, and / or pixel depth. Image data with a small diameter spot size, a high number of pixels, and / or a high number of bits may require a high-density storage media type, while image data with a large diameter spot size, a low number of pixels, and / or a low number of bits may be stored in a low-density storage media type. In some examples, image data with a high density may be written to a first storage media type, and image data with a low density may be written to a second storage media type. The first storage media type may be DRAM 212 for faster storage and access of high-density image data, and the second storage media type may be NAND 216 or SCM 214 because low-density image data can be stored and accessed from NAND 216 or SCM 214 at sufficient speeds.

[0032] An image processor 211, which may be included in the host 202, may determine the density of the image data using an algorithm 218. The algorithm 218 may receive the image data, calculate the density of the image data using a spot size, a pixel size, and / or a pixel depth of the image data, and select one or more types of storage media to write the image data to based on the calculated image density.

[0033] Algorithm 218 may determine whether the image data is high-density or low-density based on a threshold density. For example, image data may be high-density in response to being at or above a threshold density, while image data may be low-density in response to being below a threshold density. Depending on the amount of available storage in each of the memory media types, algorithm 218 may move the threshold density higher or lower. For example, if the amount of available storage in DRAM 212 is low, algorithm 218 may increase the threshold density, and if the amount of available storage in DRAM 212 is high, algorithm 218 may decrease the threshold density.

[0034] In various embodiments, host controller 209 may store reference information about each of image sensors 230. The stored reference information may be used to identify information about one or more attributes of the image data. The stored reference information may include the type of lens used with each of image sensors 230 and / or the image sensor type of each of image sensors 230. In some examples, the type of storage media may be selected in response to the stored reference information for the image sensors. For example, image data from the image sensors may be stored in volatile or non-volatile memory based on the stored reference information.

[0035] Figure 3332 is a flowchart illustrating an example method for selecting a media type for image data according to various embodiments of the present disclosure. At block 334, method 332 may include receiving image data from a first image sensor of a plurality of image sensors via a memory system comprising a plurality of media types. The memory system may include a host that can receive image data from the plurality of image sensors. Each of the plurality of image sensors may be used with a lens.

[0036] At block 336, method 332 may include identifying one or more attributes of the image data. The one or more attributes of the image may include pixel quality of the image data and / or density of the image data. The pixel quality of the image data may be based on lens type and / or image data processing performance. The density of the image data may be based on spot size, pixel size, and / or pixel depth.

[0037] At block 338, method 332 may include writing the image data to a first memory type of a plurality of memory types based at least in part on one or more attributes of the image data. The plurality of memory types may include, but are not limited to, DRAM, NAND, and SCM. In some examples, the image data may be written to DRAM in response to image data having a small diameter spot size, a high number of pixels, and / or a high number of bits, while image data having a large diameter spot size, a low number of pixels, and / or a low number of bits may be written to SCM and / or NAND.

[0038] Although specific embodiments have been illustrated and described herein, it will be understood by those skilled in the art that arrangements intended to achieve the same results may be substituted for the specific embodiments shown. The present disclosure is intended to cover modifications or variations of one or more embodiments of the present disclosure. It should be understood that the above description has been made in an illustrative manner and not in a restrictive manner. After reviewing the above description, it will be immediately clear to those skilled in the art that combinations of the above embodiments and other embodiments not specifically described herein are possible. The scope of one or more embodiments of the present disclosure includes other applications using the above structures and methods. Therefore, the scope of one or more embodiments of the present disclosure should be determined with reference to the appended claims together with the entire scope of equivalents authorized by such claims.

[0039] In the foregoing Detailed Description, some features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the disclosed embodiments of the disclosure must use more features than are expressly recited in each claim. Rather, as reflected in the appended claims, inventive subject matter lies in less than all features of a single disclosed embodiment. The appended claims are therefore hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.

Claims

1. A method for selecting a media type for image data, comprising: receiving image data from a first image sensor of a plurality of image sensors via a memory system comprising a plurality of memory media types; identifying one or more attributes of the image data, wherein the one or more attributes include at least one of: a spot size, a pixel size, or a pixel depth of the image data; and Based at least in part on the one or more attributes of the image data, the image data is written to a first memory media type among the plurality of memory media types, wherein in response to at least one of: the spot size being below a first threshold diameter, the pixel depth being above a first threshold number of bits, or the pixel size being above a first threshold number of pixels, the first memory media type is dynamic random access memory (DRAM), and wherein in response to the spot size being above a second threshold diameter, the pixel depth being below a second threshold number of bits, and the pixel size being below a second threshold number of pixels, the first memory media type is NAND. 2 . The method of claim 1 , comprising identifying the one or more attributes of the image data, including at least one of: pixel quality or density of the image data. 3 . The method of claim 2 , comprising determining the pixel quality of the image data based on at least one of: lens type or image data processing performance. 4 . The method of claim 2 , comprising determining the density of the image data based on at least one of: the spot size, the pixel size, or the pixel depth.

5. The method of claim 2, comprising writing the image data to the first memory media type in response to the density of the image data being a high density, wherein the first memory media type is the DRAM.

6. The method of claim 2, comprising writing the image data to the first memory media type in response to the density of the image data being a low density, wherein the first memory media type is the NAND.

7. The method of claim 3, comprising determining the pixel quality of the image data based on the image data processing performance comprises analyzing colors in the image data.

8. A device for selecting a media type of image data, comprising: Multiple storage media types; and a controller coupled to the plurality of storage media types, wherein the controller is configured to: receiving image data from a first image sensor of the plurality of image sensors; identifying one or more attributes of the image data, wherein the one or more attributes include at least one of: a spot size, a pixel size, or a pixel depth of the image data; selecting a storage media type from the plurality of storage media types based at least in part on the one or more attributes; and The image data is written to a selected memory medium type, wherein in response to at least one of: the spot size being below a first threshold diameter, the pixel depth being above a first threshold number of bits, or the pixel size being above a first threshold number of pixels, the selected memory medium type is dynamic random access memory (DRAM), and wherein in response to the spot size being above a second threshold diameter, the pixel depth being below a second threshold number of bits, and the pixel size being below a second threshold number of pixels, the selected memory medium type is NAND.

9. The apparatus of claim 8, wherein the plurality of memory media types comprises at least one of: the DRAM, storage class memory (SCM), or the NAND.

10. The apparatus of claim 8, wherein the controller is configured to select a storage class memory (SCM) and write the image data to the SCM in response to the spot size being above the first threshold diameter, the pixel depth being below the first threshold number of bits, and the pixel size being below the first threshold number of pixels, and at least one of: the spot size being below the second threshold diameter, the pixel depth being above the second threshold number of bits, or the pixel size being above the second threshold number of pixels.

11. A system for media type selection of image data, comprising: multiple image sensors; a plurality of storage media types coupled to the plurality of image sensors; and a controller coupled to the plurality of storage media types, wherein the controller is configured to: storing reference information about each of the plurality of image sensors; receiving image data from a first image sensor of the plurality of image sensors; identifying one or more attributes of the image data, wherein the one or more attributes include at least one of: a spot size, a pixel size, or a pixel depth of the image data; selecting a storage media type from the plurality of storage media types based at least in part on the one or more attributes; and The image data is written to the selected memory medium type, wherein in response to at least one of: the spot size being below a first threshold diameter, the pixel depth being above a first threshold number of bits, or the pixel size being above a first threshold number of pixels, the selected memory medium type is dynamic random access memory (DRAM), and wherein in response to the spot size being above a second threshold diameter, the pixel depth being below a second threshold number of bits, and the pixel size being below a second threshold number of pixels, the selected memory medium type is NAND.

12. The system of claim 11, wherein the controller is further configured to use the stored reference information to identify the one or more attributes of the image data.

13. The system of claim 11, wherein the stored reference information includes at least one of a lens type or an image sensor type.

14. The system of any one of claims 11 to 13, wherein the selected memory media type is volatile memory responsive to the stored reference information of the first image sensor.

15. The system of any one of claims 11 to 13, wherein the selected memory media type is non-volatile responsive to the stored reference information of the first image sensor.

16. The system of any one of claims 11 to 13, wherein a lens is used with each of the plurality of image sensors.

17. The system of claim 16, wherein the image data includes a lens type of the lens used with the first image sensor.

Images